Note: This is test shard 1 of 3.
[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/0
2026-02-02T02:52:09Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-02-02T02:52:09Z Disabled control of system clock
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260202 02:52:09.806000 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskYXxklX/build/asan/bin/kudu
/tmp/dist-test-taskYXxklX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.190:33981
--webserver_interface=127.0.104.190
--webserver_port=0
--builtin_ntp_servers=127.0.104.148:45945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.190:33981,127.0.104.189:46271,127.0.104.188:39077
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260202 02:52:10.169732 426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260202 02:52:10.170111 426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260202 02:52:10.170187 426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260202 02:52:10.179653 426 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260202 02:52:10.179773 426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260202 02:52:10.179833 426 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260202 02:52:10.179872 426 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260202 02:52:10.190068 426 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:45945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-0/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.190:33981,127.0.104.189:46271,127.0.104.188:39077
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.190:33981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.104.190
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3722ed49c19ce54c7feae2fa944e32b1d4ed8d2d
build type FASTDEBUG
built by None at 02 Feb 2026 02:43:23 UTC on 24a791456cd2
build id 10304
ASAN enabled
I20260202 02:52:10.191735 426 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260202 02:52:10.193733 426 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260202 02:52:10.203123 434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:52:10.203125 431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:52:10.204581 432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260202 02:52:10.204825 426 server_base.cc:1060] running on GCE node
I20260202 02:52:10.205701 426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260202 02:52:10.207352 426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260202 02:52:10.208580 426 hybrid_clock.cc:648] HybridClock initialized: now 1770000730208494 us; error 85 us; skew 500 ppm
I20260202 02:52:10.209017 426 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260202 02:52:10.212119 426 webserver.cc:492] Webserver started at http://127.0.104.190:40497/ using document root <none> and password file <none>
I20260202 02:52:10.212999 426 fs_manager.cc:362] Metadata directory not provided
I20260202 02:52:10.213122 426 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260202 02:52:10.213490 426 server_base.cc:908] This appears to be a new deployment of Kudu; creating new FS layout
I20260202 02:52:10.216002 426 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-0/data/instance:
uuid: "7dab2abef14445d688ce1335b1f9b2f8"
format_stamp: "Formatted at 2026-02-02 02:52:10 on dist-test-slave-llj2"
I20260202 02:52:10.216907 426 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-0/wal/instance:
uuid: "7dab2abef14445d688ce1335b1f9b2f8"
format_stamp: "Formatted at 2026-02-02 02:52:10 on dist-test-slave-llj2"
I20260202 02:52:10.222710 426 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260202 02:52:10.226630 440 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260202 02:52:10.228236 426 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260202 02:52:10.228490 426 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-0/data,/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-0/wal
uuid: "7dab2abef14445d688ce1335b1f9b2f8"
format_stamp: "Formatted at 2026-02-02 02:52:10 on dist-test-slave-llj2"
I20260202 02:52:10.228726 426 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260202 02:52:10.244318 426 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260202 02:52:10.245887 426 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260202 02:52:10.246378 426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260202 02:52:10.269498 426 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.190:33981
I20260202 02:52:10.269511 491 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.190:33981 every 8 connection(s)
I20260202 02:52:10.271190 426 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-0/data/info.pb
I20260202 02:52:10.275488 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskYXxklX/build/asan/bin/kudu as pid 426
I20260202 02:52:10.275830 418 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-0/wal/instance
I20260202 02:52:10.277999 492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:10.279353 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskYXxklX/build/asan/bin/kudu
/tmp/dist-test-taskYXxklX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.189:46271
--webserver_interface=127.0.104.189
--webserver_port=0
--builtin_ntp_servers=127.0.104.148:45945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.190:33981,127.0.104.189:46271,127.0.104.188:39077
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260202 02:52:10.286023 492 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } has no permanent_uuid. Determining permanent_uuid...
I20260202 02:52:10.308815 492 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } has no permanent_uuid. Determining permanent_uuid...
W20260202 02:52:10.310122 443 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.189:46271: connect: Connection refused (error 111)
W20260202 02:52:10.311749 492 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.189:46271: Network error: Client connection negotiation failed: client connection to 127.0.104.189:46271: connect: Connection refused (error 111)
I20260202 02:52:10.361127 492 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } attempt: 1
W20260202 02:52:10.363198 492 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.189:46271: Network error: Client connection negotiation failed: client connection to 127.0.104.189:46271: connect: Connection refused (error 111)
I20260202 02:52:10.431497 492 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } attempt: 2
W20260202 02:52:10.433554 492 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.189:46271: Network error: Client connection negotiation failed: client connection to 127.0.104.189:46271: connect: Connection refused (error 111)
I20260202 02:52:10.524863 492 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } attempt: 3
W20260202 02:52:10.527242 492 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.189:46271: Network error: Client connection negotiation failed: client connection to 127.0.104.189:46271: connect: Connection refused (error 111)
W20260202 02:52:10.649699 494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260202 02:52:10.650045 494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260202 02:52:10.650120 494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260202 02:52:10.659411 494 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260202 02:52:10.659518 494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260202 02:52:10.659569 494 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260202 02:52:10.659615 494 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260202 02:52:10.670256 494 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:45945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-1/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.190:33981,127.0.104.189:46271,127.0.104.188:39077
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.189:46271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.104.189
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3722ed49c19ce54c7feae2fa944e32b1d4ed8d2d
build type FASTDEBUG
built by None at 02 Feb 2026 02:43:23 UTC on 24a791456cd2
build id 10304
ASAN enabled
I20260202 02:52:10.670559 492 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } attempt: 4
I20260202 02:52:10.672065 494 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
W20260202 02:52:10.672552 492 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.189:46271: Network error: Client connection negotiation failed: client connection to 127.0.104.189:46271: connect: Connection refused (error 111)
I20260202 02:52:10.674013 494 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260202 02:52:10.683287 503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:52:10.683270 502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:52:10.683562 505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260202 02:52:10.684285 494 server_base.cc:1060] running on GCE node
I20260202 02:52:10.685581 494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260202 02:52:10.687796 494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260202 02:52:10.689141 494 hybrid_clock.cc:648] HybridClock initialized: now 1770000730689048 us; error 52 us; skew 500 ppm
I20260202 02:52:10.689771 494 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260202 02:52:10.693120 494 webserver.cc:492] Webserver started at http://127.0.104.189:41135/ using document root <none> and password file <none>
I20260202 02:52:10.694015 494 fs_manager.cc:362] Metadata directory not provided
I20260202 02:52:10.694175 494 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260202 02:52:10.694562 494 server_base.cc:908] This appears to be a new deployment of Kudu; creating new FS layout
I20260202 02:52:10.697203 494 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-1/data/instance:
uuid: "92a8191cf1e14e4d99f1c378c2d3c85b"
format_stamp: "Formatted at 2026-02-02 02:52:10 on dist-test-slave-llj2"
I20260202 02:52:10.698079 494 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-1/wal/instance:
uuid: "92a8191cf1e14e4d99f1c378c2d3c85b"
format_stamp: "Formatted at 2026-02-02 02:52:10 on dist-test-slave-llj2"
I20260202 02:52:10.703958 494 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260202 02:52:10.707711 511 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260202 02:52:10.709262 494 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260202 02:52:10.709455 494 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-1/data,/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-1/wal
uuid: "92a8191cf1e14e4d99f1c378c2d3c85b"
format_stamp: "Formatted at 2026-02-02 02:52:10 on dist-test-slave-llj2"
I20260202 02:52:10.709672 494 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260202 02:52:10.734864 494 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260202 02:52:10.735865 494 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260202 02:52:10.736148 494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260202 02:52:10.758574 494 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.189:46271
I20260202 02:52:10.758594 562 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.189:46271 every 8 connection(s)
I20260202 02:52:10.760130 494 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-1/data/info.pb
I20260202 02:52:10.764621 563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:10.769416 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskYXxklX/build/asan/bin/kudu as pid 494
I20260202 02:52:10.769614 418 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-1/wal/instance
I20260202 02:52:10.770632 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskYXxklX/build/asan/bin/kudu
/tmp/dist-test-taskYXxklX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.188:39077
--webserver_interface=127.0.104.188
--webserver_port=0
--builtin_ntp_servers=127.0.104.148:45945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.190:33981,127.0.104.189:46271,127.0.104.188:39077
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260202 02:52:10.770812 563 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } has no permanent_uuid. Determining permanent_uuid...
I20260202 02:52:10.786239 563 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } has no permanent_uuid. Determining permanent_uuid...
I20260202 02:52:10.796957 563 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } has no permanent_uuid. Determining permanent_uuid...
W20260202 02:52:10.798566 513 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.188:39077: connect: Connection refused (error 111)
W20260202 02:52:10.800316 563 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.188:39077: Network error: Client connection negotiation failed: client connection to 127.0.104.188:39077: connect: Connection refused (error 111)
I20260202 02:52:10.849707 563 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } attempt: 1
W20260202 02:52:10.851616 563 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.188:39077: Network error: Client connection negotiation failed: client connection to 127.0.104.188:39077: connect: Connection refused (error 111)
I20260202 02:52:10.919870 563 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } attempt: 2
W20260202 02:52:10.922078 563 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.188:39077: Network error: Client connection negotiation failed: client connection to 127.0.104.188:39077: connect: Connection refused (error 111)
I20260202 02:52:10.971851 492 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } attempt: 5
I20260202 02:52:10.979799 492 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } has no permanent_uuid. Determining permanent_uuid...
W20260202 02:52:10.981958 492 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.188:39077: Network error: Client connection negotiation failed: client connection to 127.0.104.188:39077: connect: Connection refused (error 111)
I20260202 02:52:11.013334 563 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } attempt: 3
W20260202 02:52:11.015403 563 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.188:39077: Network error: Client connection negotiation failed: client connection to 127.0.104.188:39077: connect: Connection refused (error 111)
I20260202 02:52:11.033213 492 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } attempt: 1
W20260202 02:52:11.035136 492 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.188:39077: Network error: Client connection negotiation failed: client connection to 127.0.104.188:39077: connect: Connection refused (error 111)
I20260202 02:52:11.103376 492 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } attempt: 2
W20260202 02:52:11.105317 492 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.188:39077: Network error: Client connection negotiation failed: client connection to 127.0.104.188:39077: connect: Connection refused (error 111)
W20260202 02:52:11.142493 565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260202 02:52:11.142848 565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260202 02:52:11.142943 565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260202 02:52:11.151343 565 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260202 02:52:11.151461 565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260202 02:52:11.151515 565 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260202 02:52:11.151564 565 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260202 02:52:11.158681 563 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } attempt: 4
W20260202 02:52:11.160526 563 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.188:39077: Network error: Client connection negotiation failed: client connection to 127.0.104.188:39077: connect: Connection refused (error 111)
I20260202 02:52:11.162663 565 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:45945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-2/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.190:33981,127.0.104.189:46271,127.0.104.188:39077
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.188:39077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.104.188
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3722ed49c19ce54c7feae2fa944e32b1d4ed8d2d
build type FASTDEBUG
built by None at 02 Feb 2026 02:43:23 UTC on 24a791456cd2
build id 10304
ASAN enabled
I20260202 02:52:11.164283 565 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260202 02:52:11.166217 565 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260202 02:52:11.174989 573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:52:11.175710 576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:52:11.174940 574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260202 02:52:11.176872 565 server_base.cc:1060] running on GCE node
I20260202 02:52:11.177664 565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260202 02:52:11.179224 565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260202 02:52:11.180437 565 hybrid_clock.cc:648] HybridClock initialized: now 1770000731180387 us; error 46 us; skew 500 ppm
I20260202 02:52:11.180857 565 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260202 02:52:11.183280 565 webserver.cc:492] Webserver started at http://127.0.104.188:35523/ using document root <none> and password file <none>
I20260202 02:52:11.184113 565 fs_manager.cc:362] Metadata directory not provided
I20260202 02:52:11.184233 565 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260202 02:52:11.184639 565 server_base.cc:908] This appears to be a new deployment of Kudu; creating new FS layout
I20260202 02:52:11.187364 565 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-2/data/instance:
uuid: "a9c3e3612bb94fb7b94c308041acfaeb"
format_stamp: "Formatted at 2026-02-02 02:52:11 on dist-test-slave-llj2"
I20260202 02:52:11.188236 565 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-2/wal/instance:
uuid: "a9c3e3612bb94fb7b94c308041acfaeb"
format_stamp: "Formatted at 2026-02-02 02:52:11 on dist-test-slave-llj2"
I20260202 02:52:11.194291 565 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260202 02:52:11.198609 582 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260202 02:52:11.200264 565 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260202 02:52:11.200505 565 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-2/data,/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-2/wal
uuid: "a9c3e3612bb94fb7b94c308041acfaeb"
format_stamp: "Formatted at 2026-02-02 02:52:11 on dist-test-slave-llj2"
I20260202 02:52:11.200734 565 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260202 02:52:11.211570 492 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } attempt: 3
W20260202 02:52:11.213711 492 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.188:39077: Network error: Client connection negotiation failed: client connection to 127.0.104.188:39077: connect: Connection refused (error 111)
I20260202 02:52:11.222225 565 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260202 02:52:11.223192 565 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260202 02:52:11.223455 565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260202 02:52:11.246372 565 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.188:39077
I20260202 02:52:11.246388 633 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.188:39077 every 8 connection(s)
I20260202 02:52:11.247939 565 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-2/data/info.pb
I20260202 02:52:11.249264 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskYXxklX/build/asan/bin/kudu as pid 565
I20260202 02:52:11.249449 418 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-2/wal/instance
I20260202 02:52:11.253750 634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:11.260628 634 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } has no permanent_uuid. Determining permanent_uuid...
I20260202 02:52:11.276635 634 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } has no permanent_uuid. Determining permanent_uuid...
I20260202 02:52:11.286900 634 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } has no permanent_uuid. Determining permanent_uuid...
I20260202 02:52:11.302933 634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb: Bootstrap starting.
I20260202 02:52:11.306237 634 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:11.307976 634 log.cc:826] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb: Log is configured to *not* fsync() on all Append() calls
I20260202 02:52:11.311223 634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb: No bootstrap required, opened a new log
I20260202 02:52:11.316262 634 raft_consensus.cc:359] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } }
I20260202 02:52:11.316632 634 raft_consensus.cc:385] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:11.316752 634 raft_consensus.cc:740] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c3e3612bb94fb7b94c308041acfaeb, State: Initialized, Role: FOLLOWER
I20260202 02:52:11.317435 634 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } }
I20260202 02:52:11.318471 643 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } } }
I20260202 02:52:11.318719 643 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:52:11.319528 634 sys_catalog.cc:565] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [sys.catalog]: configured and running, proceeding with master startup.
W20260202 02:52:11.330111 654 catalog_manager.cc:1568] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:52:11.330269 654 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260202 02:52:11.327991 565 master_runner.cc:186] Error getting master registration for 127.0.104.190:33981: OK, instance_id { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" instance_seqno: 1770000730262183 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260202 02:52:11.334015 565 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:52:11.390974 492 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } attempt: 4
I20260202 02:52:11.403981 492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8: Bootstrap starting.
I20260202 02:52:11.407354 492 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:11.408530 492 log.cc:826] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8: Log is configured to *not* fsync() on all Append() calls
I20260202 02:52:11.411979 492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8: No bootstrap required, opened a new log
I20260202 02:52:11.418191 492 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } }
I20260202 02:52:11.418525 492 raft_consensus.cc:385] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:11.418648 492 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dab2abef14445d688ce1335b1f9b2f8, State: Initialized, Role: FOLLOWER
I20260202 02:52:11.419288 492 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } }
I20260202 02:52:11.420290 657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } } }
I20260202 02:52:11.420645 657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:52:11.421543 492 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [sys.catalog]: configured and running, proceeding with master startup.
W20260202 02:52:11.432725 668 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:52:11.432883 668 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260202 02:52:11.433730 426 master_runner.cc:186] Error getting master registration for 127.0.104.189:46271: OK, instance_id { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" instance_seqno: 1770000730751952 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260202 02:52:11.440443 426 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:52:11.459838 563 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } attempt: 5
I20260202 02:52:11.474146 563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Bootstrap starting.
W20260202 02:52:11.475769 588 tablet.cc:2378] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260202 02:52:11.478636 563 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:11.480108 563 log.cc:826] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Log is configured to *not* fsync() on all Append() calls
I20260202 02:52:11.483929 563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: No bootstrap required, opened a new log
I20260202 02:52:11.493181 563 raft_consensus.cc:359] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } }
I20260202 02:52:11.493577 563 raft_consensus.cc:385] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:11.493703 563 raft_consensus.cc:740] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92a8191cf1e14e4d99f1c378c2d3c85b, State: Initialized, Role: FOLLOWER
I20260202 02:52:11.494544 563 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } }
I20260202 02:52:11.495690 671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } } }
I20260202 02:52:11.496083 671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:52:11.497191 563 sys_catalog.cc:565] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [sys.catalog]: configured and running, proceeding with master startup.
W20260202 02:52:11.499631 446 tablet.cc:2378] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260202 02:52:11.515672 494 master_runner.cc:263] No leader master found from master 92a8191cf1e14e4d99f1c378c2d3c85b
I20260202 02:52:11.515865 494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:52:11.519353 682 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:52:11.519645 682 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260202 02:52:11.610278 657 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:11.610570 657 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } }
I20260202 02:52:11.613668 609 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7dab2abef14445d688ce1335b1f9b2f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" is_pre_election: true
I20260202 02:52:11.614284 609 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dab2abef14445d688ce1335b1f9b2f8 in term 0.
I20260202 02:52:11.615255 442 leader_election.cc:304] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dab2abef14445d688ce1335b1f9b2f8, a9c3e3612bb94fb7b94c308041acfaeb; no voters:
I20260202 02:52:11.616719 538 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7dab2abef14445d688ce1335b1f9b2f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" is_pre_election: true
I20260202 02:52:11.617308 538 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dab2abef14445d688ce1335b1f9b2f8 in term 0.
I20260202 02:52:11.618136 657 leader_election.cc:290] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92a8191cf1e14e4d99f1c378c2d3c85b (127.0.104.189:46271), a9c3e3612bb94fb7b94c308041acfaeb (127.0.104.188:39077)
I20260202 02:52:11.618369 657 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:11.618492 657 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:11.618575 657 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:11.623747 657 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } }
I20260202 02:52:11.626291 538 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7dab2abef14445d688ce1335b1f9b2f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b"
I20260202 02:52:11.626543 538 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:11.629700 657 leader_election.cc:290] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [CANDIDATE]: Term 1 election: Requested vote from peers 92a8191cf1e14e4d99f1c378c2d3c85b (127.0.104.189:46271), a9c3e3612bb94fb7b94c308041acfaeb (127.0.104.188:39077)
I20260202 02:52:11.633350 538 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dab2abef14445d688ce1335b1f9b2f8 in term 1.
I20260202 02:52:11.636134 443 leader_election.cc:304] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dab2abef14445d688ce1335b1f9b2f8, 92a8191cf1e14e4d99f1c378c2d3c85b; no voters:
I20260202 02:52:11.637796 657 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:11.638273 657 raft_consensus.cc:697] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [term 1 LEADER]: Becoming Leader. State: Replica: 7dab2abef14445d688ce1335b1f9b2f8, State: Running, Role: LEADER
I20260202 02:52:11.639040 657 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } }
I20260202 02:52:11.650094 609 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7dab2abef14445d688ce1335b1f9b2f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c3e3612bb94fb7b94c308041acfaeb"
I20260202 02:52:11.650370 609 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:11.654160 609 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dab2abef14445d688ce1335b1f9b2f8 in term 1.
I20260202 02:52:11.656118 684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7dab2abef14445d688ce1335b1f9b2f8. Latest consensus state: current_term: 1 leader_uuid: "7dab2abef14445d688ce1335b1f9b2f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } } }
I20260202 02:52:11.656612 684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [sys.catalog]: This master's current role is: LEADER
I20260202 02:52:11.669379 686 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260202 02:52:11.672446 686 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260202 02:52:11.688723 609 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [term 1 FOLLOWER]: Refusing update from remote peer 7dab2abef14445d688ce1335b1f9b2f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260202 02:52:11.690021 684 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:11.697644 538 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [term 1 FOLLOWER]: Refusing update from remote peer 7dab2abef14445d688ce1335b1f9b2f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260202 02:52:11.699103 684 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260202 02:52:11.727176 643 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7dab2abef14445d688ce1335b1f9b2f8. Latest consensus state: current_term: 1 leader_uuid: "7dab2abef14445d688ce1335b1f9b2f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } } }
I20260202 02:52:11.727486 643 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:52:11.729844 687 mvcc.cc:204] Tried to move back new op lower bound from 7249922996983160832 to 7249922996807819264. Current Snapshot: MvccSnapshot[applied={T|T < 7249922996983160832}]
I20260202 02:52:11.730352 684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7dab2abef14445d688ce1335b1f9b2f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } } }
I20260202 02:52:11.730571 684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [sys.catalog]: This master's current role is: LEADER
I20260202 02:52:11.731951 688 mvcc.cc:204] Tried to move back new op lower bound from 7249922996983160832 to 7249922996807819264. Current Snapshot: MvccSnapshot[applied={T|T < 7249922996983160832}]
I20260202 02:52:11.732842 671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7dab2abef14445d688ce1335b1f9b2f8. Latest consensus state: current_term: 1 leader_uuid: "7dab2abef14445d688ce1335b1f9b2f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } } }
I20260202 02:52:11.733084 671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:52:11.733392 684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7dab2abef14445d688ce1335b1f9b2f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } } }
I20260202 02:52:11.733595 684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [sys.catalog]: This master's current role is: LEADER
I20260202 02:52:11.734925 686 catalog_manager.cc:1357] Generated new cluster ID: bdfd876443214aa8b701e616e981c0d4
I20260202 02:52:11.735035 686 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260202 02:52:11.735396 689 mvcc.cc:204] Tried to move back new op lower bound from 7249922996983160832 to 7249922996807819264. Current Snapshot: MvccSnapshot[applied={T|T < 7249922996983160832}]
I20260202 02:52:11.751283 643 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7dab2abef14445d688ce1335b1f9b2f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } } }
I20260202 02:52:11.751543 643 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:52:11.752295 671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7dab2abef14445d688ce1335b1f9b2f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } } }
I20260202 02:52:11.752527 671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:52:11.777228 686 catalog_manager.cc:1380] Generated new certificate authority record
I20260202 02:52:11.779306 686 catalog_manager.cc:1514] Loading token signing keys...
I20260202 02:52:11.807693 686 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8: Generated new TSK 0
I20260202 02:52:11.808892 686 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260202 02:52:11.827382 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskYXxklX/build/asan/bin/kudu
/tmp/dist-test-taskYXxklX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.104.129:0
--local_ip_for_outbound_sockets=127.0.104.129
--webserver_interface=127.0.104.129
--webserver_port=0
--tserver_master_addrs=127.0.104.190:33981,127.0.104.189:46271,127.0.104.188:39077
--builtin_ntp_servers=127.0.104.148:45945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
W20260202 02:52:12.275844 696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260202 02:52:12.276543 696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260202 02:52:12.276708 696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260202 02:52:12.286335 696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260202 02:52:12.286549 696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.104.129
I20260202 02:52:12.297137 696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:45945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.104.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.104.129
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.0.104.190:33981,127.0.104.189:46271,127.0.104.188:39077
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.104.129
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3722ed49c19ce54c7feae2fa944e32b1d4ed8d2d
build type FASTDEBUG
built by None at 02 Feb 2026 02:43:23 UTC on 24a791456cd2
build id 10304
ASAN enabled
I20260202 02:52:12.298998 696 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260202 02:52:12.301138 696 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260202 02:52:12.311281 702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:52:12.311323 701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:52:12.311708 704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260202 02:52:12.312048 706 raft_consensus.cc:493] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7dab2abef14445d688ce1335b1f9b2f8)
I20260202 02:52:12.312206 706 raft_consensus.cc:515] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } }
I20260202 02:52:12.312644 696 server_base.cc:1060] running on GCE node
I20260202 02:52:12.313262 696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260202 02:52:12.313616 706 leader_election.cc:290] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7dab2abef14445d688ce1335b1f9b2f8 (127.0.104.190:33981), a9c3e3612bb94fb7b94c308041acfaeb (127.0.104.188:39077)
I20260202 02:52:12.313963 608 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" is_pre_election: true
I20260202 02:52:12.314118 467 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7dab2abef14445d688ce1335b1f9b2f8" is_pre_election: true
I20260202 02:52:12.314536 696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260202 02:52:12.314931 515 leader_election.cc:304] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92a8191cf1e14e4d99f1c378c2d3c85b; no voters: 7dab2abef14445d688ce1335b1f9b2f8, a9c3e3612bb94fb7b94c308041acfaeb
I20260202 02:52:12.315304 706 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260202 02:52:12.315768 696 hybrid_clock.cc:648] HybridClock initialized: now 1770000732315673 us; error 84 us; skew 500 ppm
I20260202 02:52:12.316121 696 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260202 02:52:12.318621 696 webserver.cc:492] Webserver started at http://127.0.104.129:42115/ using document root <none> and password file <none>
I20260202 02:52:12.319450 696 fs_manager.cc:362] Metadata directory not provided
I20260202 02:52:12.319569 696 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260202 02:52:12.319945 696 server_base.cc:908] This appears to be a new deployment of Kudu; creating new FS layout
I20260202 02:52:12.323601 696 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/data/instance:
uuid: "1f442ef85029461091de847118c34c0d"
format_stamp: "Formatted at 2026-02-02 02:52:12 on dist-test-slave-llj2"
I20260202 02:52:12.324609 696 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/instance:
uuid: "1f442ef85029461091de847118c34c0d"
format_stamp: "Formatted at 2026-02-02 02:52:12 on dist-test-slave-llj2"
I20260202 02:52:12.330616 696 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260202 02:52:12.332144 654 catalog_manager.cc:1269] Loaded cluster ID: bdfd876443214aa8b701e616e981c0d4
I20260202 02:52:12.332231 654 catalog_manager.cc:1562] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb: loading cluster ID for follower catalog manager: success
I20260202 02:52:12.334867 711 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260202 02:52:12.335899 654 catalog_manager.cc:1584] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb: acquiring CA information for follower catalog manager: success
I20260202 02:52:12.336942 696 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260202 02:52:12.337131 696 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal
uuid: "1f442ef85029461091de847118c34c0d"
format_stamp: "Formatted at 2026-02-02 02:52:12 on dist-test-slave-llj2"
I20260202 02:52:12.337357 696 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260202 02:52:12.338140 654 catalog_manager.cc:1612] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260202 02:52:12.358328 696 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260202 02:52:12.359251 696 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260202 02:52:12.359516 696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260202 02:52:12.360579 696 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260202 02:52:12.362417 696 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260202 02:52:12.362529 696 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260202 02:52:12.362651 696 ts_tablet_manager.cc:616] Registered 0 tablets
I20260202 02:52:12.362730 696 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260202 02:52:12.400115 696 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.129:39403
I20260202 02:52:12.400214 823 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.129:39403 every 8 connection(s)
I20260202 02:52:12.401674 696 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/data/info.pb
I20260202 02:52:12.411876 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskYXxklX/build/asan/bin/kudu as pid 696
I20260202 02:52:12.412124 418 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/instance
I20260202 02:52:12.433518 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskYXxklX/build/asan/bin/kudu
/tmp/dist-test-taskYXxklX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.104.130:0
--local_ip_for_outbound_sockets=127.0.104.130
--webserver_interface=127.0.104.130
--webserver_port=0
--tserver_master_addrs=127.0.104.190:33981,127.0.104.189:46271,127.0.104.188:39077
--builtin_ntp_servers=127.0.104.148:45945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260202 02:52:12.441314 825 heartbeater.cc:344] Connected to a master server at 127.0.104.190:33981
I20260202 02:52:12.441493 826 heartbeater.cc:344] Connected to a master server at 127.0.104.189:46271
I20260202 02:52:12.441849 826 heartbeater.cc:461] Registering TS with master...
I20260202 02:52:12.441847 825 heartbeater.cc:461] Registering TS with master...
I20260202 02:52:12.443171 825 heartbeater.cc:507] Master 127.0.104.190:33981 requested a full tablet report, sending...
I20260202 02:52:12.443662 826 heartbeater.cc:507] Master 127.0.104.189:46271 requested a full tablet report, sending...
I20260202 02:52:12.445978 457 ts_manager.cc:194] Registered new tserver with Master: 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:12.445950 527 ts_manager.cc:194] Registered new tserver with Master: 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:12.448016 457 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.104.129:33747
I20260202 02:52:12.448407 824 heartbeater.cc:344] Connected to a master server at 127.0.104.188:39077
I20260202 02:52:12.448563 824 heartbeater.cc:461] Registering TS with master...
I20260202 02:52:12.449156 824 heartbeater.cc:507] Master 127.0.104.188:39077 requested a full tablet report, sending...
I20260202 02:52:12.451195 599 ts_manager.cc:194] Registered new tserver with Master: 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:12.521768 682 catalog_manager.cc:1269] Loaded cluster ID: bdfd876443214aa8b701e616e981c0d4
I20260202 02:52:12.521881 682 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: loading cluster ID for follower catalog manager: success
I20260202 02:52:12.524796 682 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: acquiring CA information for follower catalog manager: success
I20260202 02:52:12.526660 682 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260202 02:52:12.798040 834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260202 02:52:12.798472 834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260202 02:52:12.798600 834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260202 02:52:12.801105 838 raft_consensus.cc:493] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7dab2abef14445d688ce1335b1f9b2f8)
I20260202 02:52:12.801308 838 raft_consensus.cc:515] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } }
I20260202 02:52:12.802649 838 leader_election.cc:290] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7dab2abef14445d688ce1335b1f9b2f8 (127.0.104.190:33981), 92a8191cf1e14e4d99f1c378c2d3c85b (127.0.104.189:46271)
I20260202 02:52:12.803247 467 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7dab2abef14445d688ce1335b1f9b2f8" is_pre_election: true
I20260202 02:52:12.803252 538 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" is_pre_election: true
I20260202 02:52:12.804032 585 leader_election.cc:304] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9c3e3612bb94fb7b94c308041acfaeb; no voters: 7dab2abef14445d688ce1335b1f9b2f8, 92a8191cf1e14e4d99f1c378c2d3c85b
I20260202 02:52:12.804457 838 raft_consensus.cc:2749] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260202 02:52:12.808162 834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260202 02:52:12.808386 834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.104.130
I20260202 02:52:12.818631 834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:45945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.104.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.104.130
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.0.104.190:33981,127.0.104.189:46271,127.0.104.188:39077
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.104.130
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3722ed49c19ce54c7feae2fa944e32b1d4ed8d2d
build type FASTDEBUG
built by None at 02 Feb 2026 02:43:23 UTC on 24a791456cd2
build id 10304
ASAN enabled
I20260202 02:52:12.820240 834 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260202 02:52:12.822124 834 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20260202 02:52:12.824561 839 raft_consensus.cc:493] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7dab2abef14445d688ce1335b1f9b2f8)
I20260202 02:52:12.824739 839 raft_consensus.cc:515] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } }
I20260202 02:52:12.825577 839 leader_election.cc:290] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7dab2abef14445d688ce1335b1f9b2f8 (127.0.104.190:33981), a9c3e3612bb94fb7b94c308041acfaeb (127.0.104.188:39077)
I20260202 02:52:12.826118 467 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7dab2abef14445d688ce1335b1f9b2f8" is_pre_election: true
I20260202 02:52:12.826185 608 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" is_pre_election: true
I20260202 02:52:12.826392 608 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92a8191cf1e14e4d99f1c378c2d3c85b in term 1.
I20260202 02:52:12.826858 513 leader_election.cc:304] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92a8191cf1e14e4d99f1c378c2d3c85b, a9c3e3612bb94fb7b94c308041acfaeb; no voters: 7dab2abef14445d688ce1335b1f9b2f8
I20260202 02:52:12.827183 839 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260202 02:52:12.827289 839 raft_consensus.cc:493] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7dab2abef14445d688ce1335b1f9b2f8)
I20260202 02:52:12.827374 839 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:12.829352 839 raft_consensus.cc:515] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } }
I20260202 02:52:12.830013 839 leader_election.cc:290] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [CANDIDATE]: Term 2 election: Requested vote from peers 7dab2abef14445d688ce1335b1f9b2f8 (127.0.104.190:33981), a9c3e3612bb94fb7b94c308041acfaeb (127.0.104.188:39077)
I20260202 02:52:12.830561 467 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7dab2abef14445d688ce1335b1f9b2f8"
I20260202 02:52:12.831612 608 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a9c3e3612bb94fb7b94c308041acfaeb"
I20260202 02:52:12.831844 608 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [term 1 FOLLOWER]: Advancing to term 2
W20260202 02:52:12.832995 842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:52:12.833034 841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260202 02:52:12.834520 608 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92a8191cf1e14e4d99f1c378c2d3c85b in term 2.
I20260202 02:52:12.835078 513 leader_election.cc:304] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92a8191cf1e14e4d99f1c378c2d3c85b, a9c3e3612bb94fb7b94c308041acfaeb; no voters: 7dab2abef14445d688ce1335b1f9b2f8
I20260202 02:52:12.835454 839 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [term 2 FOLLOWER]: Leader election won for term 2
I20260202 02:52:12.835826 839 raft_consensus.cc:697] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [term 2 LEADER]: Becoming Leader. State: Replica: 92a8191cf1e14e4d99f1c378c2d3c85b, State: Running, Role: LEADER
I20260202 02:52:12.836287 839 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } }
W20260202 02:52:12.836619 844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260202 02:52:12.838127 845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92a8191cf1e14e4d99f1c378c2d3c85b. Latest consensus state: current_term: 2 leader_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } } }
I20260202 02:52:12.838340 834 server_base.cc:1060] running on GCE node
I20260202 02:52:12.838459 845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [sys.catalog]: This master's current role is: LEADER
I20260202 02:52:12.839118 848 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260202 02:52:12.839582 834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260202 02:52:12.840761 834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260202 02:52:12.841143 848 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260202 02:52:12.842015 834 hybrid_clock.cc:648] HybridClock initialized: now 1770000732841938 us; error 66 us; skew 500 ppm
I20260202 02:52:12.842128 848 catalog_manager.cc:1269] Loaded cluster ID: bdfd876443214aa8b701e616e981c0d4
I20260202 02:52:12.842211 848 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260202 02:52:12.842432 834 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260202 02:52:12.843678 848 catalog_manager.cc:1514] Loading token signing keys...
I20260202 02:52:12.844753 848 catalog_manager.cc:6038] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Loaded TSK: 0
I20260202 02:52:12.845616 608 raft_consensus.cc:1240] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [term 2 FOLLOWER]: Rejecting Update request from peer 7dab2abef14445d688ce1335b1f9b2f8 for earlier term 1. Current term is 2. Ops: []
I20260202 02:52:12.845644 834 webserver.cc:492] Webserver started at http://127.0.104.130:44953/ using document root <none> and password file <none>
I20260202 02:52:12.846024 848 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260202 02:52:12.846271 851 consensus_queue.cc:1059] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260202 02:52:12.846657 834 fs_manager.cc:362] Metadata directory not provided
I20260202 02:52:12.846879 853 raft_consensus.cc:3055] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [term 1 LEADER]: Stepping down as leader of term 1
I20260202 02:52:12.846915 834 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260202 02:52:12.846969 853 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7dab2abef14445d688ce1335b1f9b2f8, State: Running, Role: LEADER
I20260202 02:52:12.847313 853 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } }
I20260202 02:52:12.847437 834 server_base.cc:908] This appears to be a new deployment of Kudu; creating new FS layout
I20260202 02:52:12.847790 853 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:12.850090 834 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/data/instance:
uuid: "1521a6d262b7451bbef8debe970dd6b5"
format_stamp: "Formatted at 2026-02-02 02:52:12 on dist-test-slave-llj2"
I20260202 02:52:12.850976 834 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/instance:
uuid: "1521a6d262b7451bbef8debe970dd6b5"
format_stamp: "Formatted at 2026-02-02 02:52:12 on dist-test-slave-llj2"
I20260202 02:52:12.856902 834 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260202 02:52:12.860641 855 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260202 02:52:12.862223 834 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260202 02:52:12.862393 834 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal
uuid: "1521a6d262b7451bbef8debe970dd6b5"
format_stamp: "Formatted at 2026-02-02 02:52:12 on dist-test-slave-llj2"
I20260202 02:52:12.862573 834 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260202 02:52:12.881093 834 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260202 02:52:12.881973 834 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260202 02:52:12.882216 834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260202 02:52:12.883157 834 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260202 02:52:12.884917 834 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260202 02:52:12.885007 834 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260202 02:52:12.885083 834 ts_tablet_manager.cc:616] Registered 0 tablets
I20260202 02:52:12.885134 834 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260202 02:52:12.921202 834 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.130:38539
I20260202 02:52:12.921236 967 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.130:38539 every 8 connection(s)
I20260202 02:52:12.922631 834 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/data/info.pb
I20260202 02:52:12.927932 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskYXxklX/build/asan/bin/kudu as pid 834
I20260202 02:52:12.928115 418 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/instance
I20260202 02:52:12.934873 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskYXxklX/build/asan/bin/kudu
/tmp/dist-test-taskYXxklX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.104.131:0
--local_ip_for_outbound_sockets=127.0.104.131
--webserver_interface=127.0.104.131
--webserver_port=0
--tserver_master_addrs=127.0.104.190:33981,127.0.104.189:46271,127.0.104.188:39077
--builtin_ntp_servers=127.0.104.148:45945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260202 02:52:12.942700 970 heartbeater.cc:344] Connected to a master server at 127.0.104.189:46271
I20260202 02:52:12.943219 970 heartbeater.cc:461] Registering TS with master...
I20260202 02:52:12.944319 970 heartbeater.cc:507] Master 127.0.104.189:46271 requested a full tablet report, sending...
I20260202 02:52:12.945183 969 heartbeater.cc:344] Connected to a master server at 127.0.104.190:33981
I20260202 02:52:12.945246 968 heartbeater.cc:344] Connected to a master server at 127.0.104.188:39077
I20260202 02:52:12.945317 969 heartbeater.cc:461] Registering TS with master...
I20260202 02:52:12.945392 968 heartbeater.cc:461] Registering TS with master...
I20260202 02:52:12.945788 969 heartbeater.cc:507] Master 127.0.104.190:33981 requested a full tablet report, sending...
I20260202 02:52:12.945951 968 heartbeater.cc:507] Master 127.0.104.188:39077 requested a full tablet report, sending...
I20260202 02:52:12.946866 527 ts_manager.cc:194] Registered new tserver with Master: 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:12.946945 599 ts_manager.cc:194] Registered new tserver with Master: 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:12.946967 457 ts_manager.cc:194] Registered new tserver with Master: 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:12.948935 527 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.104.130:36691
I20260202 02:52:12.951005 825 heartbeater.cc:499] Master 127.0.104.190:33981 was elected leader, sending a full tablet report...
I20260202 02:52:13.241079 467 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [term 2 FOLLOWER]: Refusing update from remote peer 92a8191cf1e14e4d99f1c378c2d3c85b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260202 02:52:13.242195 845 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260202 02:52:13.244750 853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92a8191cf1e14e4d99f1c378c2d3c85b. Latest consensus state: current_term: 2 leader_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } } }
I20260202 02:52:13.245131 853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:52:13.260233 853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } } }
I20260202 02:52:13.260507 853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:52:13.262665 608 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [term 2 FOLLOWER]: Refusing update from remote peer 92a8191cf1e14e4d99f1c378c2d3c85b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260202 02:52:13.263589 845 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260202 02:52:13.267983 845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } } }
I20260202 02:52:13.268245 845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [sys.catalog]: This master's current role is: LEADER
I20260202 02:52:13.274209 838 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92a8191cf1e14e4d99f1c378c2d3c85b. Latest consensus state: current_term: 2 leader_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } } }
I20260202 02:52:13.274484 838 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:52:13.275343 982 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } } }
I20260202 02:52:13.275733 982 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [sys.catalog]: This master's current role is: FOLLOWER
W20260202 02:52:13.311008 975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260202 02:52:13.311322 975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260202 02:52:13.311446 975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260202 02:52:13.320427 975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260202 02:52:13.320637 975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.104.131
I20260202 02:52:13.331821 975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:45945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.104.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.104.131
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.0.104.190:33981,127.0.104.189:46271,127.0.104.188:39077
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.104.131
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3722ed49c19ce54c7feae2fa944e32b1d4ed8d2d
build type FASTDEBUG
built by None at 02 Feb 2026 02:43:23 UTC on 24a791456cd2
build id 10304
ASAN enabled
I20260202 02:52:13.333539 975 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260202 02:52:13.335544 975 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260202 02:52:13.345983 988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:52:13.346093 987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:52:13.346020 990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260202 02:52:13.347084 975 server_base.cc:1060] running on GCE node
I20260202 02:52:13.347738 975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260202 02:52:13.348866 975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260202 02:52:13.350142 975 hybrid_clock.cc:648] HybridClock initialized: now 1770000733350060 us; error 60 us; skew 500 ppm
I20260202 02:52:13.350456 975 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260202 02:52:13.353704 975 webserver.cc:492] Webserver started at http://127.0.104.131:45735/ using document root <none> and password file <none>
I20260202 02:52:13.354480 975 fs_manager.cc:362] Metadata directory not provided
I20260202 02:52:13.354578 975 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260202 02:52:13.354925 975 server_base.cc:908] This appears to be a new deployment of Kudu; creating new FS layout
I20260202 02:52:13.357401 975 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/data/instance:
uuid: "b1f05820b6034228bdad884cc348716d"
format_stamp: "Formatted at 2026-02-02 02:52:13 on dist-test-slave-llj2"
I20260202 02:52:13.358177 975 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/instance:
uuid: "b1f05820b6034228bdad884cc348716d"
format_stamp: "Formatted at 2026-02-02 02:52:13 on dist-test-slave-llj2"
I20260202 02:52:13.363963 975 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260202 02:52:13.367851 996 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260202 02:52:13.369380 975 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260202 02:52:13.369560 975 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal
uuid: "b1f05820b6034228bdad884cc348716d"
format_stamp: "Formatted at 2026-02-02 02:52:13 on dist-test-slave-llj2"
I20260202 02:52:13.369776 975 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260202 02:52:13.406795 975 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260202 02:52:13.407734 975 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260202 02:52:13.408004 975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260202 02:52:13.409027 975 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260202 02:52:13.410830 975 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260202 02:52:13.410946 975 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260202 02:52:13.411057 975 ts_tablet_manager.cc:616] Registered 0 tablets
I20260202 02:52:13.411125 975 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260202 02:52:13.436470 668 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260202 02:52:13.450126 826 heartbeater.cc:499] Master 127.0.104.189:46271 was elected leader, sending a full tablet report...
I20260202 02:52:13.451006 975 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.131:35257
I20260202 02:52:13.451069 1108 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.131:35257 every 8 connection(s)
I20260202 02:52:13.451521 970 heartbeater.cc:499] Master 127.0.104.189:46271 was elected leader, sending a full tablet report...
I20260202 02:52:13.452932 975 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/data/info.pb
I20260202 02:52:13.457454 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskYXxklX/build/asan/bin/kudu as pid 975
I20260202 02:52:13.457665 418 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/instance
I20260202 02:52:13.485412 1109 heartbeater.cc:344] Connected to a master server at 127.0.104.188:39077
I20260202 02:52:13.485904 1109 heartbeater.cc:461] Registering TS with master...
I20260202 02:52:13.487190 1109 heartbeater.cc:507] Master 127.0.104.188:39077 requested a full tablet report, sending...
I20260202 02:52:13.489286 599 ts_manager.cc:194] Registered new tserver with Master: b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:13.497823 1110 heartbeater.cc:344] Connected to a master server at 127.0.104.190:33981
I20260202 02:52:13.497813 1113 heartbeater.cc:344] Connected to a master server at 127.0.104.189:46271
I20260202 02:52:13.498036 1110 heartbeater.cc:461] Registering TS with master...
I20260202 02:52:13.498042 1113 heartbeater.cc:461] Registering TS with master...
I20260202 02:52:13.498780 1110 heartbeater.cc:507] Master 127.0.104.190:33981 requested a full tablet report, sending...
I20260202 02:52:13.498797 1113 heartbeater.cc:507] Master 127.0.104.189:46271 requested a full tablet report, sending...
I20260202 02:52:13.500005 457 ts_manager.cc:194] Registered new tserver with Master: b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:13.500047 527 ts_manager.cc:194] Registered new tserver with Master: b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:13.501618 527 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.104.131:43527
I20260202 02:52:13.503652 418 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20260202 02:52:13.543046 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_c65564cf55a84e899dde53d2bc369dbb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:13.536993 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_84b1848e1f86489186c1554d71a8fa2a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:13.542567 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_1a6774b42978452c81acd697ec5945bb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:13.547221 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_587a93950ec54028a83faabe215caeb0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:13.550905 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_84b1848e1f86489186c1554d71a8fa2a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:13.549201 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_f4ca994134694824a2febac878defaab"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:13.551515 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f4ca994134694824a2febac878defaab in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:13.551836 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_c774671d732f4fe784e75d08f10a010b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:13.552539 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_587a93950ec54028a83faabe215caeb0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:13.552628 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c774671d732f4fe784e75d08f10a010b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:13.554221 418 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskYXxklX/build/asan/bin/kudu pid 565
W20260202 02:52:13.554270 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c65564cf55a84e899dde53d2bc369dbb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:13.555183 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1a6774b42978452c81acd697ec5945bb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:13.555620 629 generic_service.cc:196] Checking for leaks (request via RPC)
I20260202 02:52:13.552805 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_d0a7052b37cb491fb787e9e3b287477f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:13.547828 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_6b433dc791af4d41932c1dcb1c9145a7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:13.558247 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_af8774882d7f4911b2b74be39c05a989"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:13.556900 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d0a7052b37cb491fb787e9e3b287477f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:13.567103 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6b433dc791af4d41932c1dcb1c9145a7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:13.561653 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_05293889233745d6b7b22004435aa46b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:13.573169 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_af8774882d7f4911b2b74be39c05a989 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:13.580157 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_05293889233745d6b7b22004435aa46b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:13.581336 979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } } }
I20260202 02:52:13.581665 979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [sys.catalog]: This master's current role is: LEADER
I20260202 02:52:13.592947 524 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_84b1848e1f86489186c1554d71a8fa2a" } new_table_name: "default.Table_2f7cf4b6f60845c7977bbb0b5f2b7fd6" modify_external_catalogs: true
I20260202 02:52:13.595548 526 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_587a93950ec54028a83faabe215caeb0" } new_table_name: "default.Table_9dda7d5509c6438080dc857aa97ae9e0" modify_external_catalogs: true
I20260202 02:52:13.616127 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_c65564cf55a84e899dde53d2bc369dbb" } new_table_name: "default.Table_7bc599752fa247b6b809b695a0faea51" modify_external_catalogs: true
I20260202 02:52:13.618784 522 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_d0a7052b37cb491fb787e9e3b287477f" } modify_external_catalogs: true
I20260202 02:52:13.619607 520 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_f4ca994134694824a2febac878defaab" } new_table_name: "default.Table_0867803a1ea7405baa41acb890f44d2f" modify_external_catalogs: true
I20260202 02:52:13.619735 521 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_6b433dc791af4d41932c1dcb1c9145a7" } modify_external_catalogs: true
I20260202 02:52:13.619148 519 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_c774671d732f4fe784e75d08f10a010b" } modify_external_catalogs: true
I20260202 02:52:13.619446 522 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_d0a7052b37cb491fb787e9e3b287477f" } modify_external_catalogs: true
I20260202 02:52:13.621115 528 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_1a6774b42978452c81acd697ec5945bb" } new_table_name: "default.Table_690b9477c0f64b5b81388b8d8f9b5a2b" modify_external_catalogs: true
I20260202 02:52:13.619895 521 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_6b433dc791af4d41932c1dcb1c9145a7" } modify_external_catalogs: true
I20260202 02:52:13.620077 519 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_c774671d732f4fe784e75d08f10a010b" } modify_external_catalogs: true
I20260202 02:52:13.622584 525 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_af8774882d7f4911b2b74be39c05a989" } modify_external_catalogs: true
I20260202 02:52:13.623108 525 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_af8774882d7f4911b2b74be39c05a989" } modify_external_catalogs: true
I20260202 02:52:13.624715 523 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_05293889233745d6b7b22004435aa46b" } modify_external_catalogs: true
I20260202 02:52:13.624979 523 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_05293889233745d6b7b22004435aa46b" } modify_external_catalogs: true
W20260202 02:52:13.625811 524 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5d8de32fc94d4835862fc133c6b442ef consensus state has no leader
W20260202 02:52:13.628577 526 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 51c11ad251924fb7b126465932a2e74d consensus state has no leader
W20260202 02:52:13.641347 527 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6bac811b0f9f4276ac63786d97f02188 has no consensus state
I20260202 02:52:13.642354 522 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet 5498b47f0b584d04b2a50b21c06490bf
W20260202 02:52:13.642575 520 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ee6e1a7236694d5092b3c1320291be42 has no consensus state
W20260202 02:52:13.642938 528 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 62241ababede41a1bd7f91620809f5f8 has no consensus state
I20260202 02:52:13.651448 521 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet 51a992aadd654e01b4c374b87cef43bc
I20260202 02:52:13.651806 523 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet ce997929aeca4e1eb9579820168d3adc
I20260202 02:52:13.652101 519 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet c79edcab00f546a099a8427cc3d8c893
I20260202 02:52:13.651576 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 51c11ad251924fb7b126465932a2e74d (DEFAULT_TABLE table=default.Table_587a93950ec54028a83faabe215caeb0 [id=31c0ee69dd9a473aada01e871ac80e81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.653353 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51c11ad251924fb7b126465932a2e74d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.653708 1043 tablet_service.cc:1505] Processing CreateTablet for tablet 5d8de32fc94d4835862fc133c6b442ef (DEFAULT_TABLE table=default.Table_2f7cf4b6f60845c7977bbb0b5f2b7fd6 [id=1b7d3c09f6b84bd8b8634635c0db2295]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.654281 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d8de32fc94d4835862fc133c6b442ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.653728 759 tablet_service.cc:1505] Processing CreateTablet for tablet 51c11ad251924fb7b126465932a2e74d (DEFAULT_TABLE table=default.Table_587a93950ec54028a83faabe215caeb0 [id=31c0ee69dd9a473aada01e871ac80e81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.655243 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51c11ad251924fb7b126465932a2e74d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.657014 758 tablet_service.cc:1505] Processing CreateTablet for tablet 5d8de32fc94d4835862fc133c6b442ef (DEFAULT_TABLE table=default.Table_2f7cf4b6f60845c7977bbb0b5f2b7fd6 [id=1b7d3c09f6b84bd8b8634635c0db2295]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.657480 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d8de32fc94d4835862fc133c6b442ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.656960 903 tablet_service.cc:1505] Processing CreateTablet for tablet 51c11ad251924fb7b126465932a2e74d (DEFAULT_TABLE table=default.Table_587a93950ec54028a83faabe215caeb0 [id=31c0ee69dd9a473aada01e871ac80e81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.658476 903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51c11ad251924fb7b126465932a2e74d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.658646 525 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet a74323e43a5649138747335f67cd3574
I20260202 02:52:13.660843 902 tablet_service.cc:1505] Processing CreateTablet for tablet 5d8de32fc94d4835862fc133c6b442ef (DEFAULT_TABLE table=default.Table_2f7cf4b6f60845c7977bbb0b5f2b7fd6 [id=1b7d3c09f6b84bd8b8634635c0db2295]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.661240 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d8de32fc94d4835862fc133c6b442ef. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:13.664469 514 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 5d8de32fc94d4835862fc133c6b442ef (table default.Table_2f7cf4b6f60845c7977bbb0b5f2b7fd6 [id=1b7d3c09f6b84bd8b8634635c0db2295], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 5d8de32fc94d4835862fc133c6b442ef consensus state has no leader
I20260202 02:52:13.678144 1043 tablet_service.cc:1505] Processing CreateTablet for tablet ee6e1a7236694d5092b3c1320291be42 (DEFAULT_TABLE table=default.Table_0867803a1ea7405baa41acb890f44d2f [id=1d7c11d3ee26455e995b30f69a967132]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.678651 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee6e1a7236694d5092b3c1320291be42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.680195 757 tablet_service.cc:1505] Processing CreateTablet for tablet ee6e1a7236694d5092b3c1320291be42 (DEFAULT_TABLE table=default.Table_0867803a1ea7405baa41acb890f44d2f [id=1d7c11d3ee26455e995b30f69a967132]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.680832 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee6e1a7236694d5092b3c1320291be42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.684801 755 tablet_service.cc:1505] Processing CreateTablet for tablet 6bac811b0f9f4276ac63786d97f02188 (DEFAULT_TABLE table=default.Table_7bc599752fa247b6b809b695a0faea51 [id=01b7ed0408cb46cabd569c0cf240bbaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.685415 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bac811b0f9f4276ac63786d97f02188. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.688130 1173 tablet_bootstrap.cc:492] T 51c11ad251924fb7b126465932a2e74d P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:13.689364 756 tablet_service.cc:1505] Processing CreateTablet for tablet 62241ababede41a1bd7f91620809f5f8 (DEFAULT_TABLE table=default.Table_690b9477c0f64b5b81388b8d8f9b5a2b [id=3ccb047c30974c0f9cbe778971e61644]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.690039 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62241ababede41a1bd7f91620809f5f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.690809 1175 tablet_bootstrap.cc:492] T 51c11ad251924fb7b126465932a2e74d P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:13.691299 1173 tablet_bootstrap.cc:654] T 51c11ad251924fb7b126465932a2e74d P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:13.693427 1173 log.cc:826] T 51c11ad251924fb7b126465932a2e74d P b1f05820b6034228bdad884cc348716d: Log is configured to *not* fsync() on all Append() calls
I20260202 02:52:13.695145 1042 tablet_service.cc:1505] Processing CreateTablet for tablet 6bac811b0f9f4276ac63786d97f02188 (DEFAULT_TABLE table=default.Table_7bc599752fa247b6b809b695a0faea51 [id=01b7ed0408cb46cabd569c0cf240bbaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.695885 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bac811b0f9f4276ac63786d97f02188. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.697448 902 tablet_service.cc:1505] Processing CreateTablet for tablet 62241ababede41a1bd7f91620809f5f8 (DEFAULT_TABLE table=default.Table_690b9477c0f64b5b81388b8d8f9b5a2b [id=3ccb047c30974c0f9cbe778971e61644]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.699002 901 tablet_service.cc:1505] Processing CreateTablet for tablet 6bac811b0f9f4276ac63786d97f02188 (DEFAULT_TABLE table=default.Table_7bc599752fa247b6b809b695a0faea51 [id=01b7ed0408cb46cabd569c0cf240bbaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.701030 1043 tablet_service.cc:1505] Processing CreateTablet for tablet 62241ababede41a1bd7f91620809f5f8 (DEFAULT_TABLE table=default.Table_690b9477c0f64b5b81388b8d8f9b5a2b [id=3ccb047c30974c0f9cbe778971e61644]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.702199 903 tablet_service.cc:1505] Processing CreateTablet for tablet ee6e1a7236694d5092b3c1320291be42 (DEFAULT_TABLE table=default.Table_0867803a1ea7405baa41acb890f44d2f [id=1d7c11d3ee26455e995b30f69a967132]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.702867 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62241ababede41a1bd7f91620809f5f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.709497 1174 tablet_bootstrap.cc:492] T 5d8de32fc94d4835862fc133c6b442ef P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:13.711733 1175 tablet_bootstrap.cc:654] T 51c11ad251924fb7b126465932a2e74d P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:13.712797 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bac811b0f9f4276ac63786d97f02188. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.713267 1174 tablet_bootstrap.cc:654] T 5d8de32fc94d4835862fc133c6b442ef P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:13.713343 1173 tablet_bootstrap.cc:492] T 51c11ad251924fb7b126465932a2e74d P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:13.713650 1173 ts_tablet_manager.cc:1403] T 51c11ad251924fb7b126465932a2e74d P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.026s user 0.012s sys 0.010s
I20260202 02:52:13.714751 1174 log.cc:826] T 5d8de32fc94d4835862fc133c6b442ef P 1f442ef85029461091de847118c34c0d: Log is configured to *not* fsync() on all Append() calls
I20260202 02:52:13.716552 903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee6e1a7236694d5092b3c1320291be42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.722193 1173 raft_consensus.cc:359] T 51c11ad251924fb7b126465932a2e74d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:13.722719 1173 raft_consensus.cc:385] T 51c11ad251924fb7b126465932a2e74d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:13.722829 1173 raft_consensus.cc:740] T 51c11ad251924fb7b126465932a2e74d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:13.723843 1173 consensus_queue.cc:260] T 51c11ad251924fb7b126465932a2e74d P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:13.724977 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62241ababede41a1bd7f91620809f5f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.729321 525 master_service.cc:954] ReplaceTablet: received request to replace tablet 51c11ad251924fb7b126465932a2e74d from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:13.731464 1175 log.cc:826] T 51c11ad251924fb7b126465932a2e74d P 1521a6d262b7451bbef8debe970dd6b5: Log is configured to *not* fsync() on all Append() calls
I20260202 02:52:13.733528 1173 ts_tablet_manager.cc:1434] T 51c11ad251924fb7b126465932a2e74d P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.020s user 0.011s sys 0.009s
I20260202 02:52:13.737882 519 master_service.cc:954] ReplaceTablet: received request to replace tablet 5d8de32fc94d4835862fc133c6b442ef from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:13.738154 1174 tablet_bootstrap.cc:492] T 5d8de32fc94d4835862fc133c6b442ef P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:13.738471 1174 ts_tablet_manager.cc:1403] T 5d8de32fc94d4835862fc133c6b442ef P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.029s user 0.014s sys 0.000s
I20260202 02:52:13.739362 523 master_service.cc:954] ReplaceTablet: received request to replace tablet 51c11ad251924fb7b126465932a2e74d from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:13.740249 525 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 51c11ad251924fb7b126465932a2e74d
I20260202 02:52:13.740947 1173 tablet_bootstrap.cc:492] T 5d8de32fc94d4835862fc133c6b442ef P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:13.741047 525 catalog_manager.cc:6724] ReplaceTablet: tablet 51c11ad251924fb7b126465932a2e74d deleted and replaced by tablet ea0b6bb727404522b8291cbcd12879ea
I20260202 02:52:13.741454 1113 heartbeater.cc:499] Master 127.0.104.189:46271 was elected leader, sending a full tablet report...
I20260202 02:52:13.741729 521 master_service.cc:954] ReplaceTablet: received request to replace tablet 5d8de32fc94d4835862fc133c6b442ef from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:13.743009 758 tablet_service.cc:1552] Processing DeleteTablet for tablet 51c11ad251924fb7b126465932a2e74d with delete_type TABLET_DATA_DELETED (replaced by tablet ea0b6bb727404522b8291cbcd12879ea) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:13.744107 1043 tablet_service.cc:1552] Processing DeleteTablet for tablet 51c11ad251924fb7b126465932a2e74d with delete_type TABLET_DATA_DELETED (replaced by tablet ea0b6bb727404522b8291cbcd12879ea) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:13.751006 519 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 5d8de32fc94d4835862fc133c6b442ef
I20260202 02:52:13.751484 519 catalog_manager.cc:6724] ReplaceTablet: tablet 5d8de32fc94d4835862fc133c6b442ef deleted and replaced by tablet 7781ccb7f1ea437fb1d61d3625a739a9
I20260202 02:52:13.751227 1174 raft_consensus.cc:359] T 5d8de32fc94d4835862fc133c6b442ef P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:13.751828 1174 raft_consensus.cc:385] T 5d8de32fc94d4835862fc133c6b442ef P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:13.751953 1174 raft_consensus.cc:740] T 5d8de32fc94d4835862fc133c6b442ef P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:13.752791 1174 consensus_queue.cc:260] T 5d8de32fc94d4835862fc133c6b442ef P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:13.753052 525 master_service.cc:954] ReplaceTablet: received request to replace tablet ee6e1a7236694d5092b3c1320291be42 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:13.753193 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 5d8de32fc94d4835862fc133c6b442ef with delete_type TABLET_DATA_DELETED (replaced by tablet 7781ccb7f1ea437fb1d61d3625a739a9) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:13.753540 902 tablet_service.cc:1552] Processing DeleteTablet for tablet 51c11ad251924fb7b126465932a2e74d with delete_type TABLET_DATA_DELETED (replaced by tablet ea0b6bb727404522b8291cbcd12879ea) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:13.754371 1116 tablet.cc:2378] T 51c11ad251924fb7b126465932a2e74d P b1f05820b6034228bdad884cc348716d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260202 02:52:13.755333 901 tablet_service.cc:1552] Processing DeleteTablet for tablet 5d8de32fc94d4835862fc133c6b442ef with delete_type TABLET_DATA_DELETED (replaced by tablet 7781ccb7f1ea437fb1d61d3625a739a9) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:13.755977 1175 tablet_bootstrap.cc:492] T 51c11ad251924fb7b126465932a2e74d P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:13.756299 1175 ts_tablet_manager.cc:1403] T 51c11ad251924fb7b126465932a2e74d P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.066s user 0.008s sys 0.000s
I20260202 02:52:13.756893 1042 tablet_service.cc:1552] Processing DeleteTablet for tablet 5d8de32fc94d4835862fc133c6b442ef with delete_type TABLET_DATA_DELETED (replaced by tablet 7781ccb7f1ea437fb1d61d3625a739a9) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:13.761435 1183 tablet_replica.cc:333] T 51c11ad251924fb7b126465932a2e74d P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:13.762049 1183 raft_consensus.cc:2243] T 51c11ad251924fb7b126465932a2e74d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:13.762554 1183 raft_consensus.cc:2272] T 51c11ad251924fb7b126465932a2e74d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:13.764573 1183 ts_tablet_manager.cc:1916] T 51c11ad251924fb7b126465932a2e74d P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:13.764299 1175 raft_consensus.cc:359] T 51c11ad251924fb7b126465932a2e74d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:13.764751 1175 raft_consensus.cc:385] T 51c11ad251924fb7b126465932a2e74d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:13.764860 1175 raft_consensus.cc:740] T 51c11ad251924fb7b126465932a2e74d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:13.765403 756 tablet_service.cc:1505] Processing CreateTablet for tablet ea0b6bb727404522b8291cbcd12879ea (DEFAULT_TABLE table=default.Table_9dda7d5509c6438080dc857aa97ae9e0 [id=31c0ee69dd9a473aada01e871ac80e81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.765740 901 tablet_service.cc:1505] Processing CreateTablet for tablet ea0b6bb727404522b8291cbcd12879ea (DEFAULT_TABLE table=default.Table_9dda7d5509c6438080dc857aa97ae9e0 [id=31c0ee69dd9a473aada01e871ac80e81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.765683 1175 consensus_queue.cc:260] T 51c11ad251924fb7b126465932a2e74d P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:13.765877 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea0b6bb727404522b8291cbcd12879ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.766158 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea0b6bb727404522b8291cbcd12879ea. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:13.774817 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 51c11ad251924fb7b126465932a2e74d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 51c11ad251924fb7b126465932a2e74d already in progress: creating tablet
I20260202 02:52:13.777379 1175 ts_tablet_manager.cc:1434] T 51c11ad251924fb7b126465932a2e74d P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.021s user 0.002s sys 0.012s
I20260202 02:52:13.773386 1173 tablet_bootstrap.cc:654] T 5d8de32fc94d4835862fc133c6b442ef P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:13.774572 1043 tablet_service.cc:1505] Processing CreateTablet for tablet ea0b6bb727404522b8291cbcd12879ea (DEFAULT_TABLE table=default.Table_9dda7d5509c6438080dc857aa97ae9e0 [id=31c0ee69dd9a473aada01e871ac80e81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.779138 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea0b6bb727404522b8291cbcd12879ea. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:13.780344 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 5d8de32fc94d4835862fc133c6b442ef because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5d8de32fc94d4835862fc133c6b442ef already in progress: creating tablet
I20260202 02:52:13.790997 1042 tablet_service.cc:1505] Processing CreateTablet for tablet 7781ccb7f1ea437fb1d61d3625a739a9 (DEFAULT_TABLE table=default.Table_2f7cf4b6f60845c7977bbb0b5f2b7fd6 [id=1b7d3c09f6b84bd8b8634635c0db2295]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.791496 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7781ccb7f1ea437fb1d61d3625a739a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.795055 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_53f102d02f1c4c6c9002f9142b59434e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:13.795256 1175 tablet_bootstrap.cc:492] T 5d8de32fc94d4835862fc133c6b442ef P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
W20260202 02:52:13.795686 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_53f102d02f1c4c6c9002f9142b59434e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:13.797286 525 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet ee6e1a7236694d5092b3c1320291be42
I20260202 02:52:13.797825 525 catalog_manager.cc:6724] ReplaceTablet: tablet ee6e1a7236694d5092b3c1320291be42 deleted and replaced by tablet a6940c7f38ff496abf3c2dab66da5100
I20260202 02:52:13.798079 901 tablet_service.cc:1505] Processing CreateTablet for tablet 7781ccb7f1ea437fb1d61d3625a739a9 (DEFAULT_TABLE table=default.Table_2f7cf4b6f60845c7977bbb0b5f2b7fd6 [id=1b7d3c09f6b84bd8b8634635c0db2295]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.798558 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7781ccb7f1ea437fb1d61d3625a739a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.798605 902 tablet_service.cc:1552] Processing DeleteTablet for tablet ee6e1a7236694d5092b3c1320291be42 with delete_type TABLET_DATA_DELETED (replaced by tablet a6940c7f38ff496abf3c2dab66da5100) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:13.797943 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_f03b4b80c09b4362b51aba61a6beb163"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:13.798897 755 tablet_service.cc:1552] Processing DeleteTablet for tablet ee6e1a7236694d5092b3c1320291be42 with delete_type TABLET_DATA_DELETED (replaced by tablet a6940c7f38ff496abf3c2dab66da5100) from {username='slave'} at 127.0.0.1:42950
W20260202 02:52:13.799137 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f03b4b80c09b4362b51aba61a6beb163 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:13.799711 523 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 62241ababede41a1bd7f91620809f5f8 (table default.Table_690b9477c0f64b5b81388b8d8f9b5a2b [id=3ccb047c30974c0f9cbe778971e61644]). Expected version 1 got 0
I20260202 02:52:13.799748 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet ee6e1a7236694d5092b3c1320291be42 with delete_type TABLET_DATA_DELETED (replaced by tablet a6940c7f38ff496abf3c2dab66da5100) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:13.799949 523 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 6bac811b0f9f4276ac63786d97f02188 (table default.Table_7bc599752fa247b6b809b695a0faea51 [id=01b7ed0408cb46cabd569c0cf240bbaa]). Expected version 1 got 0
I20260202 02:52:13.801827 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_482da7030b9443f7a633ba6e258471e2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:13.802457 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_482da7030b9443f7a633ba6e258471e2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:13.802737 1174 ts_tablet_manager.cc:1434] T 5d8de32fc94d4835862fc133c6b442ef P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.064s user 0.010s sys 0.007s
I20260202 02:52:13.802929 902 tablet_service.cc:1552] Processing DeleteTablet for tablet 51c11ad251924fb7b126465932a2e74d with delete_type TABLET_DATA_DELETED (replaced by tablet ea0b6bb727404522b8291cbcd12879ea) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:13.804698 756 tablet_service.cc:1505] Processing CreateTablet for tablet 7781ccb7f1ea437fb1d61d3625a739a9 (DEFAULT_TABLE table=default.Table_2f7cf4b6f60845c7977bbb0b5f2b7fd6 [id=1b7d3c09f6b84bd8b8634635c0db2295]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.805702 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_c5a4aa6f50fb472c8982feedb30982a9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:13.806097 1175 tablet_bootstrap.cc:654] T 5d8de32fc94d4835862fc133c6b442ef P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:13.806351 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c5a4aa6f50fb472c8982feedb30982a9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:13.805140 1174 tablet_bootstrap.cc:492] T 51c11ad251924fb7b126465932a2e74d P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:13.809211 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_d3c2e0661e58450595252a1a9b36063c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:13.809839 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d3c2e0661e58450595252a1a9b36063c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:13.810204 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7781ccb7f1ea437fb1d61d3625a739a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.810431 1174 tablet_bootstrap.cc:654] T 51c11ad251924fb7b126465932a2e74d P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:13.813920 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 51c11ad251924fb7b126465932a2e74d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 51c11ad251924fb7b126465932a2e74d already in progress: creating tablet
I20260202 02:52:13.814577 1180 tablet_replica.cc:333] T 5d8de32fc94d4835862fc133c6b442ef P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:13.815162 1180 raft_consensus.cc:2243] T 5d8de32fc94d4835862fc133c6b442ef P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:13.815734 1180 raft_consensus.cc:2272] T 5d8de32fc94d4835862fc133c6b442ef P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:13.816248 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_2475d75250c24b3ea51238f58fd5f32b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:13.816838 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2475d75250c24b3ea51238f58fd5f32b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:13.821573 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_16e2b25dc4f0411ba65b7bb66240ec31"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:13.822227 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_16e2b25dc4f0411ba65b7bb66240ec31 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:13.821555 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_ef6bf7a12f3b4a64b25fb14a57c26ae3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:13.823449 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ef6bf7a12f3b4a64b25fb14a57c26ae3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:13.823938 1180 ts_tablet_manager.cc:1916] T 5d8de32fc94d4835862fc133c6b442ef P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:13.824625 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_02df7f59f31c4259844bf3dd916f0c35"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:13.825150 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_02df7f59f31c4259844bf3dd916f0c35 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:13.831501 1182 tablet_replica.cc:333] T 51c11ad251924fb7b126465932a2e74d P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
W20260202 02:52:13.831570 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet ee6e1a7236694d5092b3c1320291be42 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ee6e1a7236694d5092b3c1320291be42 already in progress: creating tablet
I20260202 02:52:13.832078 1182 raft_consensus.cc:2243] T 51c11ad251924fb7b126465932a2e74d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:13.832563 1182 raft_consensus.cc:2272] T 51c11ad251924fb7b126465932a2e74d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:13.836362 522 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_0867803a1ea7405baa41acb890f44d2f" } new_table_name: "default.Table_011de32f5bb648439181d1154b82b2c5" modify_external_catalogs: true
I20260202 02:52:13.837266 1173 tablet_bootstrap.cc:492] T 5d8de32fc94d4835862fc133c6b442ef P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:13.837464 1173 ts_tablet_manager.cc:1403] T 5d8de32fc94d4835862fc133c6b442ef P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.097s user 0.004s sys 0.009s
I20260202 02:52:13.838956 1173 raft_consensus.cc:359] T 5d8de32fc94d4835862fc133c6b442ef P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:13.839183 1173 raft_consensus.cc:385] T 5d8de32fc94d4835862fc133c6b442ef P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:13.839264 1173 raft_consensus.cc:740] T 5d8de32fc94d4835862fc133c6b442ef P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:13.839581 1173 consensus_queue.cc:260] T 5d8de32fc94d4835862fc133c6b442ef P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:13.840445 1173 ts_tablet_manager.cc:1434] T 5d8de32fc94d4835862fc133c6b442ef P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260202 02:52:13.840626 524 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_9dda7d5509c6438080dc857aa97ae9e0" } new_table_name: "default.Table_c86dacfe4afb42ed9f08fa8a2a6ecc75" modify_external_catalogs: true
I20260202 02:52:13.840960 1173 tablet_bootstrap.cc:492] T ee6e1a7236694d5092b3c1320291be42 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:13.845317 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_690b9477c0f64b5b81388b8d8f9b5a2b" } new_table_name: "default.Table_4a3ef984ca894b8b8f4301fecb719b5b" modify_external_catalogs: true
I20260202 02:52:13.847129 1182 ts_tablet_manager.cc:1916] T 51c11ad251924fb7b126465932a2e74d P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:13.848238 521 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_53f102d02f1c4c6c9002f9142b59434e" } new_table_name: "default.Table_62840640773d47f5889c6a82f84ffb32" modify_external_catalogs: true
I20260202 02:52:13.848831 1173 tablet_bootstrap.cc:654] T ee6e1a7236694d5092b3c1320291be42 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:13.849730 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_ab0c372e2fbc439cbfc95db9aa9c75d0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:13.850335 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ab0c372e2fbc439cbfc95db9aa9c75d0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:13.851377 1183 ts_tablet_manager.cc:1929] T 51c11ad251924fb7b126465932a2e74d P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:13.851531 1183 log.cc:1199] T 51c11ad251924fb7b126465932a2e74d P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/51c11ad251924fb7b126465932a2e74d
I20260202 02:52:13.851964 1183 ts_tablet_manager.cc:1950] T 51c11ad251924fb7b126465932a2e74d P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:13.852118 520 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_d3c2e0661e58450595252a1a9b36063c" } modify_external_catalogs: true
I20260202 02:52:13.852293 520 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_d3c2e0661e58450595252a1a9b36063c" } modify_external_catalogs: true
I20260202 02:52:13.852746 526 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_7bc599752fa247b6b809b695a0faea51" } new_table_name: "default.Table_5b9ebdf0ffec42f88f5bd9a3d0d158e9" modify_external_catalogs: true
I20260202 02:52:13.854491 525 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_16e2b25dc4f0411ba65b7bb66240ec31" } modify_external_catalogs: true
I20260202 02:52:13.854676 525 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_16e2b25dc4f0411ba65b7bb66240ec31" } modify_external_catalogs: true
I20260202 02:52:13.855826 528 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_2f7cf4b6f60845c7977bbb0b5f2b7fd6" } modify_external_catalogs: true
I20260202 02:52:13.855985 528 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_2f7cf4b6f60845c7977bbb0b5f2b7fd6" } modify_external_catalogs: true
I20260202 02:52:13.856362 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 51c11ad251924fb7b126465932a2e74d (table default.Table_9dda7d5509c6438080dc857aa97ae9e0 [id=31c0ee69dd9a473aada01e871ac80e81]) successfully deleted
I20260202 02:52:13.856755 1182 ts_tablet_manager.cc:1929] T 51c11ad251924fb7b126465932a2e74d P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:13.856843 523 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ef6bf7a12f3b4a64b25fb14a57c26ae3" } modify_external_catalogs: true
I20260202 02:52:13.857017 523 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ef6bf7a12f3b4a64b25fb14a57c26ae3" } modify_external_catalogs: true
I20260202 02:52:13.857316 1182 log.cc:1199] T 51c11ad251924fb7b126465932a2e74d P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/51c11ad251924fb7b126465932a2e74d
I20260202 02:52:13.858665 901 tablet_service.cc:1505] Processing CreateTablet for tablet a6940c7f38ff496abf3c2dab66da5100 (DEFAULT_TABLE table=default.Table_0867803a1ea7405baa41acb890f44d2f [id=1d7c11d3ee26455e995b30f69a967132]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.858963 1042 tablet_service.cc:1505] Processing CreateTablet for tablet a6940c7f38ff496abf3c2dab66da5100 (DEFAULT_TABLE table=default.Table_0867803a1ea7405baa41acb890f44d2f [id=1d7c11d3ee26455e995b30f69a967132]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.859048 756 tablet_service.cc:1505] Processing CreateTablet for tablet a6940c7f38ff496abf3c2dab66da5100 (DEFAULT_TABLE table=default.Table_0867803a1ea7405baa41acb890f44d2f [id=1d7c11d3ee26455e995b30f69a967132]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.859177 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6940c7f38ff496abf3c2dab66da5100. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.859349 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6940c7f38ff496abf3c2dab66da5100. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.859515 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6940c7f38ff496abf3c2dab66da5100. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.865957 1182 ts_tablet_manager.cc:1950] T 51c11ad251924fb7b126465932a2e74d P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:13.868079 1174 tablet_bootstrap.cc:492] T 51c11ad251924fb7b126465932a2e74d P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:13.868286 1174 ts_tablet_manager.cc:1403] T 51c11ad251924fb7b126465932a2e74d P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.064s user 0.011s sys 0.001s
W20260202 02:52:13.868484 522 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a6940c7f38ff496abf3c2dab66da5100 consensus state has no leader
I20260202 02:52:13.868579 1183 tablet_replica.cc:333] T 5d8de32fc94d4835862fc133c6b442ef P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:13.868788 1183 raft_consensus.cc:2243] T 5d8de32fc94d4835862fc133c6b442ef P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:13.868515 1175 tablet_bootstrap.cc:492] T 5d8de32fc94d4835862fc133c6b442ef P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:13.868868 522 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_482da7030b9443f7a633ba6e258471e2" } modify_external_catalogs: true
I20260202 02:52:13.868961 1183 raft_consensus.cc:2272] T 5d8de32fc94d4835862fc133c6b442ef P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:13.869079 522 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_482da7030b9443f7a633ba6e258471e2" } modify_external_catalogs: true
I20260202 02:52:13.869401 1174 raft_consensus.cc:359] T 51c11ad251924fb7b126465932a2e74d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:13.869608 1174 raft_consensus.cc:385] T 51c11ad251924fb7b126465932a2e74d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:13.869155 1175 ts_tablet_manager.cc:1403] T 5d8de32fc94d4835862fc133c6b442ef P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.074s user 0.013s sys 0.001s
I20260202 02:52:13.869704 1174 raft_consensus.cc:740] T 51c11ad251924fb7b126465932a2e74d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:13.869995 1174 consensus_queue.cc:260] T 51c11ad251924fb7b126465932a2e74d P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:13.870529 1174 ts_tablet_manager.cc:1434] T 51c11ad251924fb7b126465932a2e74d P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:13.871125 1175 raft_consensus.cc:359] T 5d8de32fc94d4835862fc133c6b442ef P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:13.871776 1180 ts_tablet_manager.cc:1929] T 5d8de32fc94d4835862fc133c6b442ef P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:13.872519 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 51c11ad251924fb7b126465932a2e74d (table default.Table_9dda7d5509c6438080dc857aa97ae9e0 [id=31c0ee69dd9a473aada01e871ac80e81]) successfully deleted
I20260202 02:52:13.872913 1175 raft_consensus.cc:385] T 5d8de32fc94d4835862fc133c6b442ef P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:13.873155 1180 log.cc:1199] T 5d8de32fc94d4835862fc133c6b442ef P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/5d8de32fc94d4835862fc133c6b442ef
I20260202 02:52:13.873296 1183 ts_tablet_manager.cc:1916] T 5d8de32fc94d4835862fc133c6b442ef P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:13.873306 1175 raft_consensus.cc:740] T 5d8de32fc94d4835862fc133c6b442ef P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:13.873970 1180 ts_tablet_manager.cc:1950] T 5d8de32fc94d4835862fc133c6b442ef P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:13.874217 1175 consensus_queue.cc:260] T 5d8de32fc94d4835862fc133c6b442ef P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:13.875805 1173 tablet_bootstrap.cc:492] T ee6e1a7236694d5092b3c1320291be42 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:13.875999 1173 ts_tablet_manager.cc:1403] T ee6e1a7236694d5092b3c1320291be42 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.035s user 0.001s sys 0.003s
I20260202 02:52:13.876865 1175 ts_tablet_manager.cc:1434] T 5d8de32fc94d4835862fc133c6b442ef P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260202 02:52:13.876859 1173 raft_consensus.cc:359] T ee6e1a7236694d5092b3c1320291be42 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:13.872539 1174 tablet_bootstrap.cc:492] T ee6e1a7236694d5092b3c1320291be42 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:13.877059 1173 raft_consensus.cc:385] T ee6e1a7236694d5092b3c1320291be42 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:13.877147 1173 raft_consensus.cc:740] T ee6e1a7236694d5092b3c1320291be42 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:13.877382 1173 consensus_queue.cc:260] T ee6e1a7236694d5092b3c1320291be42 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:13.877866 1173 ts_tablet_manager.cc:1434] T ee6e1a7236694d5092b3c1320291be42 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:13.878156 1173 tablet_bootstrap.cc:492] T 6bac811b0f9f4276ac63786d97f02188 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:13.879813 1173 tablet_bootstrap.cc:654] T 6bac811b0f9f4276ac63786d97f02188 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:13.881502 1174 tablet_bootstrap.cc:654] T ee6e1a7236694d5092b3c1320291be42 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:13.882009 1175 tablet_bootstrap.cc:492] T ee6e1a7236694d5092b3c1320291be42 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
W20260202 02:52:13.882082 524 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ea0b6bb727404522b8291cbcd12879ea consensus state has no leader
I20260202 02:52:13.884536 520 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet b9ef98be77c54c29b1c27caf836b2a97
I20260202 02:52:13.885783 1175 tablet_bootstrap.cc:654] T ee6e1a7236694d5092b3c1320291be42 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:13.885882 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 5d8de32fc94d4835862fc133c6b442ef (table default.Table_2f7cf4b6f60845c7977bbb0b5f2b7fd6 [id=1b7d3c09f6b84bd8b8634635c0db2295]) successfully deleted
W20260202 02:52:13.886704 527 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 62241ababede41a1bd7f91620809f5f8 consensus state has no leader
W20260202 02:52:13.887529 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet ee6e1a7236694d5092b3c1320291be42 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ee6e1a7236694d5092b3c1320291be42 already in progress: creating tablet
W20260202 02:52:13.888000 526 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6bac811b0f9f4276ac63786d97f02188 consensus state has no leader
W20260202 02:52:13.888904 521 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 397ed14bb33e44e59536243d0fa7e459 has no consensus state
I20260202 02:52:13.890771 525 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet 7b73586ee49c49ceacf76b0a93a919ba
I20260202 02:52:13.891548 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 51c11ad251924fb7b126465932a2e74d with delete_type TABLET_DATA_DELETED (replaced by tablet ea0b6bb727404522b8291cbcd12879ea) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:13.893139 1180 tablet_replica.cc:333] T 51c11ad251924fb7b126465932a2e74d P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:13.893417 1180 raft_consensus.cc:2243] T 51c11ad251924fb7b126465932a2e74d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:13.893663 1180 raft_consensus.cc:2272] T 51c11ad251924fb7b126465932a2e74d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:13.894594 1183 ts_tablet_manager.cc:1929] T 5d8de32fc94d4835862fc133c6b442ef P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:13.894616 1180 ts_tablet_manager.cc:1916] T 51c11ad251924fb7b126465932a2e74d P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:13.894774 1183 log.cc:1199] T 5d8de32fc94d4835862fc133c6b442ef P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/5d8de32fc94d4835862fc133c6b442ef
I20260202 02:52:13.895201 1183 ts_tablet_manager.cc:1950] T 5d8de32fc94d4835862fc133c6b442ef P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:13.896184 1183 tablet_replica.cc:333] T ee6e1a7236694d5092b3c1320291be42 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:13.896405 1183 raft_consensus.cc:2243] T ee6e1a7236694d5092b3c1320291be42 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:13.896605 1183 raft_consensus.cc:2272] T ee6e1a7236694d5092b3c1320291be42 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:13.897411 1183 ts_tablet_manager.cc:1916] T ee6e1a7236694d5092b3c1320291be42 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:13.903628 528 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 7781ccb7f1ea437fb1d61d3625a739a9
I20260202 02:52:13.903892 522 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet 684bed0c6d7541d9bd8510ab209cd11c
I20260202 02:52:13.904253 523 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet 5cb8246c13b7487899f6521cee305d66
I20260202 02:52:13.905232 901 tablet_service.cc:1552] Processing DeleteTablet for tablet 7781ccb7f1ea437fb1d61d3625a739a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:13 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:13.905867 901 tablet_service.cc:1552] Processing DeleteTablet for tablet 5d8de32fc94d4835862fc133c6b442ef with delete_type TABLET_DATA_DELETED (replaced by tablet 7781ccb7f1ea437fb1d61d3625a739a9) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:13.905602 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 7781ccb7f1ea437fb1d61d3625a739a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:13 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:13.906469 1182 tablet_replica.cc:333] T 5d8de32fc94d4835862fc133c6b442ef P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:13.906674 1182 raft_consensus.cc:2243] T 5d8de32fc94d4835862fc133c6b442ef P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:13.906852 1182 raft_consensus.cc:2272] T 5d8de32fc94d4835862fc133c6b442ef P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:13.907994 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 7781ccb7f1ea437fb1d61d3625a739a9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7781ccb7f1ea437fb1d61d3625a739a9 already in progress: creating tablet
I20260202 02:52:13.908255 1183 ts_tablet_manager.cc:1929] T ee6e1a7236694d5092b3c1320291be42 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:13.908876 1182 ts_tablet_manager.cc:1916] T 5d8de32fc94d4835862fc133c6b442ef P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:13.909915 1183 log.cc:1199] T ee6e1a7236694d5092b3c1320291be42 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/ee6e1a7236694d5092b3c1320291be42
I20260202 02:52:13.910333 1183 ts_tablet_manager.cc:1950] T ee6e1a7236694d5092b3c1320291be42 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:13.910643 1042 tablet_service.cc:1552] Processing DeleteTablet for tablet 7781ccb7f1ea437fb1d61d3625a739a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:13 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:13.918851 1175 tablet_bootstrap.cc:492] T ee6e1a7236694d5092b3c1320291be42 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:13.919232 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet ee6e1a7236694d5092b3c1320291be42 (table default.Table_011de32f5bb648439181d1154b82b2c5 [id=1d7c11d3ee26455e995b30f69a967132]) successfully deleted
W20260202 02:52:13.919492 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 7781ccb7f1ea437fb1d61d3625a739a9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7781ccb7f1ea437fb1d61d3625a739a9 already in progress: creating tablet
I20260202 02:52:13.919742 982 raft_consensus.cc:493] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92a8191cf1e14e4d99f1c378c2d3c85b)
I20260202 02:52:13.919910 982 raft_consensus.cc:515] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } }
I20260202 02:52:13.924470 982 leader_election.cc:290] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7dab2abef14445d688ce1335b1f9b2f8 (127.0.104.190:33981), 92a8191cf1e14e4d99f1c378c2d3c85b (127.0.104.189:46271)
I20260202 02:52:13.925119 756 tablet_service.cc:1505] Processing CreateTablet for tablet 9c52693bdd214919b42cee1bbba95ed2 (DEFAULT_TABLE table=default.Table_f03b4b80c09b4362b51aba61a6beb163 [id=92e6422d4fc3439abab376a52ad8876a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.929075 538 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" is_pre_election: true
I20260202 02:52:13.931378 467 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7dab2abef14445d688ce1335b1f9b2f8" is_pre_election: true
I20260202 02:52:13.932852 1042 tablet_service.cc:1505] Processing CreateTablet for tablet 9c52693bdd214919b42cee1bbba95ed2 (DEFAULT_TABLE table=default.Table_f03b4b80c09b4362b51aba61a6beb163 [id=92e6422d4fc3439abab376a52ad8876a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.933347 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c52693bdd214919b42cee1bbba95ed2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.934006 586 leader_election.cc:304] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9c3e3612bb94fb7b94c308041acfaeb; no voters: 7dab2abef14445d688ce1335b1f9b2f8, 92a8191cf1e14e4d99f1c378c2d3c85b
I20260202 02:52:13.936501 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c52693bdd214919b42cee1bbba95ed2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.930166 902 tablet_service.cc:1505] Processing CreateTablet for tablet c9ce1b27212646e691bad953c9fbe14d (DEFAULT_TABLE table=default.Table_02df7f59f31c4259844bf3dd916f0c35 [id=9a8a252de32848288c9b3752c7d3feb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.936714 1175 ts_tablet_manager.cc:1403] T ee6e1a7236694d5092b3c1320291be42 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.055s user 0.007s sys 0.001s
I20260202 02:52:13.937070 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9ce1b27212646e691bad953c9fbe14d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.937932 1175 raft_consensus.cc:359] T ee6e1a7236694d5092b3c1320291be42 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:13.938205 1175 raft_consensus.cc:385] T ee6e1a7236694d5092b3c1320291be42 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:13.938290 1175 raft_consensus.cc:740] T ee6e1a7236694d5092b3c1320291be42 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:13.938575 1175 consensus_queue.cc:260] T ee6e1a7236694d5092b3c1320291be42 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:13.938917 1182 ts_tablet_manager.cc:1929] T 5d8de32fc94d4835862fc133c6b442ef P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:13.939083 1182 log.cc:1199] T 5d8de32fc94d4835862fc133c6b442ef P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/5d8de32fc94d4835862fc133c6b442ef
I20260202 02:52:13.939520 1182 ts_tablet_manager.cc:1950] T 5d8de32fc94d4835862fc133c6b442ef P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:13.930714 903 tablet_service.cc:1505] Processing CreateTablet for tablet 397ed14bb33e44e59536243d0fa7e459 (DEFAULT_TABLE table=default.Table_62840640773d47f5889c6a82f84ffb32 [id=583762f37e29416886d3cc6219f7fd45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.931201 900 tablet_service.cc:1505] Processing CreateTablet for tablet 16713353f2684b9d96c80a3abbcdf74b (DEFAULT_TABLE table=default.Table_2475d75250c24b3ea51238f58fd5f32b [id=2cc4567c7bd44745a5851cd8b95b22c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.930168 899 tablet_service.cc:1505] Processing CreateTablet for tablet 220c3b32f56c478fbca7fbbd6135290c (DEFAULT_TABLE table=default.Table_c5a4aa6f50fb472c8982feedb30982a9 [id=7af0d6d84db7404ebed9db660a633c12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.932608 901 tablet_service.cc:1505] Processing CreateTablet for tablet 9c52693bdd214919b42cee1bbba95ed2 (DEFAULT_TABLE table=default.Table_f03b4b80c09b4362b51aba61a6beb163 [id=92e6422d4fc3439abab376a52ad8876a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.940686 903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 397ed14bb33e44e59536243d0fa7e459. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.941439 1173 tablet_bootstrap.cc:492] T 6bac811b0f9f4276ac63786d97f02188 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:13.941394 1043 tablet_service.cc:1505] Processing CreateTablet for tablet 397ed14bb33e44e59536243d0fa7e459 (DEFAULT_TABLE table=default.Table_62840640773d47f5889c6a82f84ffb32 [id=583762f37e29416886d3cc6219f7fd45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.941577 1173 ts_tablet_manager.cc:1403] T 6bac811b0f9f4276ac63786d97f02188 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.064s user 0.015s sys 0.030s
I20260202 02:52:13.941836 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 397ed14bb33e44e59536243d0fa7e459. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.942487 1173 raft_consensus.cc:359] T 6bac811b0f9f4276ac63786d97f02188 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:13.942689 1173 raft_consensus.cc:385] T 6bac811b0f9f4276ac63786d97f02188 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:13.942780 1173 raft_consensus.cc:740] T 6bac811b0f9f4276ac63786d97f02188 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:13.944031 1175 ts_tablet_manager.cc:1434] T ee6e1a7236694d5092b3c1320291be42 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.007s user 0.002s sys 0.001s
I20260202 02:52:13.944711 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 16713353f2684b9d96c80a3abbcdf74b (DEFAULT_TABLE table=default.Table_2475d75250c24b3ea51238f58fd5f32b [id=2cc4567c7bd44745a5851cd8b95b22c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.945153 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16713353f2684b9d96c80a3abbcdf74b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.949127 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 220c3b32f56c478fbca7fbbd6135290c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.949290 982 raft_consensus.cc:2749] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260202 02:52:13.949803 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 5d8de32fc94d4835862fc133c6b442ef (table default.Table_2f7cf4b6f60845c7977bbb0b5f2b7fd6 [id=1b7d3c09f6b84bd8b8634635c0db2295]) successfully deleted
I20260202 02:52:13.950104 898 tablet_service.cc:1552] Processing DeleteTablet for tablet ee6e1a7236694d5092b3c1320291be42 with delete_type TABLET_DATA_DELETED (replaced by tablet a6940c7f38ff496abf3c2dab66da5100) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:13.954636 524 master_service.cc:954] ReplaceTablet: received request to replace tablet 7781ccb7f1ea437fb1d61d3625a739a9 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:13.955729 524 master_service.cc:954] ReplaceTablet: received request to replace tablet ee6e1a7236694d5092b3c1320291be42 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:13.957116 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16713353f2684b9d96c80a3abbcdf74b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.960134 1175 tablet_bootstrap.cc:492] T 6bac811b0f9f4276ac63786d97f02188 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:13.960628 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c52693bdd214919b42cee1bbba95ed2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.961803 1182 tablet_replica.cc:333] T ee6e1a7236694d5092b3c1320291be42 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:13.962028 1182 raft_consensus.cc:2243] T ee6e1a7236694d5092b3c1320291be42 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:13.962251 1182 raft_consensus.cc:2272] T ee6e1a7236694d5092b3c1320291be42 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:13.962611 1175 tablet_bootstrap.cc:654] T 6bac811b0f9f4276ac63786d97f02188 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:13.963601 418 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskYXxklX/build/asan/bin/kudu with pid 565
I20260202 02:52:13.964612 758 tablet_service.cc:1505] Processing CreateTablet for tablet c9ce1b27212646e691bad953c9fbe14d (DEFAULT_TABLE table=default.Table_02df7f59f31c4259844bf3dd916f0c35 [id=9a8a252de32848288c9b3752c7d3feb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.965167 1042 tablet_service.cc:1505] Processing CreateTablet for tablet c9ce1b27212646e691bad953c9fbe14d (DEFAULT_TABLE table=default.Table_02df7f59f31c4259844bf3dd916f0c35 [id=9a8a252de32848288c9b3752c7d3feb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.965313 1174 tablet_bootstrap.cc:492] T ee6e1a7236694d5092b3c1320291be42 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:13.965741 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9ce1b27212646e691bad953c9fbe14d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.966251 1174 ts_tablet_manager.cc:1403] T ee6e1a7236694d5092b3c1320291be42 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.093s user 0.001s sys 0.006s
I20260202 02:52:13.966830 1041 tablet_service.cc:1505] Processing CreateTablet for tablet 220c3b32f56c478fbca7fbbd6135290c (DEFAULT_TABLE table=default.Table_c5a4aa6f50fb472c8982feedb30982a9 [id=7af0d6d84db7404ebed9db660a633c12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.967082 1174 raft_consensus.cc:359] T ee6e1a7236694d5092b3c1320291be42 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:13.967669 1174 raft_consensus.cc:385] T ee6e1a7236694d5092b3c1320291be42 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:13.967775 1174 raft_consensus.cc:740] T ee6e1a7236694d5092b3c1320291be42 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:13.968693 1174 consensus_queue.cc:260] T ee6e1a7236694d5092b3c1320291be42 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:13.971485 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9ce1b27212646e691bad953c9fbe14d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.971876 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 220c3b32f56c478fbca7fbbd6135290c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.973043 757 tablet_service.cc:1505] Processing CreateTablet for tablet 16713353f2684b9d96c80a3abbcdf74b (DEFAULT_TABLE table=default.Table_2475d75250c24b3ea51238f58fd5f32b [id=2cc4567c7bd44745a5851cd8b95b22c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.973517 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16713353f2684b9d96c80a3abbcdf74b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.975448 755 tablet_service.cc:1505] Processing CreateTablet for tablet 397ed14bb33e44e59536243d0fa7e459 (DEFAULT_TABLE table=default.Table_62840640773d47f5889c6a82f84ffb32 [id=583762f37e29416886d3cc6219f7fd45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.975699 903 tablet_service.cc:1505] Processing CreateTablet for tablet 80eb3013f5354022a2dd78bb294a8497 (DEFAULT_TABLE table=default.Table_ab0c372e2fbc439cbfc95db9aa9c75d0 [id=316b8a276d9b415fafe8511aa7c32fc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.976146 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 397ed14bb33e44e59536243d0fa7e459. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.977996 1182 ts_tablet_manager.cc:1916] T ee6e1a7236694d5092b3c1320291be42 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:13.978206 754 tablet_service.cc:1505] Processing CreateTablet for tablet 80eb3013f5354022a2dd78bb294a8497 (DEFAULT_TABLE table=default.Table_ab0c372e2fbc439cbfc95db9aa9c75d0 [id=316b8a276d9b415fafe8511aa7c32fc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.978721 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80eb3013f5354022a2dd78bb294a8497. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.978925 903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80eb3013f5354022a2dd78bb294a8497. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.984419 759 tablet_service.cc:1505] Processing CreateTablet for tablet 220c3b32f56c478fbca7fbbd6135290c (DEFAULT_TABLE table=default.Table_c5a4aa6f50fb472c8982feedb30982a9 [id=7af0d6d84db7404ebed9db660a633c12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.984544 1173 consensus_queue.cc:260] T 6bac811b0f9f4276ac63786d97f02188 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:13.984855 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 220c3b32f56c478fbca7fbbd6135290c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.987736 1173 ts_tablet_manager.cc:1434] T 6bac811b0f9f4276ac63786d97f02188 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.046s user 0.003s sys 0.000s
I20260202 02:52:13.988108 1173 tablet_bootstrap.cc:492] T 62241ababede41a1bd7f91620809f5f8 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:13.989774 1041 tablet_service.cc:1505] Processing CreateTablet for tablet 80eb3013f5354022a2dd78bb294a8497 (DEFAULT_TABLE table=default.Table_ab0c372e2fbc439cbfc95db9aa9c75d0 [id=316b8a276d9b415fafe8511aa7c32fc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:13.990386 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80eb3013f5354022a2dd78bb294a8497. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:13.999346 523 master_service.cc:954] ReplaceTablet: received request to replace tablet 7781ccb7f1ea437fb1d61d3625a739a9 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:14.001816 524 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 6bac811b0f9f4276ac63786d97f02188 (table default.Table_5b9ebdf0ffec42f88f5bd9a3d0d158e9 [id=01b7ed0408cb46cabd569c0cf240bbaa]). Expected version 2 got 0
W20260202 02:52:14.002877 713 connection.cc:537] client connection to 127.0.104.188:39077 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260202 02:52:14.003736 824 heartbeater.cc:646] Failed to heartbeat to 127.0.104.188:39077 (0 consecutive failures): Network error: Failed to send heartbeat to master: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260202 02:52:14.004734 968 heartbeater.cc:646] Failed to heartbeat to 127.0.104.188:39077 (0 consecutive failures): Network error: Failed to send heartbeat to master: recv got EOF from 127.0.104.188:39077 (error 108)
I20260202 02:52:14.002391 523 master_service.cc:954] ReplaceTablet: received request to replace tablet c9ce1b27212646e691bad953c9fbe14d from {username='slave'} at 127.0.0.1:47430
W20260202 02:52:13.998881 998 connection.cc:537] client connection to 127.0.104.188:39077 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260202 02:52:14.007335 1109 heartbeater.cc:646] Failed to heartbeat to 127.0.104.188:39077 (0 consecutive failures): Network error: Failed to send heartbeat to master: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260202 02:52:14.009836 1175 tablet_bootstrap.cc:492] T 6bac811b0f9f4276ac63786d97f02188 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:14.010016 1175 ts_tablet_manager.cc:1403] T 6bac811b0f9f4276ac63786d97f02188 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.050s user 0.011s sys 0.000s
I20260202 02:52:14.010955 1175 raft_consensus.cc:359] T 6bac811b0f9f4276ac63786d97f02188 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.011212 1175 raft_consensus.cc:385] T 6bac811b0f9f4276ac63786d97f02188 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.011318 1175 raft_consensus.cc:740] T 6bac811b0f9f4276ac63786d97f02188 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.011570 1175 consensus_queue.cc:260] T 6bac811b0f9f4276ac63786d97f02188 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.012234 1175 ts_tablet_manager.cc:1434] T 6bac811b0f9f4276ac63786d97f02188 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260202 02:52:14.012609 1174 ts_tablet_manager.cc:1434] T ee6e1a7236694d5092b3c1320291be42 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.025s user 0.001s sys 0.000s
I20260202 02:52:14.012986 1175 tablet_bootstrap.cc:492] T 62241ababede41a1bd7f91620809f5f8 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:14.014088 524 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 6bac811b0f9f4276ac63786d97f02188 (table default.Table_5b9ebdf0ffec42f88f5bd9a3d0d158e9 [id=01b7ed0408cb46cabd569c0cf240bbaa]). Expected version 2 got 0
I20260202 02:52:13.998723 1173 tablet_bootstrap.cc:654] T 62241ababede41a1bd7f91620809f5f8 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.016062 528 master_service.cc:954] ReplaceTablet: received request to replace tablet 7781ccb7f1ea437fb1d61d3625a739a9 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:14.016695 1182 ts_tablet_manager.cc:1929] T ee6e1a7236694d5092b3c1320291be42 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:14.016805 1175 tablet_bootstrap.cc:654] T 62241ababede41a1bd7f91620809f5f8 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.016954 1182 log.cc:1199] T ee6e1a7236694d5092b3c1320291be42 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/ee6e1a7236694d5092b3c1320291be42
I20260202 02:52:14.017433 1182 ts_tablet_manager.cc:1950] T ee6e1a7236694d5092b3c1320291be42 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:14.018221 1174 tablet_bootstrap.cc:492] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:14.020983 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet ee6e1a7236694d5092b3c1320291be42 (table default.Table_011de32f5bb648439181d1154b82b2c5 [id=1d7c11d3ee26455e995b30f69a967132]) successfully deleted
I20260202 02:52:14.023962 523 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet c9ce1b27212646e691bad953c9fbe14d
I20260202 02:52:14.024524 523 catalog_manager.cc:6724] ReplaceTablet: tablet c9ce1b27212646e691bad953c9fbe14d deleted and replaced by tablet 2730d8b6812e487ea0aff4da45d24044
I20260202 02:52:14.024957 900 tablet_service.cc:1552] Processing DeleteTablet for tablet c9ce1b27212646e691bad953c9fbe14d with delete_type TABLET_DATA_DELETED (replaced by tablet 2730d8b6812e487ea0aff4da45d24044) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:14.025681 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet c9ce1b27212646e691bad953c9fbe14d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c9ce1b27212646e691bad953c9fbe14d already in progress: creating tablet
I20260202 02:52:14.026453 1042 tablet_service.cc:1552] Processing DeleteTablet for tablet c9ce1b27212646e691bad953c9fbe14d with delete_type TABLET_DATA_DELETED (replaced by tablet 2730d8b6812e487ea0aff4da45d24044) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:14.027112 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet c9ce1b27212646e691bad953c9fbe14d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c9ce1b27212646e691bad953c9fbe14d already in progress: creating tablet
I20260202 02:52:14.028453 1173 tablet_bootstrap.cc:492] T 62241ababede41a1bd7f91620809f5f8 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:14.028641 1173 ts_tablet_manager.cc:1403] T 62241ababede41a1bd7f91620809f5f8 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.041s user 0.003s sys 0.000s
W20260202 02:52:14.028846 1195 negotiation.cc:337] Failed RPC negotiation. Trace:
0202 02:52:14.007840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.104.188:39077 (local address 127.0.104.131:35545)
0202 02:52:14.021324 (+ 13484us) negotiation.cc:107] Waiting for socket to connect
0202 02:52:14.021472 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.104.188:39077: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":13367,"thread_start_us":13591,"threads_started":1}
I20260202 02:52:14.029609 1173 raft_consensus.cc:359] T 62241ababede41a1bd7f91620809f5f8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.029783 1173 raft_consensus.cc:385] T 62241ababede41a1bd7f91620809f5f8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.029856 1173 raft_consensus.cc:740] T 62241ababede41a1bd7f91620809f5f8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.030087 1173 consensus_queue.cc:260] T 62241ababede41a1bd7f91620809f5f8 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.030709 1173 ts_tablet_manager.cc:1434] T 62241ababede41a1bd7f91620809f5f8 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:14.031037 1173 tablet_bootstrap.cc:492] T ea0b6bb727404522b8291cbcd12879ea P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:14.032197 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_3a7da0df67ae4429a45f8d8ac84491f1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:14.032840 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3a7da0df67ae4429a45f8d8ac84491f1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.033286 1173 tablet_bootstrap.cc:654] T ea0b6bb727404522b8291cbcd12879ea P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.037068 528 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 6bac811b0f9f4276ac63786d97f02188 (table default.Table_5b9ebdf0ffec42f88f5bd9a3d0d158e9 [id=01b7ed0408cb46cabd569c0cf240bbaa]). Expected version 2 got 0
I20260202 02:52:14.042971 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_90124527a3ca45d3b1c779818516993f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:14.043629 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_90124527a3ca45d3b1c779818516993f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.050455 1180 ts_tablet_manager.cc:1929] T 51c11ad251924fb7b126465932a2e74d P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:14.050642 1180 log.cc:1199] T 51c11ad251924fb7b126465932a2e74d P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/51c11ad251924fb7b126465932a2e74d
I20260202 02:52:14.051030 1180 ts_tablet_manager.cc:1950] T 51c11ad251924fb7b126465932a2e74d P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:14.051435 900 tablet_service.cc:1505] Processing CreateTablet for tablet 2730d8b6812e487ea0aff4da45d24044 (DEFAULT_TABLE table=default.Table_02df7f59f31c4259844bf3dd916f0c35 [id=9a8a252de32848288c9b3752c7d3feb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.051949 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2730d8b6812e487ea0aff4da45d24044. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.055423 1174 tablet_bootstrap.cc:654] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.059756 1041 tablet_service.cc:1505] Processing CreateTablet for tablet 2730d8b6812e487ea0aff4da45d24044 (DEFAULT_TABLE table=default.Table_02df7f59f31c4259844bf3dd916f0c35 [id=9a8a252de32848288c9b3752c7d3feb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.060341 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2730d8b6812e487ea0aff4da45d24044. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.060541 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_ca030d309ab3480d8fa40b2cc071b79a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:14.061110 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ca030d309ab3480d8fa40b2cc071b79a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.062505 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_e6d3c2dc7ad1416c949573796c5dc398"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:14.063112 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e6d3c2dc7ad1416c949573796c5dc398 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.064453 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_9fd715491a6a4bbc8685138d0194362d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:14.065044 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9fd715491a6a4bbc8685138d0194362d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.066375 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_fc38149c352143958e0fb729fd754e2e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:14.066960 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fc38149c352143958e0fb729fd754e2e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.069420 1174 tablet_bootstrap.cc:492] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:14.069589 1174 ts_tablet_manager.cc:1403] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.053s user 0.002s sys 0.004s
I20260202 02:52:14.070431 1174 raft_consensus.cc:359] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.070659 1174 raft_consensus.cc:385] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.070747 1174 raft_consensus.cc:740] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.071009 1174 consensus_queue.cc:260] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.073926 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_0b2c7ff473964e0da5c026225b766340"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:14.074398 1174 ts_tablet_manager.cc:1434] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
W20260202 02:52:14.074579 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0b2c7ff473964e0da5c026225b766340 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.075873 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_629c3da327a740f684072799e959c2ca"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:14.060237 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_15c57b6d38ab4f2fb7dca2a405f01058"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:14.077737 1174 tablet_bootstrap.cc:492] T 62241ababede41a1bd7f91620809f5f8 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
W20260202 02:52:14.077968 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_15c57b6d38ab4f2fb7dca2a405f01058 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.078704 522 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 62241ababede41a1bd7f91620809f5f8 (table default.Table_4a3ef984ca894b8b8f4301fecb719b5b [id=3ccb047c30974c0f9cbe778971e61644]). Expected version 2 got 0
I20260202 02:52:14.079505 1174 tablet_bootstrap.cc:654] T 62241ababede41a1bd7f91620809f5f8 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.080245 528 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 6bac811b0f9f4276ac63786d97f02188 (table default.Table_5b9ebdf0ffec42f88f5bd9a3d0d158e9 [id=01b7ed0408cb46cabd569c0cf240bbaa]). Expected version 2 got 0
W20260202 02:52:14.081247 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_629c3da327a740f684072799e959c2ca in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.082782 528 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_f03b4b80c09b4362b51aba61a6beb163" } new_table_name: "default.Table_3c631c97b1034c4ebe744d09da22559f" modify_external_catalogs: true
I20260202 02:52:14.085971 524 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet 6bac811b0f9f4276ac63786d97f02188 (table default.Table_5b9ebdf0ffec42f88f5bd9a3d0d158e9 [id=01b7ed0408cb46cabd569c0cf240bbaa]). Expected version 2 got 0
W20260202 02:52:14.086349 513 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.188:39077, user_credentials={real_user=slave}} blocked reactor thread for 83987.8us
I20260202 02:52:14.087313 758 tablet_service.cc:1552] Processing DeleteTablet for tablet ee6e1a7236694d5092b3c1320291be42 with delete_type TABLET_DATA_DELETED (replaced by tablet a6940c7f38ff496abf3c2dab66da5100) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:14.087600 754 tablet_service.cc:1552] Processing DeleteTablet for tablet c9ce1b27212646e691bad953c9fbe14d with delete_type TABLET_DATA_DELETED (replaced by tablet 2730d8b6812e487ea0aff4da45d24044) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:14.087880 1180 tablet_replica.cc:333] T ee6e1a7236694d5092b3c1320291be42 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:14.088078 1180 raft_consensus.cc:2243] T ee6e1a7236694d5092b3c1320291be42 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:14.088296 1180 raft_consensus.cc:2272] T ee6e1a7236694d5092b3c1320291be42 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.088413 754 tablet_service.cc:1505] Processing CreateTablet for tablet 2730d8b6812e487ea0aff4da45d24044 (DEFAULT_TABLE table=default.Table_02df7f59f31c4259844bf3dd916f0c35 [id=9a8a252de32848288c9b3752c7d3feb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.088697 758 tablet_service.cc:1505] Processing CreateTablet for tablet 005cda97ab5c4b70b8fc8ff66a05d996 (DEFAULT_TABLE table=default.Table_3a7da0df67ae4429a45f8d8ac84491f1 [id=4e076b2023be4c888ac1c27babde1607]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.088858 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2730d8b6812e487ea0aff4da45d24044. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.089046 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 005cda97ab5c4b70b8fc8ff66a05d996. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.090466 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 51c11ad251924fb7b126465932a2e74d (table default.Table_c86dacfe4afb42ed9f08fa8a2a6ecc75 [id=31c0ee69dd9a473aada01e871ac80e81]) successfully deleted
W20260202 02:52:14.090708 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 7781ccb7f1ea437fb1d61d3625a739a9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7781ccb7f1ea437fb1d61d3625a739a9 already in progress: creating tablet
W20260202 02:52:14.090904 513 consensus_peers.cc:597] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b -> Peer a9c3e3612bb94fb7b94c308041acfaeb (127.0.104.188:39077): Couldn't send request to peer a9c3e3612bb94fb7b94c308041acfaeb. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.188:39077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:14.093279 1042 tablet_service.cc:1505] Processing CreateTablet for tablet 005cda97ab5c4b70b8fc8ff66a05d996 (DEFAULT_TABLE table=default.Table_3a7da0df67ae4429a45f8d8ac84491f1 [id=4e076b2023be4c888ac1c27babde1607]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.093722 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 005cda97ab5c4b70b8fc8ff66a05d996. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.093999 1181 raft_consensus.cc:493] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:14.094246 1181 raft_consensus.cc:515] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.096412 1181 leader_election.cc:290] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:14.097126 522 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_90124527a3ca45d3b1c779818516993f" } new_table_name: "default.Table_04527bd0309f433aa915767172dcd280" modify_external_catalogs: true
I20260202 02:52:14.098093 519 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_c5a4aa6f50fb472c8982feedb30982a9" } modify_external_catalogs: true
I20260202 02:52:14.098253 519 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_c5a4aa6f50fb472c8982feedb30982a9" } modify_external_catalogs: true
I20260202 02:52:14.098655 900 tablet_service.cc:1505] Processing CreateTablet for tablet 005cda97ab5c4b70b8fc8ff66a05d996 (DEFAULT_TABLE table=default.Table_3a7da0df67ae4429a45f8d8ac84491f1 [id=4e076b2023be4c888ac1c27babde1607]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.098743 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_3a7da0df67ae4429a45f8d8ac84491f1" } new_table_name: "default.Table_76150965bafb4889b2b08e618f28011a" modify_external_catalogs: true
I20260202 02:52:14.099102 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 005cda97ab5c4b70b8fc8ff66a05d996. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.100037 521 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_02df7f59f31c4259844bf3dd916f0c35" } new_table_name: "default.Table_506775b4d1a14bf1b91c9d951c9a88ec" modify_external_catalogs: true
I20260202 02:52:14.103233 1180 ts_tablet_manager.cc:1916] T ee6e1a7236694d5092b3c1320291be42 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.109213 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6bac811b0f9f4276ac63786d97f02188" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:14.109686 1064 raft_consensus.cc:2468] T 6bac811b0f9f4276ac63786d97f02188 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:14.110316 525 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_5b9ebdf0ffec42f88f5bd9a3d0d158e9" } modify_external_catalogs: true
I20260202 02:52:14.110502 525 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_5b9ebdf0ffec42f88f5bd9a3d0d158e9" } modify_external_catalogs: true
I20260202 02:52:14.113569 526 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_fc38149c352143958e0fb729fd754e2e" } new_table_name: "default.Table_692487d5d3454bd28e0229336d7779a3" modify_external_catalogs: true
I20260202 02:52:14.115134 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_08b43dcf669342bbb6a9910f002d3c5b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:14.115722 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_08b43dcf669342bbb6a9910f002d3c5b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.116243 1180 ts_tablet_manager.cc:1929] T ee6e1a7236694d5092b3c1320291be42 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:14.117287 1173 tablet_bootstrap.cc:492] T ea0b6bb727404522b8291cbcd12879ea P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:14.117551 1173 ts_tablet_manager.cc:1403] T ea0b6bb727404522b8291cbcd12879ea P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.087s user 0.005s sys 0.002s
W20260202 02:52:14.118611 527 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 005cda97ab5c4b70b8fc8ff66a05d996 consensus state has no leader
I20260202 02:52:14.118657 1180 log.cc:1199] T ee6e1a7236694d5092b3c1320291be42 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/ee6e1a7236694d5092b3c1320291be42
I20260202 02:52:14.118574 1173 raft_consensus.cc:359] T ea0b6bb727404522b8291cbcd12879ea P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.118800 1173 raft_consensus.cc:385] T ea0b6bb727404522b8291cbcd12879ea P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.118878 1173 raft_consensus.cc:740] T ea0b6bb727404522b8291cbcd12879ea P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.117470 715 leader_election.cc:304] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:14.119067 1180 ts_tablet_manager.cc:1950] T ee6e1a7236694d5092b3c1320291be42 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:14.119118 1173 consensus_queue.cc:260] T ea0b6bb727404522b8291cbcd12879ea P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.119383 1181 raft_consensus.cc:2804] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:14.119540 1181 raft_consensus.cc:493] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:14.119647 1181 raft_consensus.cc:3060] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:14.121639 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6bac811b0f9f4276ac63786d97f02188" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:14.122131 923 raft_consensus.cc:2468] T 6bac811b0f9f4276ac63786d97f02188 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:14.123447 1181 raft_consensus.cc:515] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.124276 1173 ts_tablet_manager.cc:1434] T ea0b6bb727404522b8291cbcd12879ea P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260202 02:52:14.124466 520 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_c86dacfe4afb42ed9f08fa8a2a6ecc75" } modify_external_catalogs: true
I20260202 02:52:14.124675 520 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_c86dacfe4afb42ed9f08fa8a2a6ecc75" } modify_external_catalogs: true
W20260202 02:52:14.124997 528 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9c52693bdd214919b42cee1bbba95ed2 consensus state has no leader
I20260202 02:52:14.125365 528 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_62840640773d47f5889c6a82f84ffb32" } modify_external_catalogs: true
I20260202 02:52:14.125509 528 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_62840640773d47f5889c6a82f84ffb32" } modify_external_catalogs: true
I20260202 02:52:14.125855 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6bac811b0f9f4276ac63786d97f02188" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:14.126058 923 raft_consensus.cc:3060] T 6bac811b0f9f4276ac63786d97f02188 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:14.128201 1173 tablet_bootstrap.cc:492] T 7781ccb7f1ea437fb1d61d3625a739a9 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:14.129712 923 raft_consensus.cc:2468] T 6bac811b0f9f4276ac63786d97f02188 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:14.130654 1173 tablet_bootstrap.cc:654] T 7781ccb7f1ea437fb1d61d3625a739a9 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:14.131603 522 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6399b63cd8974ee49796f8779f9abc0e has no consensus state
I20260202 02:52:14.131985 522 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_011de32f5bb648439181d1154b82b2c5" } modify_external_catalogs: true
I20260202 02:52:14.132182 522 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_011de32f5bb648439181d1154b82b2c5" } modify_external_catalogs: true
I20260202 02:52:14.132310 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet ee6e1a7236694d5092b3c1320291be42 (table default.Table_011de32f5bb648439181d1154b82b2c5 [id=1d7c11d3ee26455e995b30f69a967132]) successfully deleted
W20260202 02:52:14.135085 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet c9ce1b27212646e691bad953c9fbe14d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c9ce1b27212646e691bad953c9fbe14d already in progress: creating tablet
I20260202 02:52:14.134652 527 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet ea0b6bb727404522b8291cbcd12879ea (table default.Table_c86dacfe4afb42ed9f08fa8a2a6ecc75 [id=31c0ee69dd9a473aada01e871ac80e81]). Expected version 2 got 0
I20260202 02:52:14.135687 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6bac811b0f9f4276ac63786d97f02188" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:14.135934 1064 raft_consensus.cc:3060] T 6bac811b0f9f4276ac63786d97f02188 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:14.136198 758 tablet_service.cc:1505] Processing CreateTablet for tablet 6399b63cd8974ee49796f8779f9abc0e (DEFAULT_TABLE table=default.Table_04527bd0309f433aa915767172dcd280 [id=dd2c220f5b88440295facca8da0b1752]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.136672 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6399b63cd8974ee49796f8779f9abc0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.137274 1181 leader_election.cc:290] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:14.137420 1173 tablet_bootstrap.cc:492] T 7781ccb7f1ea437fb1d61d3625a739a9 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:14.134208 900 tablet_service.cc:1505] Processing CreateTablet for tablet 6399b63cd8974ee49796f8779f9abc0e (DEFAULT_TABLE table=default.Table_04527bd0309f433aa915767172dcd280 [id=dd2c220f5b88440295facca8da0b1752]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.137593 1173 ts_tablet_manager.cc:1403] T 7781ccb7f1ea437fb1d61d3625a739a9 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.002s
I20260202 02:52:14.137852 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6399b63cd8974ee49796f8779f9abc0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.138453 1173 raft_consensus.cc:359] T 7781ccb7f1ea437fb1d61d3625a739a9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.138650 1173 raft_consensus.cc:385] T 7781ccb7f1ea437fb1d61d3625a739a9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.138715 1173 raft_consensus.cc:740] T 7781ccb7f1ea437fb1d61d3625a739a9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.139006 1173 consensus_queue.cc:260] T 7781ccb7f1ea437fb1d61d3625a739a9 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.139101 1064 raft_consensus.cc:2468] T 6bac811b0f9f4276ac63786d97f02188 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:14.139946 1041 tablet_service.cc:1505] Processing CreateTablet for tablet 6399b63cd8974ee49796f8779f9abc0e (DEFAULT_TABLE table=default.Table_04527bd0309f433aa915767172dcd280 [id=dd2c220f5b88440295facca8da0b1752]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.140439 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6399b63cd8974ee49796f8779f9abc0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.141568 1173 ts_tablet_manager.cc:1434] T 7781ccb7f1ea437fb1d61d3625a739a9 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260202 02:52:14.142961 1173 tablet_bootstrap.cc:492] T a6940c7f38ff496abf3c2dab66da5100 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:14.134655 524 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet 6bac811b0f9f4276ac63786d97f02188 (table default.Table_5b9ebdf0ffec42f88f5bd9a3d0d158e9 [id=01b7ed0408cb46cabd569c0cf240bbaa]). Expected version 2 got 0
I20260202 02:52:14.143630 714 leader_election.cc:304] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters:
I20260202 02:52:14.144315 1181 raft_consensus.cc:2804] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:14.134714 1175 tablet_bootstrap.cc:492] T 62241ababede41a1bd7f91620809f5f8 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:14.144842 1175 ts_tablet_manager.cc:1403] T 62241ababede41a1bd7f91620809f5f8 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.132s user 0.002s sys 0.013s
I20260202 02:52:14.145890 1175 raft_consensus.cc:359] T 62241ababede41a1bd7f91620809f5f8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.146093 1175 raft_consensus.cc:385] T 62241ababede41a1bd7f91620809f5f8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.146164 1175 raft_consensus.cc:740] T 62241ababede41a1bd7f91620809f5f8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.146895 1175 consensus_queue.cc:260] T 62241ababede41a1bd7f91620809f5f8 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.148602 1173 tablet_bootstrap.cc:654] T a6940c7f38ff496abf3c2dab66da5100 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:14.148619 521 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2730d8b6812e487ea0aff4da45d24044 consensus state has no leader
I20260202 02:52:14.149147 1175 ts_tablet_manager.cc:1434] T 62241ababede41a1bd7f91620809f5f8 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260202 02:52:14.149479 1175 tablet_bootstrap.cc:492] T ea0b6bb727404522b8291cbcd12879ea P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:14.151127 1175 tablet_bootstrap.cc:654] T ea0b6bb727404522b8291cbcd12879ea P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.152906 1174 tablet_bootstrap.cc:492] T 62241ababede41a1bd7f91620809f5f8 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:14.153187 1174 ts_tablet_manager.cc:1403] T 62241ababede41a1bd7f91620809f5f8 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.076s user 0.006s sys 0.018s
I20260202 02:52:14.154070 1174 raft_consensus.cc:359] T 62241ababede41a1bd7f91620809f5f8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.154376 1174 raft_consensus.cc:385] T 62241ababede41a1bd7f91620809f5f8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.154498 1174 raft_consensus.cc:740] T 62241ababede41a1bd7f91620809f5f8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.154816 1174 consensus_queue.cc:260] T 62241ababede41a1bd7f91620809f5f8 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
W20260202 02:52:14.156508 827 tablet.cc:2378] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260202 02:52:14.157094 521 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 62241ababede41a1bd7f91620809f5f8 (table default.Table_4a3ef984ca894b8b8f4301fecb719b5b [id=3ccb047c30974c0f9cbe778971e61644]). Expected version 2 got 0
I20260202 02:52:14.161880 1181 raft_consensus.cc:697] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:14.162117 521 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 62241ababede41a1bd7f91620809f5f8 (table default.Table_4a3ef984ca894b8b8f4301fecb719b5b [id=3ccb047c30974c0f9cbe778971e61644]). Expected version 2 got 0
I20260202 02:52:14.162489 1181 consensus_queue.cc:237] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.162889 521 master_service.cc:954] ReplaceTablet: received request to replace tablet 62241ababede41a1bd7f91620809f5f8 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:14.164815 1175 tablet_bootstrap.cc:492] T ea0b6bb727404522b8291cbcd12879ea P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:14.165086 1175 ts_tablet_manager.cc:1403] T ea0b6bb727404522b8291cbcd12879ea P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.001s
W20260202 02:52:14.165655 526 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fae7e3941c124cc2871390a0b6471f27 has no consensus state
I20260202 02:52:14.165966 526 master_service.cc:954] ReplaceTablet: received request to replace tablet a6940c7f38ff496abf3c2dab66da5100 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:14.166002 1175 raft_consensus.cc:359] T ea0b6bb727404522b8291cbcd12879ea P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.166213 1175 raft_consensus.cc:385] T ea0b6bb727404522b8291cbcd12879ea P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.166303 1175 raft_consensus.cc:740] T ea0b6bb727404522b8291cbcd12879ea P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.166566 1175 consensus_queue.cc:260] T ea0b6bb727404522b8291cbcd12879ea P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.168732 1173 tablet_bootstrap.cc:492] T a6940c7f38ff496abf3c2dab66da5100 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:14.168924 1173 ts_tablet_manager.cc:1403] T a6940c7f38ff496abf3c2dab66da5100 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.026s user 0.005s sys 0.001s
I20260202 02:52:14.169443 1175 ts_tablet_manager.cc:1434] T ea0b6bb727404522b8291cbcd12879ea P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260202 02:52:14.169736 1175 tablet_bootstrap.cc:492] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:14.169829 1173 raft_consensus.cc:359] T a6940c7f38ff496abf3c2dab66da5100 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.170042 1173 raft_consensus.cc:385] T a6940c7f38ff496abf3c2dab66da5100 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.170130 1173 raft_consensus.cc:740] T a6940c7f38ff496abf3c2dab66da5100 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.171546 1175 tablet_bootstrap.cc:654] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.172101 1173 consensus_queue.cc:260] T a6940c7f38ff496abf3c2dab66da5100 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
W20260202 02:52:14.176345 971 tablet.cc:2378] T 62241ababede41a1bd7f91620809f5f8 P 1521a6d262b7451bbef8debe970dd6b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260202 02:52:14.176496 1175 tablet_bootstrap.cc:492] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:14.176597 1173 ts_tablet_manager.cc:1434] T a6940c7f38ff496abf3c2dab66da5100 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.008s user 0.003s sys 0.000s
I20260202 02:52:14.176647 1175 ts_tablet_manager.cc:1403] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260202 02:52:14.177452 1175 raft_consensus.cc:359] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.177665 1175 raft_consensus.cc:385] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.177733 1175 raft_consensus.cc:740] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.177973 1175 consensus_queue.cc:260] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.178449 1175 ts_tablet_manager.cc:1434] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260202 02:52:14.178795 1175 tablet_bootstrap.cc:492] T a6940c7f38ff496abf3c2dab66da5100 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:14.179711 1173 tablet_bootstrap.cc:492] T 9c52693bdd214919b42cee1bbba95ed2 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:14.180840 1175 tablet_bootstrap.cc:654] T a6940c7f38ff496abf3c2dab66da5100 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.190490 1173 tablet_bootstrap.cc:654] T 9c52693bdd214919b42cee1bbba95ed2 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.192873 1174 ts_tablet_manager.cc:1434] T 62241ababede41a1bd7f91620809f5f8 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.040s user 0.026s sys 0.013s
I20260202 02:52:14.194088 1175 tablet_bootstrap.cc:492] T a6940c7f38ff496abf3c2dab66da5100 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:14.194350 1175 ts_tablet_manager.cc:1403] T a6940c7f38ff496abf3c2dab66da5100 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.007s
I20260202 02:52:14.194514 1174 tablet_bootstrap.cc:492] T ea0b6bb727404522b8291cbcd12879ea P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:14.195358 528 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 397ed14bb33e44e59536243d0fa7e459
I20260202 02:52:14.195331 1175 raft_consensus.cc:359] T a6940c7f38ff496abf3c2dab66da5100 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.195715 1175 raft_consensus.cc:385] T a6940c7f38ff496abf3c2dab66da5100 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.195854 1175 raft_consensus.cc:740] T a6940c7f38ff496abf3c2dab66da5100 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.196157 1175 consensus_queue.cc:260] T a6940c7f38ff496abf3c2dab66da5100 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.196273 1174 tablet_bootstrap.cc:654] T ea0b6bb727404522b8291cbcd12879ea P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.196677 900 tablet_service.cc:1552] Processing DeleteTablet for tablet 397ed14bb33e44e59536243d0fa7e459 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:14.197314 1175 ts_tablet_manager.cc:1434] T a6940c7f38ff496abf3c2dab66da5100 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260202 02:52:14.197414 758 tablet_service.cc:1552] Processing DeleteTablet for tablet 397ed14bb33e44e59536243d0fa7e459 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:14.197615 1175 tablet_bootstrap.cc:492] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
W20260202 02:52:14.198933 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 397ed14bb33e44e59536243d0fa7e459 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 397ed14bb33e44e59536243d0fa7e459 already in progress: creating tablet
I20260202 02:52:14.199493 1175 tablet_bootstrap.cc:654] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:14.199698 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 397ed14bb33e44e59536243d0fa7e459 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 397ed14bb33e44e59536243d0fa7e459 already in progress: creating tablet
I20260202 02:52:14.200527 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet 397ed14bb33e44e59536243d0fa7e459 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:14.201391 1173 tablet_bootstrap.cc:492] T 9c52693bdd214919b42cee1bbba95ed2 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:14.201573 1173 ts_tablet_manager.cc:1403] T 9c52693bdd214919b42cee1bbba95ed2 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.022s user 0.007s sys 0.000s
I20260202 02:52:14.202957 1173 raft_consensus.cc:359] T 9c52693bdd214919b42cee1bbba95ed2 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.203200 1173 raft_consensus.cc:385] T 9c52693bdd214919b42cee1bbba95ed2 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.203270 1173 raft_consensus.cc:740] T 9c52693bdd214919b42cee1bbba95ed2 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.203557 1173 consensus_queue.cc:260] T 9c52693bdd214919b42cee1bbba95ed2 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.203848 1174 tablet_bootstrap.cc:492] T ea0b6bb727404522b8291cbcd12879ea P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:14.204026 1174 ts_tablet_manager.cc:1403] T ea0b6bb727404522b8291cbcd12879ea P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260202 02:52:14.204838 1174 raft_consensus.cc:359] T ea0b6bb727404522b8291cbcd12879ea P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.205052 1174 raft_consensus.cc:385] T ea0b6bb727404522b8291cbcd12879ea P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.205143 1174 raft_consensus.cc:740] T ea0b6bb727404522b8291cbcd12879ea P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.205401 1174 consensus_queue.cc:260] T ea0b6bb727404522b8291cbcd12879ea P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.205921 1174 ts_tablet_manager.cc:1434] T ea0b6bb727404522b8291cbcd12879ea P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260202 02:52:14.206053 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 397ed14bb33e44e59536243d0fa7e459 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 397ed14bb33e44e59536243d0fa7e459 already in progress: creating tablet
I20260202 02:52:14.207899 1173 ts_tablet_manager.cc:1434] T 9c52693bdd214919b42cee1bbba95ed2 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260202 02:52:14.210014 1175 tablet_bootstrap.cc:492] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:14.210222 1175 ts_tablet_manager.cc:1403] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.005s
I20260202 02:52:14.210976 1175 raft_consensus.cc:359] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.211194 1175 raft_consensus.cc:385] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.211275 1175 raft_consensus.cc:740] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.211542 1175 consensus_queue.cc:260] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.212419 1174 tablet_bootstrap.cc:492] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:14.213078 1175 ts_tablet_manager.cc:1434] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260202 02:52:14.214250 1174 tablet_bootstrap.cc:654] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.214293 1175 tablet_bootstrap.cc:492] T 220c3b32f56c478fbca7fbbd6135290c P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:14.215085 522 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet a6940c7f38ff496abf3c2dab66da5100
I20260202 02:52:14.216112 519 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 220c3b32f56c478fbca7fbbd6135290c
I20260202 02:52:14.216182 521 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 62241ababede41a1bd7f91620809f5f8
I20260202 02:52:14.216598 521 catalog_manager.cc:6724] ReplaceTablet: tablet 62241ababede41a1bd7f91620809f5f8 deleted and replaced by tablet ffd8aa2bc6e344b78dce518411997fb5
I20260202 02:52:14.216804 1175 tablet_bootstrap.cc:654] T 220c3b32f56c478fbca7fbbd6135290c P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.217729 1173 tablet_bootstrap.cc:492] T 397ed14bb33e44e59536243d0fa7e459 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:14.218744 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 62241ababede41a1bd7f91620809f5f8 with delete_type TABLET_DATA_DELETED (replaced by tablet ffd8aa2bc6e344b78dce518411997fb5) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:14.219046 1180 tablet_replica.cc:333] T 62241ababede41a1bd7f91620809f5f8 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:14.219230 758 tablet_service.cc:1552] Processing DeleteTablet for tablet a6940c7f38ff496abf3c2dab66da5100 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:14.219286 1180 raft_consensus.cc:2243] T 62241ababede41a1bd7f91620809f5f8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:14.219446 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 220c3b32f56c478fbca7fbbd6135290c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:14.219660 1173 tablet_bootstrap.cc:654] T 397ed14bb33e44e59536243d0fa7e459 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.219698 900 tablet_service.cc:1552] Processing DeleteTablet for tablet a6940c7f38ff496abf3c2dab66da5100 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:14.219462 1180 raft_consensus.cc:2272] T 62241ababede41a1bd7f91620809f5f8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.219954 903 tablet_service.cc:1552] Processing DeleteTablet for tablet 220c3b32f56c478fbca7fbbd6135290c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:14.220148 901 tablet_service.cc:1552] Processing DeleteTablet for tablet 62241ababede41a1bd7f91620809f5f8 with delete_type TABLET_DATA_DELETED (replaced by tablet ffd8aa2bc6e344b78dce518411997fb5) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:14.220366 1182 tablet_replica.cc:333] T a6940c7f38ff496abf3c2dab66da5100 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:14.220548 1182 raft_consensus.cc:2243] T a6940c7f38ff496abf3c2dab66da5100 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:14.220671 1180 ts_tablet_manager.cc:1916] T 62241ababede41a1bd7f91620809f5f8 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.220709 1182 raft_consensus.cc:2272] T a6940c7f38ff496abf3c2dab66da5100 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.221110 520 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet ea0b6bb727404522b8291cbcd12879ea
I20260202 02:52:14.223878 1042 tablet_service.cc:1552] Processing DeleteTablet for tablet 220c3b32f56c478fbca7fbbd6135290c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:14.223985 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 62241ababede41a1bd7f91620809f5f8 with delete_type TABLET_DATA_DELETED (replaced by tablet ffd8aa2bc6e344b78dce518411997fb5) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:14.223905 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet a6940c7f38ff496abf3c2dab66da5100 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:14.224184 1043 tablet_service.cc:1552] Processing DeleteTablet for tablet ea0b6bb727404522b8291cbcd12879ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:14.224608 1183 tablet_replica.cc:333] T a6940c7f38ff496abf3c2dab66da5100 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:14.224803 1183 raft_consensus.cc:2243] T a6940c7f38ff496abf3c2dab66da5100 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:14.224869 1182 ts_tablet_manager.cc:1916] T a6940c7f38ff496abf3c2dab66da5100 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.224858 527 master_service.cc:954] ReplaceTablet: received request to replace tablet 2730d8b6812e487ea0aff4da45d24044 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:14.224970 1183 raft_consensus.cc:2272] T a6940c7f38ff496abf3c2dab66da5100 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.225512 901 tablet_service.cc:1552] Processing DeleteTablet for tablet ea0b6bb727404522b8291cbcd12879ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:14.226043 520 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 9c52693bdd214919b42cee1bbba95ed2 (table default.Table_3c631c97b1034c4ebe744d09da22559f [id=92e6422d4fc3439abab376a52ad8876a]). Expected version 1 got 0
I20260202 02:52:14.226081 754 tablet_service.cc:1552] Processing DeleteTablet for tablet ea0b6bb727404522b8291cbcd12879ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:14.228049 1183 ts_tablet_manager.cc:1916] T a6940c7f38ff496abf3c2dab66da5100 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:14.228461 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 220c3b32f56c478fbca7fbbd6135290c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 220c3b32f56c478fbca7fbbd6135290c already in progress: creating tablet
I20260202 02:52:14.228771 1043 tablet_service.cc:1552] Processing DeleteTablet for tablet 7781ccb7f1ea437fb1d61d3625a739a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:13 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:14.229382 525 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 6bac811b0f9f4276ac63786d97f02188
I20260202 02:52:14.231194 1043 tablet_service.cc:1552] Processing DeleteTablet for tablet 6bac811b0f9f4276ac63786d97f02188 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:14.231911 524 master_service.cc:954] ReplaceTablet: received request to replace tablet 2730d8b6812e487ea0aff4da45d24044 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:14.232191 525 master_service.cc:954] ReplaceTablet: received request to replace tablet 7781ccb7f1ea437fb1d61d3625a739a9 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:14.232991 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 6bac811b0f9f4276ac63786d97f02188 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:14.234617 901 tablet_service.cc:1552] Processing DeleteTablet for tablet 6bac811b0f9f4276ac63786d97f02188 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:14.238336 1183 ts_tablet_manager.cc:1929] T a6940c7f38ff496abf3c2dab66da5100 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:14.238616 1183 log.cc:1199] T a6940c7f38ff496abf3c2dab66da5100 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/a6940c7f38ff496abf3c2dab66da5100
I20260202 02:52:14.239091 1183 ts_tablet_manager.cc:1950] T a6940c7f38ff496abf3c2dab66da5100 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:14.239583 1180 ts_tablet_manager.cc:1929] T 62241ababede41a1bd7f91620809f5f8 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:14.239777 1180 log.cc:1199] T 62241ababede41a1bd7f91620809f5f8 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/62241ababede41a1bd7f91620809f5f8
I20260202 02:52:14.240190 1180 ts_tablet_manager.cc:1950] T 62241ababede41a1bd7f91620809f5f8 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:14.240692 1183 tablet_replica.cc:333] T 62241ababede41a1bd7f91620809f5f8 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:14.241045 1183 raft_consensus.cc:2243] T 62241ababede41a1bd7f91620809f5f8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:14.241292 1183 raft_consensus.cc:2272] T 62241ababede41a1bd7f91620809f5f8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.241493 1180 tablet_replica.cc:333] T ea0b6bb727404522b8291cbcd12879ea P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:14.241667 1180 raft_consensus.cc:2243] T ea0b6bb727404522b8291cbcd12879ea P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:14.241843 1180 raft_consensus.cc:2272] T ea0b6bb727404522b8291cbcd12879ea P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.242635 1183 ts_tablet_manager.cc:1916] T 62241ababede41a1bd7f91620809f5f8 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.242633 1180 ts_tablet_manager.cc:1916] T ea0b6bb727404522b8291cbcd12879ea P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.242774 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 62241ababede41a1bd7f91620809f5f8 (table default.Table_4a3ef984ca894b8b8f4301fecb719b5b [id=3ccb047c30974c0f9cbe778971e61644]) successfully deleted
W20260202 02:52:14.243130 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet a6940c7f38ff496abf3c2dab66da5100 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a6940c7f38ff496abf3c2dab66da5100 already in progress: creating tablet
W20260202 02:52:14.243357 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 220c3b32f56c478fbca7fbbd6135290c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 220c3b32f56c478fbca7fbbd6135290c already in progress: creating tablet
I20260202 02:52:14.244298 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet a6940c7f38ff496abf3c2dab66da5100 (table default.Table_011de32f5bb648439181d1154b82b2c5 [id=1d7c11d3ee26455e995b30f69a967132]) successfully deleted
I20260202 02:52:14.248049 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_5fa327d665484811934aa3b11188a76d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:14.248757 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5fa327d665484811934aa3b11188a76d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.249236 1175 tablet_bootstrap.cc:492] T 220c3b32f56c478fbca7fbbd6135290c P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:14.249387 1175 ts_tablet_manager.cc:1403] T 220c3b32f56c478fbca7fbbd6135290c P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.035s user 0.002s sys 0.007s
I20260202 02:52:14.250201 1175 raft_consensus.cc:359] T 220c3b32f56c478fbca7fbbd6135290c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.250401 1175 raft_consensus.cc:385] T 220c3b32f56c478fbca7fbbd6135290c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.250485 1175 raft_consensus.cc:740] T 220c3b32f56c478fbca7fbbd6135290c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.250701 1173 tablet_bootstrap.cc:492] T 397ed14bb33e44e59536243d0fa7e459 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:14.250730 1175 consensus_queue.cc:260] T 220c3b32f56c478fbca7fbbd6135290c P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.250851 1173 ts_tablet_manager.cc:1403] T 397ed14bb33e44e59536243d0fa7e459 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.033s user 0.004s sys 0.003s
I20260202 02:52:14.251236 1175 ts_tablet_manager.cc:1434] T 220c3b32f56c478fbca7fbbd6135290c P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260202 02:52:14.251533 1175 tablet_bootstrap.cc:492] T c9ce1b27212646e691bad953c9fbe14d P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:14.251538 1180 ts_tablet_manager.cc:1929] T ea0b6bb727404522b8291cbcd12879ea P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:14.251662 1180 log.cc:1199] T ea0b6bb727404522b8291cbcd12879ea P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/ea0b6bb727404522b8291cbcd12879ea
I20260202 02:52:14.252043 1180 ts_tablet_manager.cc:1950] T ea0b6bb727404522b8291cbcd12879ea P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:14.253178 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet ea0b6bb727404522b8291cbcd12879ea (table default.Table_c86dacfe4afb42ed9f08fa8a2a6ecc75 [id=31c0ee69dd9a473aada01e871ac80e81]) successfully deleted
I20260202 02:52:14.253394 1175 tablet_bootstrap.cc:654] T c9ce1b27212646e691bad953c9fbe14d P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.253373 1173 raft_consensus.cc:359] T 397ed14bb33e44e59536243d0fa7e459 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.253566 1173 raft_consensus.cc:385] T 397ed14bb33e44e59536243d0fa7e459 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.253588 1180 tablet_replica.cc:333] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:14.253664 1173 raft_consensus.cc:740] T 397ed14bb33e44e59536243d0fa7e459 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.253767 1180 raft_consensus.cc:2243] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:14.254232 1180 pending_rounds.cc:70] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d: Trying to abort 1 pending ops.
I20260202 02:52:14.254810 1173 consensus_queue.cc:260] T 397ed14bb33e44e59536243d0fa7e459 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.254748 1180 pending_rounds.cc:77] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923007135440896 op_type: NO_OP noop_request { }
I20260202 02:52:14.255203 1180 raft_consensus.cc:2889] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:14.255388 1180 raft_consensus.cc:2272] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.258755 1173 ts_tablet_manager.cc:1434] T 397ed14bb33e44e59536243d0fa7e459 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260202 02:52:14.262837 1173 tablet_bootstrap.cc:492] T 16713353f2684b9d96c80a3abbcdf74b P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:14.264504 527 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 2730d8b6812e487ea0aff4da45d24044
I20260202 02:52:14.265240 527 catalog_manager.cc:6724] ReplaceTablet: tablet 2730d8b6812e487ea0aff4da45d24044 deleted and replaced by tablet f24c0b99574d437a99b273204f105460
I20260202 02:52:14.265420 1184 raft_consensus.cc:493] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:14.265620 1184 raft_consensus.cc:515] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.265982 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 2730d8b6812e487ea0aff4da45d24044 with delete_type TABLET_DATA_DELETED (replaced by tablet f24c0b99574d437a99b273204f105460) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:14.266911 901 tablet_service.cc:1552] Processing DeleteTablet for tablet 2730d8b6812e487ea0aff4da45d24044 with delete_type TABLET_DATA_DELETED (replaced by tablet f24c0b99574d437a99b273204f105460) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:14.267639 1184 leader_election.cc:290] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:14.268290 1043 tablet_service.cc:1552] Processing DeleteTablet for tablet 2730d8b6812e487ea0aff4da45d24044 with delete_type TABLET_DATA_DELETED (replaced by tablet f24c0b99574d437a99b273204f105460) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:14.275184 1174 tablet_bootstrap.cc:492] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:14.275421 1174 ts_tablet_manager.cc:1403] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.063s user 0.013s sys 0.012s
I20260202 02:52:14.275348 754 tablet_service.cc:1505] Processing CreateTablet for tablet 81f35df38f284f0ca5c3791f3d9d9293 (DEFAULT_TABLE table=default.Table_15c57b6d38ab4f2fb7dca2a405f01058 [id=d54948158c124d00ab0e2b1f43d45cc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.275908 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81f35df38f284f0ca5c3791f3d9d9293. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.276180 1173 tablet_bootstrap.cc:654] T 16713353f2684b9d96c80a3abbcdf74b P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.276437 1174 raft_consensus.cc:359] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.276644 1174 raft_consensus.cc:385] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.276724 1174 raft_consensus.cc:740] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.276988 1174 consensus_queue.cc:260] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.277511 1174 ts_tablet_manager.cc:1434] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:14.281850 1174 tablet_bootstrap.cc:492] T a6940c7f38ff496abf3c2dab66da5100 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:14.283761 1174 tablet_bootstrap.cc:654] T a6940c7f38ff496abf3c2dab66da5100 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.284019 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "397ed14bb33e44e59536243d0fa7e459" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:14.284260 1064 raft_consensus.cc:2468] T 397ed14bb33e44e59536243d0fa7e459 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
I20260202 02:52:14.284732 758 tablet_service.cc:1552] Processing DeleteTablet for tablet ea0b6bb727404522b8291cbcd12879ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:14.284998 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 62241ababede41a1bd7f91620809f5f8 with delete_type TABLET_DATA_DELETED (replaced by tablet ffd8aa2bc6e344b78dce518411997fb5) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:14.284883 859 leader_election.cc:304] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:14.285642 1184 raft_consensus.cc:2804] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:14.285786 1184 raft_consensus.cc:493] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:14.285881 1184 raft_consensus.cc:3060] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:14.281519 1183 ts_tablet_manager.cc:1929] T 62241ababede41a1bd7f91620809f5f8 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:14.286331 1183 log.cc:1199] T 62241ababede41a1bd7f91620809f5f8 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/62241ababede41a1bd7f91620809f5f8
I20260202 02:52:14.286692 1183 ts_tablet_manager.cc:1950] T 62241ababede41a1bd7f91620809f5f8 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:14.289692 1184 raft_consensus.cc:515] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.290548 1183 tablet_replica.cc:333] T ea0b6bb727404522b8291cbcd12879ea P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:14.290777 1183 raft_consensus.cc:2243] T ea0b6bb727404522b8291cbcd12879ea P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:14.290970 1183 raft_consensus.cc:2272] T ea0b6bb727404522b8291cbcd12879ea P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.292641 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "397ed14bb33e44e59536243d0fa7e459" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:14.292902 1064 raft_consensus.cc:3060] T 397ed14bb33e44e59536243d0fa7e459 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:14.293742 1183 ts_tablet_manager.cc:1916] T ea0b6bb727404522b8291cbcd12879ea P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.295778 1064 raft_consensus.cc:2468] T 397ed14bb33e44e59536243d0fa7e459 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
I20260202 02:52:14.295825 1182 ts_tablet_manager.cc:1929] T a6940c7f38ff496abf3c2dab66da5100 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:14.296025 1182 log.cc:1199] T a6940c7f38ff496abf3c2dab66da5100 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/a6940c7f38ff496abf3c2dab66da5100
I20260202 02:52:14.297231 859 leader_election.cc:304] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:14.298242 1043 tablet_service.cc:1552] Processing DeleteTablet for tablet 397ed14bb33e44e59536243d0fa7e459 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:14.298241 757 tablet_service.cc:1505] Processing CreateTablet for tablet 7e81d5780d9647b9baac0066282376ea (DEFAULT_TABLE table=default.Table_629c3da327a740f684072799e959c2ca [id=30cd87ed1178421bb31ee622ce3254c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.298568 1224 raft_consensus.cc:2804] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:14.298870 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e81d5780d9647b9baac0066282376ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.299893 1224 raft_consensus.cc:697] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:14.300977 1224 consensus_queue.cc:237] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.305998 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 62241ababede41a1bd7f91620809f5f8 (table default.Table_4a3ef984ca894b8b8f4301fecb719b5b [id=3ccb047c30974c0f9cbe778971e61644]) successfully deleted
I20260202 02:52:14.306675 1184 leader_election.cc:290] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:14.306849 758 tablet_service.cc:1505] Processing CreateTablet for tablet fde2c1fba78248329f3662dd1036c7a2 (DEFAULT_TABLE table=default.Table_9fd715491a6a4bbc8685138d0194362d [id=98ddcbfb5ad14ae480a54845ec7c45a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.307366 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_9fb69280bd6048d898c8e92f2ef9f61e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:14.308027 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9fb69280bd6048d898c8e92f2ef9f61e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.308187 1174 tablet_bootstrap.cc:492] T a6940c7f38ff496abf3c2dab66da5100 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:14.308604 1174 ts_tablet_manager.cc:1403] T a6940c7f38ff496abf3c2dab66da5100 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.027s user 0.003s sys 0.001s
I20260202 02:52:14.309005 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_a2258ce8a41c4be1b2ee74dc0e31cd84"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:14.309609 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a2258ce8a41c4be1b2ee74dc0e31cd84 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.309768 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_2ae489b852d84e119c265efad1950d4b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:14.310322 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2ae489b852d84e119c265efad1950d4b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.311235 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_85da33c295954de58e2c6290005ac865"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:14.311784 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_85da33c295954de58e2c6290005ac865 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.311944 1043 tablet_service.cc:1505] Processing CreateTablet for tablet 81f35df38f284f0ca5c3791f3d9d9293 (DEFAULT_TABLE table=default.Table_15c57b6d38ab4f2fb7dca2a405f01058 [id=d54948158c124d00ab0e2b1f43d45cc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.312402 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81f35df38f284f0ca5c3791f3d9d9293. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.312453 1174 raft_consensus.cc:359] T a6940c7f38ff496abf3c2dab66da5100 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.312629 1174 raft_consensus.cc:385] T a6940c7f38ff496abf3c2dab66da5100 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.312721 1174 raft_consensus.cc:740] T a6940c7f38ff496abf3c2dab66da5100 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.312943 1174 consensus_queue.cc:260] T a6940c7f38ff496abf3c2dab66da5100 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.313459 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_09d8f825c4c546839c0a820f1ddf0047"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:14.314087 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_09d8f825c4c546839c0a820f1ddf0047 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.314579 1174 ts_tablet_manager.cc:1434] T a6940c7f38ff496abf3c2dab66da5100 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260202 02:52:14.316416 1174 tablet_bootstrap.cc:492] T 397ed14bb33e44e59536243d0fa7e459 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:14.316681 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 7e81d5780d9647b9baac0066282376ea (DEFAULT_TABLE table=default.Table_629c3da327a740f684072799e959c2ca [id=30cd87ed1178421bb31ee622ce3254c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.317071 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e81d5780d9647b9baac0066282376ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.317699 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fde2c1fba78248329f3662dd1036c7a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.318187 1174 tablet_bootstrap.cc:654] T 397ed14bb33e44e59536243d0fa7e459 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.322649 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 7781ccb7f1ea437fb1d61d3625a739a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:13 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:14.325156 519 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ca030d309ab3480d8fa40b2cc071b79a" } new_table_name: "default.Table_d210d1530706429197d6294b29f18c87" modify_external_catalogs: true
I20260202 02:52:14.325380 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_2ed2b2d4b5dc4198bb62ee0e72f0e6ab"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:14.325994 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2ed2b2d4b5dc4198bb62ee0e72f0e6ab in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.327116 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_8aee8d27e02942fb91879229b190457f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:14.327636 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8aee8d27e02942fb91879229b190457f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.328867 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_3704b9992d3a420381b3dfe9f8635f95"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:14.329416 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3704b9992d3a420381b3dfe9f8635f95 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.330859 901 tablet_service.cc:1552] Processing DeleteTablet for tablet 7781ccb7f1ea437fb1d61d3625a739a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:13 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:14.331390 901 tablet_service.cc:1552] Processing DeleteTablet for tablet 397ed14bb33e44e59536243d0fa7e459 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:14.332094 901 tablet_service.cc:1505] Processing CreateTablet for tablet 81f35df38f284f0ca5c3791f3d9d9293 (DEFAULT_TABLE table=default.Table_15c57b6d38ab4f2fb7dca2a405f01058 [id=d54948158c124d00ab0e2b1f43d45cc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.332613 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81f35df38f284f0ca5c3791f3d9d9293. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.333195 757 tablet_service.cc:1552] Processing DeleteTablet for tablet a6940c7f38ff496abf3c2dab66da5100 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:14.333439 523 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_3c631c97b1034c4ebe744d09da22559f" } new_table_name: "default.Table_c2deb2be896d43e0a7bb49582c9276f9" modify_external_catalogs: true
I20260202 02:52:14.345676 901 tablet_service.cc:1505] Processing CreateTablet for tablet 7e81d5780d9647b9baac0066282376ea (DEFAULT_TABLE table=default.Table_629c3da327a740f684072799e959c2ca [id=30cd87ed1178421bb31ee622ce3254c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.346174 525 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ab0c372e2fbc439cbfc95db9aa9c75d0" } new_table_name: "default.Table_437c92ec0b8441f18dc31cfc0c2f2972" modify_external_catalogs: true
I20260202 02:52:14.346688 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e81d5780d9647b9baac0066282376ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.349974 1173 tablet_bootstrap.cc:492] T 16713353f2684b9d96c80a3abbcdf74b P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:14.352600 1173 ts_tablet_manager.cc:1403] T 16713353f2684b9d96c80a3abbcdf74b P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.090s user 0.003s sys 0.007s
I20260202 02:52:14.357062 1182 ts_tablet_manager.cc:1950] T a6940c7f38ff496abf3c2dab66da5100 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:14.357302 1044 tablet_service.cc:1505] Processing CreateTablet for tablet fde2c1fba78248329f3662dd1036c7a2 (DEFAULT_TABLE table=default.Table_9fd715491a6a4bbc8685138d0194362d [id=98ddcbfb5ad14ae480a54845ec7c45a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.357908 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fde2c1fba78248329f3662dd1036c7a2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:14.359380 519 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c2cc2250701d46a7abe7be03b11d45c8 consensus state has no leader
I20260202 02:52:14.360474 526 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_692487d5d3454bd28e0229336d7779a3" } new_table_name: "default.Table_3d26f7a34d0c48379e3cebda66e1ef13" modify_external_catalogs: true
I20260202 02:52:14.365135 903 tablet_service.cc:1505] Processing CreateTablet for tablet fde2c1fba78248329f3662dd1036c7a2 (DEFAULT_TABLE table=default.Table_9fd715491a6a4bbc8685138d0194362d [id=98ddcbfb5ad14ae480a54845ec7c45a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.365698 903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fde2c1fba78248329f3662dd1036c7a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.366044 758 tablet_service.cc:1505] Processing CreateTablet for tablet e6b76967e5d3476997fa019c4a8afb91 (DEFAULT_TABLE table=default.Table_e6d3c2dc7ad1416c949573796c5dc398 [id=a9429c7c44f144baa985c0731279243e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.366562 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6b76967e5d3476997fa019c4a8afb91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.370081 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_08b43dcf669342bbb6a9910f002d3c5b" } new_table_name: "default.Table_86f34199535c451cb54a2e20e70f1149" modify_external_catalogs: true
I20260202 02:52:14.376024 903 tablet_service.cc:1505] Processing CreateTablet for tablet e6b76967e5d3476997fa019c4a8afb91 (DEFAULT_TABLE table=default.Table_e6d3c2dc7ad1416c949573796c5dc398 [id=a9429c7c44f144baa985c0731279243e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.376797 903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6b76967e5d3476997fa019c4a8afb91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.378139 1182 tablet_replica.cc:333] T 220c3b32f56c478fbca7fbbd6135290c P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:14.378924 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet a6940c7f38ff496abf3c2dab66da5100 (table default.Table_011de32f5bb648439181d1154b82b2c5 [id=1d7c11d3ee26455e995b30f69a967132]) successfully deleted
I20260202 02:52:14.380646 757 tablet_service.cc:1505] Processing CreateTablet for tablet c2cc2250701d46a7abe7be03b11d45c8 (DEFAULT_TABLE table=default.Table_d210d1530706429197d6294b29f18c87 [id=7c76848b98614fbf8b2950e5d55cc609]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.381203 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2cc2250701d46a7abe7be03b11d45c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.381161 1043 tablet_service.cc:1505] Processing CreateTablet for tablet e6b76967e5d3476997fa019c4a8afb91 (DEFAULT_TABLE table=default.Table_e6d3c2dc7ad1416c949573796c5dc398 [id=a9429c7c44f144baa985c0731279243e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.381610 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6b76967e5d3476997fa019c4a8afb91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.384114 1178 raft_consensus.cc:493] T 7781ccb7f1ea437fb1d61d3625a739a9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:14.384352 1178 raft_consensus.cc:515] T 7781ccb7f1ea437fb1d61d3625a739a9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.386670 1178 leader_election.cc:290] T 7781ccb7f1ea437fb1d61d3625a739a9 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:14.393812 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_15539ec1a3fc43aab4083fc71964814e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:14.394793 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_15539ec1a3fc43aab4083fc71964814e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.401566 1180 ts_tablet_manager.cc:1916] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.407114 1173 raft_consensus.cc:359] T 16713353f2684b9d96c80a3abbcdf74b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.412586 1173 raft_consensus.cc:385] T 16713353f2684b9d96c80a3abbcdf74b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.415673 1173 raft_consensus.cc:740] T 16713353f2684b9d96c80a3abbcdf74b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.420395 1182 raft_consensus.cc:2243] T 220c3b32f56c478fbca7fbbd6135290c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:14.421047 1182 raft_consensus.cc:2272] T 220c3b32f56c478fbca7fbbd6135290c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.422158 900 tablet_service.cc:1505] Processing CreateTablet for tablet c2cc2250701d46a7abe7be03b11d45c8 (DEFAULT_TABLE table=default.Table_d210d1530706429197d6294b29f18c87 [id=7c76848b98614fbf8b2950e5d55cc609]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.422593 1173 consensus_queue.cc:260] T 16713353f2684b9d96c80a3abbcdf74b P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.422840 754 tablet_service.cc:1505] Processing CreateTablet for tablet 58f09c607cce4ccc82afd75c002ed1a8 (DEFAULT_TABLE table=default.Table_0b2c7ff473964e0da5c026225b766340 [id=0da4db61b3fe4cf4846edd5530b9327f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.423238 1182 ts_tablet_manager.cc:1916] T 220c3b32f56c478fbca7fbbd6135290c P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.424926 756 tablet_service.cc:1505] Processing CreateTablet for tablet fae7e3941c124cc2871390a0b6471f27 (DEFAULT_TABLE table=default.Table_692487d5d3454bd28e0229336d7779a3 [id=45e6d83589fa493195b06f633e295b00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.434732 1041 tablet_service.cc:1505] Processing CreateTablet for tablet fae7e3941c124cc2871390a0b6471f27 (DEFAULT_TABLE table=default.Table_692487d5d3454bd28e0229336d7779a3 [id=45e6d83589fa493195b06f633e295b00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.435302 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fae7e3941c124cc2871390a0b6471f27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.435454 524 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_e6d3c2dc7ad1416c949573796c5dc398" } modify_external_catalogs: true
I20260202 02:52:14.435681 524 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_e6d3c2dc7ad1416c949573796c5dc398" } modify_external_catalogs: true
I20260202 02:52:14.436031 520 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_15c57b6d38ab4f2fb7dca2a405f01058" } modify_external_catalogs: true
I20260202 02:52:14.436385 522 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_629c3da327a740f684072799e959c2ca" } modify_external_catalogs: true
W20260202 02:52:14.437331 523 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9c52693bdd214919b42cee1bbba95ed2 consensus state has no leader
I20260202 02:52:14.437640 522 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_629c3da327a740f684072799e959c2ca" } modify_external_catalogs: true
I20260202 02:52:14.437693 523 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_09d8f825c4c546839c0a820f1ddf0047" } modify_external_catalogs: true
I20260202 02:52:14.437830 523 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_09d8f825c4c546839c0a820f1ddf0047" } modify_external_catalogs: true
I20260202 02:52:14.438059 520 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_15c57b6d38ab4f2fb7dca2a405f01058" } modify_external_catalogs: true
I20260202 02:52:14.438306 528 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_9fb69280bd6048d898c8e92f2ef9f61e" } modify_external_catalogs: true
I20260202 02:52:14.438457 528 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_9fb69280bd6048d898c8e92f2ef9f61e" } modify_external_catalogs: true
W20260202 02:52:14.442204 526 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fae7e3941c124cc2871390a0b6471f27 consensus state has no leader
W20260202 02:52:14.443146 525 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 80eb3013f5354022a2dd78bb294a8497 consensus state has no leader
I20260202 02:52:14.447866 1043 tablet_service.cc:1505] Processing CreateTablet for tablet 58f09c607cce4ccc82afd75c002ed1a8 (DEFAULT_TABLE table=default.Table_0b2c7ff473964e0da5c026225b766340 [id=0da4db61b3fe4cf4846edd5530b9327f]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:14.452539 527 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 251455168e33430aafea8e0de42ca1b8 has no consensus state
I20260202 02:52:14.452726 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2cc2250701d46a7abe7be03b11d45c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.455662 1209 raft_consensus.cc:493] T a6940c7f38ff496abf3c2dab66da5100 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:14.455844 1209 raft_consensus.cc:515] T a6940c7f38ff496abf3c2dab66da5100 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.456739 1209 leader_election.cc:290] T a6940c7f38ff496abf3c2dab66da5100 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
W20260202 02:52:14.458135 715 leader_election.cc:343] T a6940c7f38ff496abf3c2dab66da5100 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Not found: Tablet not found: a6940c7f38ff496abf3c2dab66da5100
I20260202 02:52:14.459460 1175 tablet_bootstrap.cc:492] T c9ce1b27212646e691bad953c9fbe14d P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:14.459647 1175 ts_tablet_manager.cc:1403] T c9ce1b27212646e691bad953c9fbe14d P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.208s user 0.054s sys 0.086s
I20260202 02:52:14.460021 902 tablet_service.cc:1505] Processing CreateTablet for tablet 58f09c607cce4ccc82afd75c002ed1a8 (DEFAULT_TABLE table=default.Table_0b2c7ff473964e0da5c026225b766340 [id=0da4db61b3fe4cf4846edd5530b9327f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.460500 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58f09c607cce4ccc82afd75c002ed1a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.460526 1175 raft_consensus.cc:359] T c9ce1b27212646e691bad953c9fbe14d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.460752 1175 raft_consensus.cc:385] T c9ce1b27212646e691bad953c9fbe14d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.460842 1175 raft_consensus.cc:740] T c9ce1b27212646e691bad953c9fbe14d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.461709 1175 consensus_queue.cc:260] T c9ce1b27212646e691bad953c9fbe14d P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.462270 1175 ts_tablet_manager.cc:1434] T c9ce1b27212646e691bad953c9fbe14d P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260202 02:52:14.463510 1175 tablet_bootstrap.cc:492] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
W20260202 02:52:14.463810 714 leader_election.cc:343] T a6940c7f38ff496abf3c2dab66da5100 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: a6940c7f38ff496abf3c2dab66da5100
I20260202 02:52:14.463970 714 leader_election.cc:304] T a6940c7f38ff496abf3c2dab66da5100 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:14.464495 1209 raft_consensus.cc:2749] T a6940c7f38ff496abf3c2dab66da5100 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:14.464500 899 tablet_service.cc:1505] Processing CreateTablet for tablet fae7e3941c124cc2871390a0b6471f27 (DEFAULT_TABLE table=default.Table_692487d5d3454bd28e0229336d7779a3 [id=45e6d83589fa493195b06f633e295b00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.464888 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fae7e3941c124cc2871390a0b6471f27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.464775 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_601ac359670d4a44a24253267b53a875"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:14.465353 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_601ac359670d4a44a24253267b53a875 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.466615 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7781ccb7f1ea437fb1d61d3625a739a9" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:14.466862 923 raft_consensus.cc:2468] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:14.468698 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "397ed14bb33e44e59536243d0fa7e459" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:14.469172 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "397ed14bb33e44e59536243d0fa7e459" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:14.469812 1175 tablet_bootstrap.cc:654] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:14.472924 857 leader_election.cc:343] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:14.473546 857 leader_election.cc:343] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:14.471762 999 leader_election.cc:304] T 7781ccb7f1ea437fb1d61d3625a739a9 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:14.475183 1178 raft_consensus.cc:2804] T 7781ccb7f1ea437fb1d61d3625a739a9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:14.475306 1178 raft_consensus.cc:493] T 7781ccb7f1ea437fb1d61d3625a739a9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:14.475392 1178 raft_consensus.cc:3060] T 7781ccb7f1ea437fb1d61d3625a739a9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:14.478039 1178 raft_consensus.cc:515] T 7781ccb7f1ea437fb1d61d3625a739a9 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.479027 1178 leader_election.cc:290] T 7781ccb7f1ea437fb1d61d3625a739a9 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:14.479681 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7781ccb7f1ea437fb1d61d3625a739a9" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:14.479853 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58f09c607cce4ccc82afd75c002ed1a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.479924 923 raft_consensus.cc:3060] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:14.482191 1182 ts_tablet_manager.cc:1929] T 220c3b32f56c478fbca7fbbd6135290c P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:14.482888 1173 ts_tablet_manager.cc:1434] T 16713353f2684b9d96c80a3abbcdf74b P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.130s user 0.001s sys 0.001s
I20260202 02:52:14.483551 1044 tablet_service.cc:1505] Processing CreateTablet for tablet c2cc2250701d46a7abe7be03b11d45c8 (DEFAULT_TABLE table=default.Table_d210d1530706429197d6294b29f18c87 [id=7c76848b98614fbf8b2950e5d55cc609]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.484004 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2cc2250701d46a7abe7be03b11d45c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.488377 1183 ts_tablet_manager.cc:1929] T ea0b6bb727404522b8291cbcd12879ea P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:14.488591 1183 log.cc:1199] T ea0b6bb727404522b8291cbcd12879ea P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/ea0b6bb727404522b8291cbcd12879ea
I20260202 02:52:14.488969 1183 ts_tablet_manager.cc:1950] T ea0b6bb727404522b8291cbcd12879ea P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:14.482935 923 raft_consensus.cc:2468] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:14.489511 1182 log.cc:1199] T 220c3b32f56c478fbca7fbbd6135290c P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/220c3b32f56c478fbca7fbbd6135290c
I20260202 02:52:14.490082 1182 ts_tablet_manager.cc:1950] T 220c3b32f56c478fbca7fbbd6135290c P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:14.490096 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet ea0b6bb727404522b8291cbcd12879ea (table default.Table_c86dacfe4afb42ed9f08fa8a2a6ecc75 [id=31c0ee69dd9a473aada01e871ac80e81]) successfully deleted
I20260202 02:52:14.490228 999 leader_election.cc:304] T 7781ccb7f1ea437fb1d61d3625a739a9 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:14.490711 1178 raft_consensus.cc:2804] T 7781ccb7f1ea437fb1d61d3625a739a9 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:14.491218 1182 tablet_replica.cc:333] T 62241ababede41a1bd7f91620809f5f8 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:14.491503 1182 raft_consensus.cc:2243] T 62241ababede41a1bd7f91620809f5f8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:14.491734 1182 raft_consensus.cc:2272] T 62241ababede41a1bd7f91620809f5f8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.492414 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 220c3b32f56c478fbca7fbbd6135290c (table default.Table_c5a4aa6f50fb472c8982feedb30982a9 [id=7af0d6d84db7404ebed9db660a633c12]) successfully deleted
I20260202 02:52:14.492836 1182 ts_tablet_manager.cc:1916] T 62241ababede41a1bd7f91620809f5f8 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.500285 1174 tablet_bootstrap.cc:492] T 397ed14bb33e44e59536243d0fa7e459 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:14.500525 1174 ts_tablet_manager.cc:1403] T 397ed14bb33e44e59536243d0fa7e459 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.184s user 0.057s sys 0.069s
I20260202 02:52:14.500802 1173 tablet_bootstrap.cc:492] T 220c3b32f56c478fbca7fbbd6135290c P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:14.501421 1174 raft_consensus.cc:359] T 397ed14bb33e44e59536243d0fa7e459 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.501662 1174 raft_consensus.cc:385] T 397ed14bb33e44e59536243d0fa7e459 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.501737 1174 raft_consensus.cc:740] T 397ed14bb33e44e59536243d0fa7e459 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.502017 1174 consensus_queue.cc:260] T 397ed14bb33e44e59536243d0fa7e459 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.502929 1174 ts_tablet_manager.cc:1434] T 397ed14bb33e44e59536243d0fa7e459 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260202 02:52:14.503258 1174 tablet_bootstrap.cc:492] T 80eb3013f5354022a2dd78bb294a8497 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:14.506966 1182 ts_tablet_manager.cc:1929] T 62241ababede41a1bd7f91620809f5f8 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:14.507239 1182 log.cc:1199] T 62241ababede41a1bd7f91620809f5f8 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/62241ababede41a1bd7f91620809f5f8
I20260202 02:52:14.508886 1182 ts_tablet_manager.cc:1950] T 62241ababede41a1bd7f91620809f5f8 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:14.509649 1182 tablet_replica.cc:333] T ea0b6bb727404522b8291cbcd12879ea P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:14.509958 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 62241ababede41a1bd7f91620809f5f8 (table default.Table_4a3ef984ca894b8b8f4301fecb719b5b [id=3ccb047c30974c0f9cbe778971e61644]) successfully deleted
I20260202 02:52:14.510425 1182 raft_consensus.cc:2243] T ea0b6bb727404522b8291cbcd12879ea P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:14.514298 523 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 72475ea720c6429aa68de9478b374280
I20260202 02:52:14.514663 1182 raft_consensus.cc:2272] T ea0b6bb727404522b8291cbcd12879ea P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.515363 902 tablet_service.cc:1552] Processing DeleteTablet for tablet 72475ea720c6429aa68de9478b374280 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:14.518844 1182 ts_tablet_manager.cc:1916] T ea0b6bb727404522b8291cbcd12879ea P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.519903 522 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 7e81d5780d9647b9baac0066282376ea
I20260202 02:52:14.519975 520 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 81f35df38f284f0ca5c3791f3d9d9293
I20260202 02:52:14.521760 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet 72475ea720c6429aa68de9478b374280 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:14.522414 902 tablet_service.cc:1552] Processing DeleteTablet for tablet 81f35df38f284f0ca5c3791f3d9d9293 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:14.522596 528 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet add934bf70ae4de8a40066f06fd94795
I20260202 02:52:14.523466 902 tablet_service.cc:1552] Processing DeleteTablet for tablet 7e81d5780d9647b9baac0066282376ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:14.523957 902 tablet_service.cc:1552] Processing DeleteTablet for tablet add934bf70ae4de8a40066f06fd94795 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:14.524840 520 master_service.cc:954] ReplaceTablet: received request to replace tablet 9c52693bdd214919b42cee1bbba95ed2 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:14.524842 1042 tablet_service.cc:1552] Processing DeleteTablet for tablet 81f35df38f284f0ca5c3791f3d9d9293 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:14.525115 1040 tablet_service.cc:1552] Processing DeleteTablet for tablet add934bf70ae4de8a40066f06fd94795 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:14.525315 522 master_service.cc:954] ReplaceTablet: received request to replace tablet 005cda97ab5c4b70b8fc8ff66a05d996 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:14.525424 521 master_service.cc:954] ReplaceTablet: received request to replace tablet c2cc2250701d46a7abe7be03b11d45c8 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:14.525691 1175 tablet_bootstrap.cc:492] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:14.525838 1175 ts_tablet_manager.cc:1403] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.063s user 0.020s sys 0.009s
I20260202 02:52:14.526134 519 master_service.cc:954] ReplaceTablet: received request to replace tablet fde2c1fba78248329f3662dd1036c7a2 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:14.526407 528 master_service.cc:954] ReplaceTablet: received request to replace tablet c9ce1b27212646e691bad953c9fbe14d from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:14.526702 1175 raft_consensus.cc:359] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.526887 1175 raft_consensus.cc:385] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.526981 1175 raft_consensus.cc:740] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.527225 1175 consensus_queue.cc:260] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.527742 1175 ts_tablet_manager.cc:1434] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260202 02:52:14.528200 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet 7e81d5780d9647b9baac0066282376ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:14.530372 1224 raft_consensus.cc:493] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:14.530611 1224 raft_consensus.cc:515] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.531934 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9c52693bdd214919b42cee1bbba95ed2" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:14.532152 1064 raft_consensus.cc:2468] T 9c52693bdd214919b42cee1bbba95ed2 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
I20260202 02:52:14.533036 859 leader_election.cc:304] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:14.533074 979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } } }
I20260202 02:52:14.533298 979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [sys.catalog]: This master's current role is: LEADER
I20260202 02:52:14.533478 1182 ts_tablet_manager.cc:1929] T ea0b6bb727404522b8291cbcd12879ea P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:14.533525 1224 leader_election.cc:290] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:14.533720 1225 raft_consensus.cc:2804] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:14.533864 1225 raft_consensus.cc:493] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:14.534147 1225 raft_consensus.cc:3060] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:14.534701 524 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet e6b76967e5d3476997fa019c4a8afb91
I20260202 02:52:14.538986 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet e6b76967e5d3476997fa019c4a8afb91 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:14.539703 1225 raft_consensus.cc:515] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.540925 1225 leader_election.cc:290] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:14.542758 1175 tablet_bootstrap.cc:492] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:14.542804 1183 tablet_replica.cc:333] T 7781ccb7f1ea437fb1d61d3625a739a9 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:14.543701 1173 tablet_bootstrap.cc:654] T 220c3b32f56c478fbca7fbbd6135290c P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.543915 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9c52693bdd214919b42cee1bbba95ed2" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:14.544157 1064 raft_consensus.cc:3060] T 9c52693bdd214919b42cee1bbba95ed2 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:14.544513 1175 tablet_bootstrap.cc:654] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.544911 1178 raft_consensus.cc:697] T 7781ccb7f1ea437fb1d61d3625a739a9 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:14.545578 1178 consensus_queue.cc:237] T 7781ccb7f1ea437fb1d61d3625a739a9 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.548018 520 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 9c52693bdd214919b42cee1bbba95ed2
I20260202 02:52:14.547051 1064 raft_consensus.cc:2468] T 9c52693bdd214919b42cee1bbba95ed2 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
I20260202 02:52:14.549199 520 catalog_manager.cc:6724] ReplaceTablet: tablet 9c52693bdd214919b42cee1bbba95ed2 deleted and replaced by tablet fc690a7c8e214d1d921488e7f0b1ce8d
I20260202 02:52:14.549400 859 leader_election.cc:304] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:14.550282 522 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 005cda97ab5c4b70b8fc8ff66a05d996
I20260202 02:52:14.550719 522 catalog_manager.cc:6724] ReplaceTablet: tablet 005cda97ab5c4b70b8fc8ff66a05d996 deleted and replaced by tablet 4f2a9404f7424dc78b5bd062ebd8cb53
I20260202 02:52:14.551355 521 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet c2cc2250701d46a7abe7be03b11d45c8
I20260202 02:52:14.551764 521 catalog_manager.cc:6724] ReplaceTablet: tablet c2cc2250701d46a7abe7be03b11d45c8 deleted and replaced by tablet 0d297b6243c348ee82c146a8339cf6c4
I20260202 02:52:14.552419 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_5610c9ec35d74b1b909b83c855f21c68"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:14.552557 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet 9c52693bdd214919b42cee1bbba95ed2 with delete_type TABLET_DATA_DELETED (replaced by tablet fc690a7c8e214d1d921488e7f0b1ce8d) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:14.553612 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_6a67d7cee0254eba9a53f4d52a30ee39"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:14.553843 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5610c9ec35d74b1b909b83c855f21c68 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:14.554204 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6a67d7cee0254eba9a53f4d52a30ee39 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.555572 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_53eaadb859904d6b95ef0e536019ad79"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:14.556187 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_53eaadb859904d6b95ef0e536019ad79 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.557222 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_a2838c9f404646eea3ee18d69eb32a12"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:14.557776 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a2838c9f404646eea3ee18d69eb32a12 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.559127 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_f23c311557a14f4a8c66eaf4af7a8a92"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:14.559659 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f23c311557a14f4a8c66eaf4af7a8a92 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.560003 1040 tablet_service.cc:1552] Processing DeleteTablet for tablet c2cc2250701d46a7abe7be03b11d45c8 with delete_type TABLET_DATA_DELETED (replaced by tablet 0d297b6243c348ee82c146a8339cf6c4) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:14.560182 1175 tablet_bootstrap.cc:492] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:14.559482 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet 005cda97ab5c4b70b8fc8ff66a05d996 with delete_type TABLET_DATA_DELETED (replaced by tablet 4f2a9404f7424dc78b5bd062ebd8cb53) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:14.560397 1175 ts_tablet_manager.cc:1403] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.006s
I20260202 02:52:14.561256 1175 raft_consensus.cc:359] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.561465 1175 raft_consensus.cc:385] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.561532 1175 raft_consensus.cc:740] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.561656 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_b3f54615950542f2b090477b297f80ed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:14.561815 1175 consensus_queue.cc:260] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.562131 1184 raft_consensus.cc:2804] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election won for term 1
W20260202 02:52:14.562222 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b3f54615950542f2b090477b297f80ed in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.562240 1184 raft_consensus.cc:697] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:14.562477 1184 consensus_queue.cc:237] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.566505 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_0b2c7ff473964e0da5c026225b766340" } new_table_name: "default.Table_3b335a0806154f98a907f551596ba353" modify_external_catalogs: true
I20260202 02:52:14.505214 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 397ed14bb33e44e59536243d0fa7e459 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:14.517015 759 tablet_service.cc:1552] Processing DeleteTablet for tablet 72475ea720c6429aa68de9478b374280 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:14.521252 755 tablet_service.cc:1552] Processing DeleteTablet for tablet 81f35df38f284f0ca5c3791f3d9d9293 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:14.522274 753 tablet_service.cc:1552] Processing DeleteTablet for tablet 7e81d5780d9647b9baac0066282376ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:14.523633 752 tablet_service.cc:1552] Processing DeleteTablet for tablet add934bf70ae4de8a40066f06fd94795 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:14.537672 751 tablet_service.cc:1552] Processing DeleteTablet for tablet e6b76967e5d3476997fa019c4a8afb91 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:14.546214 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58f09c607cce4ccc82afd75c002ed1a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.568061 1175 ts_tablet_manager.cc:1434] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260202 02:52:14.568131 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fae7e3941c124cc2871390a0b6471f27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.568456 1175 tablet_bootstrap.cc:492] T 16713353f2684b9d96c80a3abbcdf74b P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:14.533620 1182 log.cc:1199] T ea0b6bb727404522b8291cbcd12879ea P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/ea0b6bb727404522b8291cbcd12879ea
I20260202 02:52:14.569504 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7781ccb7f1ea437fb1d61d3625a739a9" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:14.569731 1182 ts_tablet_manager.cc:1950] T ea0b6bb727404522b8291cbcd12879ea P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:14.569849 776 raft_consensus.cc:3060] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:14.570722 1182 tablet_replica.cc:333] T 6bac811b0f9f4276ac63786d97f02188 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:14.570928 1182 raft_consensus.cc:2243] T 6bac811b0f9f4276ac63786d97f02188 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:14.571110 1182 raft_consensus.cc:2272] T 6bac811b0f9f4276ac63786d97f02188 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.571725 1182 ts_tablet_manager.cc:1916] T 6bac811b0f9f4276ac63786d97f02188 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.551879 750 tablet_service.cc:1552] Processing DeleteTablet for tablet 005cda97ab5c4b70b8fc8ff66a05d996 with delete_type TABLET_DATA_DELETED (replaced by tablet 4f2a9404f7424dc78b5bd062ebd8cb53) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:14.552572 749 tablet_service.cc:1552] Processing DeleteTablet for tablet c2cc2250701d46a7abe7be03b11d45c8 with delete_type TABLET_DATA_DELETED (replaced by tablet 0d297b6243c348ee82c146a8339cf6c4) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:14.573339 776 raft_consensus.cc:2468] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:14.556557 758 tablet_service.cc:1552] Processing DeleteTablet for tablet 9c52693bdd214919b42cee1bbba95ed2 with delete_type TABLET_DATA_DELETED (replaced by tablet fc690a7c8e214d1d921488e7f0b1ce8d) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:14.564057 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9c52693bdd214919b42cee1bbba95ed2" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:14.566066 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9c52693bdd214919b42cee1bbba95ed2" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:14.566150 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7781ccb7f1ea437fb1d61d3625a739a9" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:14.574558 777 raft_consensus.cc:2376] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1. Re-sending same reply.
I20260202 02:52:14.576143 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_9c6ddf42d084449998f1f7925b0b0c3b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:14.576814 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9c6ddf42d084449998f1f7925b0b0c3b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.578583 1174 tablet_bootstrap.cc:654] T 80eb3013f5354022a2dd78bb294a8497 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.579087 902 tablet_service.cc:1552] Processing DeleteTablet for tablet e6b76967e5d3476997fa019c4a8afb91 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:14.579342 899 tablet_service.cc:1552] Processing DeleteTablet for tablet 9c52693bdd214919b42cee1bbba95ed2 with delete_type TABLET_DATA_DELETED (replaced by tablet fc690a7c8e214d1d921488e7f0b1ce8d) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:14.579938 1183 raft_consensus.cc:2243] T 7781ccb7f1ea437fb1d61d3625a739a9 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:14.580003 1175 tablet_bootstrap.cc:654] T 16713353f2684b9d96c80a3abbcdf74b P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.581179 1183 pending_rounds.cc:70] T 7781ccb7f1ea437fb1d61d3625a739a9 P b1f05820b6034228bdad884cc348716d: Trying to abort 1 pending ops.
I20260202 02:52:14.581427 1183 pending_rounds.cc:77] T 7781ccb7f1ea437fb1d61d3625a739a9 P b1f05820b6034228bdad884cc348716d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923008721162240 op_type: NO_OP noop_request { }
I20260202 02:52:14.581640 1183 raft_consensus.cc:2889] T 7781ccb7f1ea437fb1d61d3625a739a9 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:14.581776 1183 raft_consensus.cc:2272] T 7781ccb7f1ea437fb1d61d3625a739a9 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.584383 526 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 80eb3013f5354022a2dd78bb294a8497 (table default.Table_437c92ec0b8441f18dc31cfc0c2f2972 [id=316b8a276d9b415fafe8511aa7c32fc5]). Expected version 1 got 0
I20260202 02:52:14.586936 1183 ts_tablet_manager.cc:1916] T 7781ccb7f1ea437fb1d61d3625a739a9 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.588006 900 tablet_service.cc:1552] Processing DeleteTablet for tablet 005cda97ab5c4b70b8fc8ff66a05d996 with delete_type TABLET_DATA_DELETED (replaced by tablet 4f2a9404f7424dc78b5bd062ebd8cb53) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:14.589870 526 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 80eb3013f5354022a2dd78bb294a8497 (table default.Table_437c92ec0b8441f18dc31cfc0c2f2972 [id=316b8a276d9b415fafe8511aa7c32fc5]). Expected version 1 got 0
I20260202 02:52:14.592358 900 tablet_service.cc:1552] Processing DeleteTablet for tablet c2cc2250701d46a7abe7be03b11d45c8 with delete_type TABLET_DATA_DELETED (replaced by tablet 0d297b6243c348ee82c146a8339cf6c4) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:14.592830 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet ea0b6bb727404522b8291cbcd12879ea (table default.Table_c86dacfe4afb42ed9f08fa8a2a6ecc75 [id=31c0ee69dd9a473aada01e871ac80e81]) successfully deleted
I20260202 02:52:14.593282 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 72475ea720c6429aa68de9478b374280
W20260202 02:52:14.593792 857 leader_election.cc:343] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:14.595227 857 leader_election.cc:343] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:14.595405 1174 tablet_bootstrap.cc:492] T 80eb3013f5354022a2dd78bb294a8497 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:14.595553 1174 ts_tablet_manager.cc:1403] T 80eb3013f5354022a2dd78bb294a8497 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.092s user 0.002s sys 0.007s
I20260202 02:52:14.595851 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet add934bf70ae4de8a40066f06fd94795
I20260202 02:52:14.596534 1174 raft_consensus.cc:359] T 80eb3013f5354022a2dd78bb294a8497 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.596735 1174 raft_consensus.cc:385] T 80eb3013f5354022a2dd78bb294a8497 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.596833 754 tablet_service.cc:1552] Processing DeleteTablet for tablet add934bf70ae4de8a40066f06fd94795 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:14.596936 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 72475ea720c6429aa68de9478b374280 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:14.596840 1174 raft_consensus.cc:740] T 80eb3013f5354022a2dd78bb294a8497 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.597298 1174 consensus_queue.cc:260] T 80eb3013f5354022a2dd78bb294a8497 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.599032 1174 ts_tablet_manager.cc:1434] T 80eb3013f5354022a2dd78bb294a8497 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260202 02:52:14.599380 903 tablet_service.cc:1552] Processing DeleteTablet for tablet c9ce1b27212646e691bad953c9fbe14d with delete_type TABLET_DATA_DELETED (replaced by tablet 2730d8b6812e487ea0aff4da45d24044) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:14.600304 1174 tablet_bootstrap.cc:492] T 220c3b32f56c478fbca7fbbd6135290c P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:14.601938 903 tablet_service.cc:1552] Processing DeleteTablet for tablet 72475ea720c6429aa68de9478b374280 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:14.602238 756 tablet_service.cc:1505] Processing CreateTablet for tablet 2ff6af17a6854451a1d7b904aa4f1f55 (DEFAULT_TABLE table=default.Table_2ed2b2d4b5dc4198bb62ee0e72f0e6ab [id=b4ab2ba86e1a49338a98d855ded99f5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.601989 900 tablet_service.cc:1552] Processing DeleteTablet for tablet add934bf70ae4de8a40066f06fd94795 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:14.602731 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ff6af17a6854451a1d7b904aa4f1f55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.603163 1175 tablet_bootstrap.cc:492] T 16713353f2684b9d96c80a3abbcdf74b P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:14.603327 1175 ts_tablet_manager.cc:1403] T 16713353f2684b9d96c80a3abbcdf74b P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.035s user 0.004s sys 0.004s
I20260202 02:52:14.604164 1175 raft_consensus.cc:359] T 16713353f2684b9d96c80a3abbcdf74b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.604462 1175 raft_consensus.cc:385] T 16713353f2684b9d96c80a3abbcdf74b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.604564 1175 raft_consensus.cc:740] T 16713353f2684b9d96c80a3abbcdf74b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.604817 1175 consensus_queue.cc:260] T 16713353f2684b9d96c80a3abbcdf74b P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.606379 1175 ts_tablet_manager.cc:1434] T 16713353f2684b9d96c80a3abbcdf74b P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260202 02:52:14.606256 903 tablet_service.cc:1505] Processing CreateTablet for tablet 2ff6af17a6854451a1d7b904aa4f1f55 (DEFAULT_TABLE table=default.Table_2ed2b2d4b5dc4198bb62ee0e72f0e6ab [id=b4ab2ba86e1a49338a98d855ded99f5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.606710 1175 tablet_bootstrap.cc:492] T 2730d8b6812e487ea0aff4da45d24044 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:14.608572 1175 tablet_bootstrap.cc:654] T 2730d8b6812e487ea0aff4da45d24044 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.608721 1174 tablet_bootstrap.cc:654] T 220c3b32f56c478fbca7fbbd6135290c P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.609247 903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ff6af17a6854451a1d7b904aa4f1f55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.611263 1183 ts_tablet_manager.cc:1929] T 7781ccb7f1ea437fb1d61d3625a739a9 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:14.611207 754 tablet_service.cc:1505] Processing CreateTablet for tablet 72475ea720c6429aa68de9478b374280 (DEFAULT_TABLE table=default.Table_09d8f825c4c546839c0a820f1ddf0047 [id=ce957713e7fb49928d3fe57c3682f26f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.611439 1183 log.cc:1199] T 7781ccb7f1ea437fb1d61d3625a739a9 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/7781ccb7f1ea437fb1d61d3625a739a9
I20260202 02:52:14.611629 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72475ea720c6429aa68de9478b374280. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.611874 1183 ts_tablet_manager.cc:1950] T 7781ccb7f1ea437fb1d61d3625a739a9 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:14.616851 1183 tablet_replica.cc:333] T 6bac811b0f9f4276ac63786d97f02188 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:14.617089 1183 raft_consensus.cc:2243] T 6bac811b0f9f4276ac63786d97f02188 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:14.617332 1183 raft_consensus.cc:2272] T 6bac811b0f9f4276ac63786d97f02188 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.617722 1180 ts_tablet_manager.cc:1929] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:14.617882 1180 log.cc:1199] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/6bac811b0f9f4276ac63786d97f02188
I20260202 02:52:14.617975 1183 ts_tablet_manager.cc:1916] T 6bac811b0f9f4276ac63786d97f02188 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.618680 1180 ts_tablet_manager.cc:1950] T 6bac811b0f9f4276ac63786d97f02188 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:14.619905 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 6bac811b0f9f4276ac63786d97f02188 (table default.Table_5b9ebdf0ffec42f88f5bd9a3d0d158e9 [id=01b7ed0408cb46cabd569c0cf240bbaa]) successfully deleted
W20260202 02:52:14.620716 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 2730d8b6812e487ea0aff4da45d24044 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2730d8b6812e487ea0aff4da45d24044 already in progress: creating tablet
W20260202 02:52:14.621352 513 catalog_manager.cc:4952] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet ea0b6bb727404522b8291cbcd12879ea because the tablet was not found. No further retry: Not found: Tablet not found: ea0b6bb727404522b8291cbcd12879ea
W20260202 02:52:14.621938 513 catalog_manager.cc:4952] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 62241ababede41a1bd7f91620809f5f8 because the tablet was not found. No further retry: Not found: Tablet not found: 62241ababede41a1bd7f91620809f5f8
I20260202 02:52:14.622221 1180 tablet_replica.cc:333] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:14.622449 1180 raft_consensus.cc:2243] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:14.622711 1180 raft_consensus.cc:2272] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.623558 1174 tablet_bootstrap.cc:492] T 220c3b32f56c478fbca7fbbd6135290c P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:14.613452 1182 ts_tablet_manager.cc:1929] T 6bac811b0f9f4276ac63786d97f02188 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:14.623737 1180 ts_tablet_manager.cc:1916] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.624076 1182 log.cc:1199] T 6bac811b0f9f4276ac63786d97f02188 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/6bac811b0f9f4276ac63786d97f02188
I20260202 02:52:14.623814 1174 ts_tablet_manager.cc:1403] T 220c3b32f56c478fbca7fbbd6135290c P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.004s
I20260202 02:52:14.624610 1182 ts_tablet_manager.cc:1950] T 6bac811b0f9f4276ac63786d97f02188 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:14.625473 1174 raft_consensus.cc:359] T 220c3b32f56c478fbca7fbbd6135290c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.625730 1174 raft_consensus.cc:385] T 220c3b32f56c478fbca7fbbd6135290c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.625814 1174 raft_consensus.cc:740] T 220c3b32f56c478fbca7fbbd6135290c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.626020 1182 tablet_replica.cc:333] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:14.626094 1174 consensus_queue.cc:260] T 220c3b32f56c478fbca7fbbd6135290c P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.626339 1182 raft_consensus.cc:2243] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:14.626570 1182 raft_consensus.cc:2272] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.627696 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 6bac811b0f9f4276ac63786d97f02188 (table default.Table_5b9ebdf0ffec42f88f5bd9a3d0d158e9 [id=01b7ed0408cb46cabd569c0cf240bbaa]) successfully deleted
W20260202 02:52:14.628201 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 2730d8b6812e487ea0aff4da45d24044 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2730d8b6812e487ea0aff4da45d24044 already in progress: creating tablet
I20260202 02:52:14.630002 1182 ts_tablet_manager.cc:1916] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.631624 1041 tablet_service.cc:1505] Processing CreateTablet for tablet 2ff6af17a6854451a1d7b904aa4f1f55 (DEFAULT_TABLE table=default.Table_2ed2b2d4b5dc4198bb62ee0e72f0e6ab [id=b4ab2ba86e1a49338a98d855ded99f5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.632175 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ff6af17a6854451a1d7b904aa4f1f55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.633273 526 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet 80eb3013f5354022a2dd78bb294a8497 (table default.Table_437c92ec0b8441f18dc31cfc0c2f2972 [id=316b8a276d9b415fafe8511aa7c32fc5]). Expected version 1 got 0
I20260202 02:52:14.633627 519 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet fde2c1fba78248329f3662dd1036c7a2
I20260202 02:52:14.634117 519 catalog_manager.cc:6724] ReplaceTablet: tablet fde2c1fba78248329f3662dd1036c7a2 deleted and replaced by tablet 23c38ef4e3c840808ec36f859cf6f6de
I20260202 02:52:14.634243 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 7781ccb7f1ea437fb1d61d3625a739a9 (table default.Table_2f7cf4b6f60845c7977bbb0b5f2b7fd6 [id=1b7d3c09f6b84bd8b8634635c0db2295]) successfully deleted
I20260202 02:52:14.636783 1040 tablet_service.cc:1505] Processing CreateTablet for tablet 72475ea720c6429aa68de9478b374280 (DEFAULT_TABLE table=default.Table_09d8f825c4c546839c0a820f1ddf0047 [id=ce957713e7fb49928d3fe57c3682f26f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.637369 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72475ea720c6429aa68de9478b374280. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.643605 903 tablet_service.cc:1505] Processing CreateTablet for tablet 72475ea720c6429aa68de9478b374280 (DEFAULT_TABLE table=default.Table_09d8f825c4c546839c0a820f1ddf0047 [id=ce957713e7fb49928d3fe57c3682f26f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.644672 903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72475ea720c6429aa68de9478b374280. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.644922 1042 tablet_service.cc:1552] Processing DeleteTablet for tablet fde2c1fba78248329f3662dd1036c7a2 with delete_type TABLET_DATA_DELETED (replaced by tablet 23c38ef4e3c840808ec36f859cf6f6de) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:14.645289 1043 tablet_service.cc:1552] Processing DeleteTablet for tablet add934bf70ae4de8a40066f06fd94795 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:14.645560 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 72475ea720c6429aa68de9478b374280 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:14.645778 900 tablet_service.cc:1552] Processing DeleteTablet for tablet fde2c1fba78248329f3662dd1036c7a2 with delete_type TABLET_DATA_DELETED (replaced by tablet 23c38ef4e3c840808ec36f859cf6f6de) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:14.651429 1175 tablet_bootstrap.cc:492] T 2730d8b6812e487ea0aff4da45d24044 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:14.651646 1175 ts_tablet_manager.cc:1403] T 2730d8b6812e487ea0aff4da45d24044 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.045s user 0.011s sys 0.032s
I20260202 02:52:14.651998 1182 ts_tablet_manager.cc:1929] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:14.652146 1182 log.cc:1199] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/7781ccb7f1ea437fb1d61d3625a739a9
I20260202 02:52:14.652575 1175 raft_consensus.cc:359] T 2730d8b6812e487ea0aff4da45d24044 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.652815 1175 raft_consensus.cc:385] T 2730d8b6812e487ea0aff4da45d24044 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.652911 1175 raft_consensus.cc:740] T 2730d8b6812e487ea0aff4da45d24044 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.653276 1175 consensus_queue.cc:260] T 2730d8b6812e487ea0aff4da45d24044 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.655227 1175 ts_tablet_manager.cc:1434] T 2730d8b6812e487ea0aff4da45d24044 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260202 02:52:14.655604 1182 ts_tablet_manager.cc:1950] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:14.656262 756 tablet_service.cc:1552] Processing DeleteTablet for tablet fde2c1fba78248329f3662dd1036c7a2 with delete_type TABLET_DATA_DELETED (replaced by tablet 23c38ef4e3c840808ec36f859cf6f6de) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:14.657348 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_d1ead30f822b4a99ab5777a06313d94c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:14.658016 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d1ead30f822b4a99ab5777a06313d94c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.658217 1175 tablet_bootstrap.cc:492] T 005cda97ab5c4b70b8fc8ff66a05d996 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:14.658653 1182 tablet_replica.cc:333] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:14.658875 1182 raft_consensus.cc:2243] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:14.659372 1182 pending_rounds.cc:70] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5: Trying to abort 1 pending ops.
I20260202 02:52:14.659632 1182 pending_rounds.cc:77] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923007723425792 op_type: NO_OP noop_request { }
I20260202 02:52:14.659687 1183 ts_tablet_manager.cc:1929] T 6bac811b0f9f4276ac63786d97f02188 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:14.659832 1183 log.cc:1199] T 6bac811b0f9f4276ac63786d97f02188 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/6bac811b0f9f4276ac63786d97f02188
I20260202 02:52:14.659854 1182 raft_consensus.cc:2889] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:14.660003 1182 raft_consensus.cc:2272] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.660204 1183 ts_tablet_manager.cc:1950] T 6bac811b0f9f4276ac63786d97f02188 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:14.660519 1175 tablet_bootstrap.cc:654] T 005cda97ab5c4b70b8fc8ff66a05d996 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.661747 1183 tablet_replica.cc:333] T 397ed14bb33e44e59536243d0fa7e459 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:14.661979 1183 raft_consensus.cc:2243] T 397ed14bb33e44e59536243d0fa7e459 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:14.662158 1183 raft_consensus.cc:2272] T 397ed14bb33e44e59536243d0fa7e459 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.662989 1183 ts_tablet_manager.cc:1916] T 397ed14bb33e44e59536243d0fa7e459 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.663300 1174 ts_tablet_manager.cc:1434] T 220c3b32f56c478fbca7fbbd6135290c P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.039s user 0.001s sys 0.001s
I20260202 02:52:14.666222 522 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet 80eb3013f5354022a2dd78bb294a8497 (table default.Table_437c92ec0b8441f18dc31cfc0c2f2972 [id=316b8a276d9b415fafe8511aa7c32fc5]). Expected version 1 got 0
I20260202 02:52:14.667024 522 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_76150965bafb4889b2b08e618f28011a" } new_table_name: "default.Table_0842fd6a82e145e7a98b3fed2c0c6993" modify_external_catalogs: true
I20260202 02:52:14.667464 1174 tablet_bootstrap.cc:492] T 9c52693bdd214919b42cee1bbba95ed2 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:14.668670 521 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_3704b9992d3a420381b3dfe9f8635f95" } new_table_name: "default.Table_d142ff2cb10e491997bf0e302d6a9fa5" modify_external_catalogs: true
I20260202 02:52:14.669250 1175 tablet_bootstrap.cc:492] T 005cda97ab5c4b70b8fc8ff66a05d996 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:14.669328 1174 tablet_bootstrap.cc:654] T 9c52693bdd214919b42cee1bbba95ed2 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.669509 1175 ts_tablet_manager.cc:1403] T 005cda97ab5c4b70b8fc8ff66a05d996 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260202 02:52:14.669587 1183 ts_tablet_manager.cc:1929] T 397ed14bb33e44e59536243d0fa7e459 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:14.669735 1183 log.cc:1199] T 397ed14bb33e44e59536243d0fa7e459 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/397ed14bb33e44e59536243d0fa7e459
I20260202 02:52:14.670094 1183 ts_tablet_manager.cc:1950] T 397ed14bb33e44e59536243d0fa7e459 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:14.670280 524 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_437c92ec0b8441f18dc31cfc0c2f2972" } new_table_name: "default.Table_25c6d490f243450ab2d9b43ec9cda7bf" modify_external_catalogs: true
I20260202 02:52:14.670679 1175 raft_consensus.cc:359] T 005cda97ab5c4b70b8fc8ff66a05d996 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.670949 1175 raft_consensus.cc:385] T 005cda97ab5c4b70b8fc8ff66a05d996 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.671096 1175 raft_consensus.cc:740] T 005cda97ab5c4b70b8fc8ff66a05d996 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.671283 1182 ts_tablet_manager.cc:1916] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.671473 1175 consensus_queue.cc:260] T 005cda97ab5c4b70b8fc8ff66a05d996 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.671813 523 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_a2258ce8a41c4be1b2ee74dc0e31cd84" } new_table_name: "default.Table_4720e764353243a495245b92e5b913fb" modify_external_catalogs: true
I20260202 02:52:14.672762 1180 ts_tablet_manager.cc:1929] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:14.672909 1180 log.cc:1199] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/7781ccb7f1ea437fb1d61d3625a739a9
W20260202 02:52:14.673204 527 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 58f09c607cce4ccc82afd75c002ed1a8 consensus state has no leader
I20260202 02:52:14.674115 1180 ts_tablet_manager.cc:1950] T 7781ccb7f1ea437fb1d61d3625a739a9 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:14.674829 1175 ts_tablet_manager.cc:1434] T 005cda97ab5c4b70b8fc8ff66a05d996 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260202 02:52:14.675383 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 7781ccb7f1ea437fb1d61d3625a739a9 (table default.Table_2f7cf4b6f60845c7977bbb0b5f2b7fd6 [id=1b7d3c09f6b84bd8b8634635c0db2295]) successfully deleted
I20260202 02:52:14.675352 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 7781ccb7f1ea437fb1d61d3625a739a9 (table default.Table_2f7cf4b6f60845c7977bbb0b5f2b7fd6 [id=1b7d3c09f6b84bd8b8634635c0db2295]) successfully deleted
I20260202 02:52:14.676236 1183 tablet_replica.cc:333] T 9c52693bdd214919b42cee1bbba95ed2 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:14.676440 1183 raft_consensus.cc:2243] T 9c52693bdd214919b42cee1bbba95ed2 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:14.676635 1183 raft_consensus.cc:2272] T 9c52693bdd214919b42cee1bbba95ed2 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.677295 1180 tablet_replica.cc:333] T a6940c7f38ff496abf3c2dab66da5100 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:14.677320 1175 tablet_bootstrap.cc:492] T 6399b63cd8974ee49796f8779f9abc0e P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:14.677518 1180 raft_consensus.cc:2243] T a6940c7f38ff496abf3c2dab66da5100 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:14.677729 1180 raft_consensus.cc:2272] T a6940c7f38ff496abf3c2dab66da5100 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:14.677944 514 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 80eb3013f5354022a2dd78bb294a8497 (table default.Table_437c92ec0b8441f18dc31cfc0c2f2972 [id=316b8a276d9b415fafe8511aa7c32fc5], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 80eb3013f5354022a2dd78bb294a8497 consensus state has no leader
I20260202 02:52:14.679683 1175 tablet_bootstrap.cc:654] T 6399b63cd8974ee49796f8779f9abc0e P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.679689 1180 ts_tablet_manager.cc:1916] T a6940c7f38ff496abf3c2dab66da5100 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.681134 1182 ts_tablet_manager.cc:1929] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:14.681277 1182 log.cc:1199] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/397ed14bb33e44e59536243d0fa7e459
I20260202 02:52:14.682221 1183 ts_tablet_manager.cc:1916] T 9c52693bdd214919b42cee1bbba95ed2 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.683282 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_115696f06e984b47bd26eab721b76bf7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:14.683936 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_115696f06e984b47bd26eab721b76bf7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.685557 1174 tablet_bootstrap.cc:492] T 9c52693bdd214919b42cee1bbba95ed2 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:14.685559 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_0276ce9d8aaa44fa832bb88a8b1f9446"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:14.685724 1174 ts_tablet_manager.cc:1403] T 9c52693bdd214919b42cee1bbba95ed2 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.003s
W20260202 02:52:14.686172 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0276ce9d8aaa44fa832bb88a8b1f9446 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.686573 1174 raft_consensus.cc:359] T 9c52693bdd214919b42cee1bbba95ed2 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.686779 1174 raft_consensus.cc:385] T 9c52693bdd214919b42cee1bbba95ed2 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.686875 1174 raft_consensus.cc:740] T 9c52693bdd214919b42cee1bbba95ed2 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.687186 1174 consensus_queue.cc:260] T 9c52693bdd214919b42cee1bbba95ed2 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.687340 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 6bac811b0f9f4276ac63786d97f02188 (table default.Table_5b9ebdf0ffec42f88f5bd9a3d0d158e9 [id=01b7ed0408cb46cabd569c0cf240bbaa]) successfully deleted
W20260202 02:52:14.687623 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 2730d8b6812e487ea0aff4da45d24044 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2730d8b6812e487ea0aff4da45d24044 already in progress: creating tablet
I20260202 02:52:14.687835 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 397ed14bb33e44e59536243d0fa7e459 (table default.Table_62840640773d47f5889c6a82f84ffb32 [id=583762f37e29416886d3cc6219f7fd45]) successfully deleted
W20260202 02:52:14.688058 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 72475ea720c6429aa68de9478b374280 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 72475ea720c6429aa68de9478b374280 already in progress: creating tablet
W20260202 02:52:14.688266 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 81f35df38f284f0ca5c3791f3d9d9293 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 81f35df38f284f0ca5c3791f3d9d9293 already in progress: creating tablet
W20260202 02:52:14.688508 515 catalog_manager.cc:4952] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet add934bf70ae4de8a40066f06fd94795 because the tablet was not found. No further retry: Not found: Tablet not found: add934bf70ae4de8a40066f06fd94795
W20260202 02:52:14.688701 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 7e81d5780d9647b9baac0066282376ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7e81d5780d9647b9baac0066282376ea already in progress: creating tablet
W20260202 02:52:14.688971 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet e6b76967e5d3476997fa019c4a8afb91 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e6b76967e5d3476997fa019c4a8afb91 already in progress: creating tablet
I20260202 02:52:14.689381 1175 tablet_bootstrap.cc:492] T 6399b63cd8974ee49796f8779f9abc0e P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:14.689524 1182 ts_tablet_manager.cc:1950] T 397ed14bb33e44e59536243d0fa7e459 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:14.689657 1175 ts_tablet_manager.cc:1403] T 6399b63cd8974ee49796f8779f9abc0e P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.006s
I20260202 02:52:14.690449 1175 raft_consensus.cc:359] T 6399b63cd8974ee49796f8779f9abc0e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.690721 1175 raft_consensus.cc:385] T 6399b63cd8974ee49796f8779f9abc0e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.690871 1175 raft_consensus.cc:740] T 6399b63cd8974ee49796f8779f9abc0e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.690951 756 tablet_service.cc:1505] Processing CreateTablet for tablet 938813ee0a8a40b6bf0e537e3ec1a0d4 (DEFAULT_TABLE table=default.Table_2ae489b852d84e119c265efad1950d4b [id=1229c6d37b994a669d3363fdf6f8c6c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.691354 1182 tablet_replica.cc:333] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:14.691265 1175 consensus_queue.cc:260] T 6399b63cd8974ee49796f8779f9abc0e P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.691504 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 938813ee0a8a40b6bf0e537e3ec1a0d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.691545 1182 raft_consensus.cc:2243] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:14.691771 1182 pending_rounds.cc:70] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5: Trying to abort 1 pending ops.
I20260202 02:52:14.691855 1182 pending_rounds.cc:77] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923008770203648 op_type: NO_OP noop_request { }
I20260202 02:52:14.691952 1182 raft_consensus.cc:2889] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:14.692070 1182 raft_consensus.cc:2272] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.693603 1175 ts_tablet_manager.cc:1434] T 6399b63cd8974ee49796f8779f9abc0e P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260202 02:52:14.694195 1175 tablet_bootstrap.cc:492] T 81f35df38f284f0ca5c3791f3d9d9293 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:14.694432 1173 tablet_bootstrap.cc:492] T 220c3b32f56c478fbca7fbbd6135290c P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:14.694597 1173 ts_tablet_manager.cc:1403] T 220c3b32f56c478fbca7fbbd6135290c P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.194s user 0.008s sys 0.008s
I20260202 02:52:14.694655 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskYXxklX/build/asan/bin/kudu
/tmp/dist-test-taskYXxklX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.188:39077
--webserver_interface=127.0.104.188
--webserver_port=35523
--builtin_ntp_servers=127.0.104.148:45945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.190:33981,127.0.104.189:46271,127.0.104.188:39077
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260202 02:52:14.694868 1183 ts_tablet_manager.cc:1929] T 9c52693bdd214919b42cee1bbba95ed2 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:14.695004 1183 log.cc:1199] T 9c52693bdd214919b42cee1bbba95ed2 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/9c52693bdd214919b42cee1bbba95ed2
I20260202 02:52:14.695482 1183 ts_tablet_manager.cc:1950] T 9c52693bdd214919b42cee1bbba95ed2 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:14.695453 1173 raft_consensus.cc:359] T 220c3b32f56c478fbca7fbbd6135290c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.695658 1173 raft_consensus.cc:385] T 220c3b32f56c478fbca7fbbd6135290c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.695752 1173 raft_consensus.cc:740] T 220c3b32f56c478fbca7fbbd6135290c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.696061 1173 consensus_queue.cc:260] T 220c3b32f56c478fbca7fbbd6135290c P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.696732 903 tablet_service.cc:1505] Processing CreateTablet for tablet 938813ee0a8a40b6bf0e537e3ec1a0d4 (DEFAULT_TABLE table=default.Table_2ae489b852d84e119c265efad1950d4b [id=1229c6d37b994a669d3363fdf6f8c6c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.696918 1182 ts_tablet_manager.cc:1916] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.697171 903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 938813ee0a8a40b6bf0e537e3ec1a0d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.697467 1175 tablet_bootstrap.cc:654] T 81f35df38f284f0ca5c3791f3d9d9293 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.700416 1173 ts_tablet_manager.cc:1434] T 220c3b32f56c478fbca7fbbd6135290c P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260202 02:52:14.701124 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 397ed14bb33e44e59536243d0fa7e459 (table default.Table_62840640773d47f5889c6a82f84ffb32 [id=583762f37e29416886d3cc6219f7fd45]) successfully deleted
W20260202 02:52:14.701411 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 72475ea720c6429aa68de9478b374280 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 72475ea720c6429aa68de9478b374280 already in progress: creating tablet
W20260202 02:52:14.701655 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 81f35df38f284f0ca5c3791f3d9d9293 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 81f35df38f284f0ca5c3791f3d9d9293 already in progress: creating tablet
W20260202 02:52:14.701860 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 7e81d5780d9647b9baac0066282376ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7e81d5780d9647b9baac0066282376ea already in progress: creating tablet
W20260202 02:52:14.702103 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet add934bf70ae4de8a40066f06fd94795 because the tablet was not found. No further retry: Not found: Tablet not found: add934bf70ae4de8a40066f06fd94795
W20260202 02:52:14.702311 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet e6b76967e5d3476997fa019c4a8afb91 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e6b76967e5d3476997fa019c4a8afb91 already in progress: creating tablet
I20260202 02:52:14.703891 1173 tablet_bootstrap.cc:492] T c9ce1b27212646e691bad953c9fbe14d P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:14.704797 758 tablet_service.cc:1505] Processing CreateTablet for tablet 672ea3f731ba42ff8b1ff47db039ed22 (DEFAULT_TABLE table=default.Table_85da33c295954de58e2c6290005ac865 [id=ed3c3f07e9ef4a568d36ebd62e0bddc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.705231 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 672ea3f731ba42ff8b1ff47db039ed22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.705442 1182 ts_tablet_manager.cc:1929] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:14.705655 1173 tablet_bootstrap.cc:654] T c9ce1b27212646e691bad953c9fbe14d P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.705718 1182 log.cc:1199] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/9c52693bdd214919b42cee1bbba95ed2
I20260202 02:52:14.706496 1182 ts_tablet_manager.cc:1950] T 9c52693bdd214919b42cee1bbba95ed2 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:14.707654 1182 tablet_replica.cc:333] T 005cda97ab5c4b70b8fc8ff66a05d996 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:14.707911 1182 raft_consensus.cc:2243] T 005cda97ab5c4b70b8fc8ff66a05d996 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:14.708106 1182 raft_consensus.cc:2272] T 005cda97ab5c4b70b8fc8ff66a05d996 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.708302 749 tablet_service.cc:1505] Processing CreateTablet for tablet 4de41396cffd4d32b86c701b538d1e01 (DEFAULT_TABLE table=default.Table_3704b9992d3a420381b3dfe9f8635f95 [id=a0b8442c02974d6596db002c9292be0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.708829 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4de41396cffd4d32b86c701b538d1e01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.710263 1182 ts_tablet_manager.cc:1916] T 005cda97ab5c4b70b8fc8ff66a05d996 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.716526 754 tablet_service.cc:1505] Processing CreateTablet for tablet fc93352b24e345f99d15085e4886c748 (DEFAULT_TABLE table=default.Table_8aee8d27e02942fb91879229b190457f [id=c59468c8c5b7480d88cb4b7d447a5975]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.717000 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc93352b24e345f99d15085e4886c748. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.717898 1182 ts_tablet_manager.cc:1929] T 005cda97ab5c4b70b8fc8ff66a05d996 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:14.718071 1182 log.cc:1199] T 005cda97ab5c4b70b8fc8ff66a05d996 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/005cda97ab5c4b70b8fc8ff66a05d996
I20260202 02:52:14.718475 1182 ts_tablet_manager.cc:1950] T 005cda97ab5c4b70b8fc8ff66a05d996 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:14.719697 1182 tablet_replica.cc:333] T c9ce1b27212646e691bad953c9fbe14d P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:14.720968 1040 tablet_service.cc:1505] Processing CreateTablet for tablet 938813ee0a8a40b6bf0e537e3ec1a0d4 (DEFAULT_TABLE table=default.Table_2ae489b852d84e119c265efad1950d4b [id=1229c6d37b994a669d3363fdf6f8c6c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.721449 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 938813ee0a8a40b6bf0e537e3ec1a0d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.724764 1041 tablet_service.cc:1505] Processing CreateTablet for tablet fc93352b24e345f99d15085e4886c748 (DEFAULT_TABLE table=default.Table_8aee8d27e02942fb91879229b190457f [id=c59468c8c5b7480d88cb4b7d447a5975]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.725323 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc93352b24e345f99d15085e4886c748. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.725893 1173 tablet_bootstrap.cc:492] T c9ce1b27212646e691bad953c9fbe14d P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:14.726064 1173 ts_tablet_manager.cc:1403] T c9ce1b27212646e691bad953c9fbe14d P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.000s
I20260202 02:52:14.726851 1173 raft_consensus.cc:359] T c9ce1b27212646e691bad953c9fbe14d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.727048 1173 raft_consensus.cc:385] T c9ce1b27212646e691bad953c9fbe14d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.727121 1173 raft_consensus.cc:740] T c9ce1b27212646e691bad953c9fbe14d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.727779 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 672ea3f731ba42ff8b1ff47db039ed22 (DEFAULT_TABLE table=default.Table_85da33c295954de58e2c6290005ac865 [id=ed3c3f07e9ef4a568d36ebd62e0bddc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.728315 1043 tablet_service.cc:1552] Processing DeleteTablet for tablet 220c3b32f56c478fbca7fbbd6135290c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:14.728698 1183 tablet_replica.cc:333] T 220c3b32f56c478fbca7fbbd6135290c P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:14.732193 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 672ea3f731ba42ff8b1ff47db039ed22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.732057 1173 consensus_queue.cc:260] T c9ce1b27212646e691bad953c9fbe14d P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.734974 1173 ts_tablet_manager.cc:1434] T c9ce1b27212646e691bad953c9fbe14d P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260202 02:52:14.736400 1173 tablet_bootstrap.cc:492] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:14.736574 1183 raft_consensus.cc:2243] T 220c3b32f56c478fbca7fbbd6135290c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:14.736835 1183 raft_consensus.cc:2272] T 220c3b32f56c478fbca7fbbd6135290c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.738456 1173 tablet_bootstrap.cc:654] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.739691 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 9c52693bdd214919b42cee1bbba95ed2 (table default.Table_c2deb2be896d43e0a7bb49582c9276f9 [id=92e6422d4fc3439abab376a52ad8876a]) successfully deleted
W20260202 02:52:14.740038 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet c2cc2250701d46a7abe7be03b11d45c8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c2cc2250701d46a7abe7be03b11d45c8 already in progress: creating tablet
W20260202 02:52:14.740257 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 005cda97ab5c4b70b8fc8ff66a05d996 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 005cda97ab5c4b70b8fc8ff66a05d996 already in progress: creating tablet
W20260202 02:52:14.740482 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet fde2c1fba78248329f3662dd1036c7a2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fde2c1fba78248329f3662dd1036c7a2 already in progress: creating tablet
I20260202 02:52:14.740552 1183 ts_tablet_manager.cc:1916] T 220c3b32f56c478fbca7fbbd6135290c P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:14.740658 515 catalog_manager.cc:4952] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet add934bf70ae4de8a40066f06fd94795 because the tablet was not found. No further retry: Not found: Tablet not found: add934bf70ae4de8a40066f06fd94795
W20260202 02:52:14.740890 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 72475ea720c6429aa68de9478b374280 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 72475ea720c6429aa68de9478b374280 already in progress: creating tablet
I20260202 02:52:14.741621 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 9c52693bdd214919b42cee1bbba95ed2 (table default.Table_c2deb2be896d43e0a7bb49582c9276f9 [id=92e6422d4fc3439abab376a52ad8876a]) successfully deleted
I20260202 02:52:14.741806 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 005cda97ab5c4b70b8fc8ff66a05d996 (table default.Table_76150965bafb4889b2b08e618f28011a [id=4e076b2023be4c888ac1c27babde1607]) successfully deleted
W20260202 02:52:14.742031 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet c2cc2250701d46a7abe7be03b11d45c8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c2cc2250701d46a7abe7be03b11d45c8 already in progress: creating tablet
I20260202 02:52:14.743979 525 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_c2deb2be896d43e0a7bb49582c9276f9" } modify_external_catalogs: true
I20260202 02:52:14.744282 525 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_c2deb2be896d43e0a7bb49582c9276f9" } modify_external_catalogs: true
I20260202 02:52:14.747532 527 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_5610c9ec35d74b1b909b83c855f21c68" } modify_external_catalogs: true
I20260202 02:52:14.747745 527 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_5610c9ec35d74b1b909b83c855f21c68" } modify_external_catalogs: true
I20260202 02:52:14.748199 1183 ts_tablet_manager.cc:1929] T 220c3b32f56c478fbca7fbbd6135290c P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:14.753824 1041 tablet_service.cc:1505] Processing CreateTablet for tablet 4de41396cffd4d32b86c701b538d1e01 (DEFAULT_TABLE table=default.Table_3704b9992d3a420381b3dfe9f8635f95 [id=a0b8442c02974d6596db002c9292be0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.754474 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4de41396cffd4d32b86c701b538d1e01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.755038 750 tablet_service.cc:1505] Processing CreateTablet for tablet 13309b20a96240b5aa9fbb691cbed981 (DEFAULT_TABLE table=default.Table_5fa327d665484811934aa3b11188a76d [id=f7c5b24c065443d89a2b1f9c34409c9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.755429 1183 log.cc:1199] T 220c3b32f56c478fbca7fbbd6135290c P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/220c3b32f56c478fbca7fbbd6135290c
I20260202 02:52:14.755635 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13309b20a96240b5aa9fbb691cbed981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.756019 1183 ts_tablet_manager.cc:1950] T 220c3b32f56c478fbca7fbbd6135290c P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:14.741941 902 tablet_service.cc:1505] Processing CreateTablet for tablet 4de41396cffd4d32b86c701b538d1e01 (DEFAULT_TABLE table=default.Table_3704b9992d3a420381b3dfe9f8635f95 [id=a0b8442c02974d6596db002c9292be0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.719925 1182 raft_consensus.cc:2243] T c9ce1b27212646e691bad953c9fbe14d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260202 02:52:14.758721 521 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4de41396cffd4d32b86c701b538d1e01 consensus state has no leader
I20260202 02:52:14.758890 1182 raft_consensus.cc:2272] T c9ce1b27212646e691bad953c9fbe14d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.759027 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4de41396cffd4d32b86c701b538d1e01. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:14.759274 524 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 80eb3013f5354022a2dd78bb294a8497 consensus state has no leader
I20260202 02:52:14.759711 1182 ts_tablet_manager.cc:1916] T c9ce1b27212646e691bad953c9fbe14d P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.762153 901 tablet_service.cc:1505] Processing CreateTablet for tablet 13309b20a96240b5aa9fbb691cbed981 (DEFAULT_TABLE table=default.Table_5fa327d665484811934aa3b11188a76d [id=f7c5b24c065443d89a2b1f9c34409c9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.762631 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13309b20a96240b5aa9fbb691cbed981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.741695 899 tablet_service.cc:1505] Processing CreateTablet for tablet 672ea3f731ba42ff8b1ff47db039ed22 (DEFAULT_TABLE table=default.Table_85da33c295954de58e2c6290005ac865 [id=ed3c3f07e9ef4a568d36ebd62e0bddc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.741124 900 tablet_service.cc:1505] Processing CreateTablet for tablet fc93352b24e345f99d15085e4886c748 (DEFAULT_TABLE table=default.Table_8aee8d27e02942fb91879229b190457f [id=c59468c8c5b7480d88cb4b7d447a5975]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.765450 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc93352b24e345f99d15085e4886c748. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.765410 903 tablet_service.cc:1505] Processing CreateTablet for tablet f24c0b99574d437a99b273204f105460 (DEFAULT_TABLE table=default.Table_506775b4d1a14bf1b91c9d951c9a88ec [id=9a8a252de32848288c9b3752c7d3feb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.765790 903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f24c0b99574d437a99b273204f105460. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:14.765972 522 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4f2a9404f7424dc78b5bd062ebd8cb53 has no consensus state
I20260202 02:52:14.766286 522 master_service.cc:954] ReplaceTablet: received request to replace tablet e6b76967e5d3476997fa019c4a8afb91 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:14.764621 751 tablet_service.cc:1505] Processing CreateTablet for tablet ffd8aa2bc6e344b78dce518411997fb5 (DEFAULT_TABLE table=default.Table_4a3ef984ca894b8b8f4301fecb719b5b [id=3ccb047c30974c0f9cbe778971e61644]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.767697 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffd8aa2bc6e344b78dce518411997fb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.773025 520 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_8aee8d27e02942fb91879229b190457f" } modify_external_catalogs: true
I20260202 02:52:14.773265 520 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_8aee8d27e02942fb91879229b190457f" } modify_external_catalogs: true
I20260202 02:52:14.774654 524 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_04527bd0309f433aa915767172dcd280" } modify_external_catalogs: true
I20260202 02:52:14.774868 524 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_04527bd0309f433aa915767172dcd280" } modify_external_catalogs: true
I20260202 02:52:14.776191 521 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_b3f54615950542f2b090477b297f80ed" } modify_external_catalogs: true
I20260202 02:52:14.776381 521 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_b3f54615950542f2b090477b297f80ed" } modify_external_catalogs: true
I20260202 02:52:14.776294 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 220c3b32f56c478fbca7fbbd6135290c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:14.777326 749 tablet_service.cc:1505] Processing CreateTablet for tablet f24c0b99574d437a99b273204f105460 (DEFAULT_TABLE table=default.Table_506775b4d1a14bf1b91c9d951c9a88ec [id=9a8a252de32848288c9b3752c7d3feb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.777761 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f24c0b99574d437a99b273204f105460. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.776981 898 tablet_service.cc:1505] Processing CreateTablet for tablet ffd8aa2bc6e344b78dce518411997fb5 (DEFAULT_TABLE table=default.Table_4a3ef984ca894b8b8f4301fecb719b5b [id=3ccb047c30974c0f9cbe778971e61644]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.779700 897 tablet_service.cc:1552] Processing DeleteTablet for tablet 2730d8b6812e487ea0aff4da45d24044 with delete_type TABLET_DATA_DELETED (replaced by tablet f24c0b99574d437a99b273204f105460) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:14.779736 896 tablet_service.cc:1552] Processing DeleteTablet for tablet 005cda97ab5c4b70b8fc8ff66a05d996 with delete_type TABLET_DATA_DELETED (replaced by tablet 4f2a9404f7424dc78b5bd062ebd8cb53) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:14.779970 1174 ts_tablet_manager.cc:1434] T 9c52693bdd214919b42cee1bbba95ed2 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.094s user 0.001s sys 0.001s
I20260202 02:52:14.782111 1174 tablet_bootstrap.cc:492] T 16713353f2684b9d96c80a3abbcdf74b P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
W20260202 02:52:14.786770 515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.190:33981, user_credentials={real_user=slave}} blocked reactor thread for 44677.6us
I20260202 02:52:14.788297 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 672ea3f731ba42ff8b1ff47db039ed22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.791308 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffd8aa2bc6e344b78dce518411997fb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.792582 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 13309b20a96240b5aa9fbb691cbed981 (DEFAULT_TABLE table=default.Table_5fa327d665484811934aa3b11188a76d [id=f7c5b24c065443d89a2b1f9c34409c9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.792765 1174 tablet_bootstrap.cc:654] T 16713353f2684b9d96c80a3abbcdf74b P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.793133 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13309b20a96240b5aa9fbb691cbed981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.794967 1173 tablet_bootstrap.cc:492] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:14.795207 1173 ts_tablet_manager.cc:1403] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.059s user 0.003s sys 0.000s
I20260202 02:52:14.796072 1173 raft_consensus.cc:359] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.796383 1173 raft_consensus.cc:385] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.796489 1173 raft_consensus.cc:740] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.797875 754 tablet_service.cc:1505] Processing CreateTablet for tablet 575016c0e9f84e99b962c5f14ffa96cc (DEFAULT_TABLE table=default.Table_a2258ce8a41c4be1b2ee74dc0e31cd84 [id=f2276e1c2ca04a70804053da61dd3447]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.798365 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 575016c0e9f84e99b962c5f14ffa96cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.800547 1184 raft_consensus.cc:493] T 6399b63cd8974ee49796f8779f9abc0e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:14.800786 1184 raft_consensus.cc:515] T 6399b63cd8974ee49796f8779f9abc0e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.801569 1184 leader_election.cc:290] T 6399b63cd8974ee49796f8779f9abc0e P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:14.802376 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6399b63cd8974ee49796f8779f9abc0e" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:14.803107 859 leader_election.cc:343] T 6399b63cd8974ee49796f8779f9abc0e P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:14.803521 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6399b63cd8974ee49796f8779f9abc0e" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:14.805876 1040 tablet_service.cc:1552] Processing DeleteTablet for tablet c9ce1b27212646e691bad953c9fbe14d with delete_type TABLET_DATA_DELETED (replaced by tablet 2730d8b6812e487ea0aff4da45d24044) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:14.806150 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 220c3b32f56c478fbca7fbbd6135290c (table default.Table_c5a4aa6f50fb472c8982feedb30982a9 [id=7af0d6d84db7404ebed9db660a633c12]) successfully deleted
I20260202 02:52:14.806362 1183 tablet_replica.cc:333] T c9ce1b27212646e691bad953c9fbe14d P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:14.806599 1183 raft_consensus.cc:2243] T c9ce1b27212646e691bad953c9fbe14d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:14.806833 522 master_service.cc:954] ReplaceTablet: received request to replace tablet c2cc2250701d46a7abe7be03b11d45c8 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:14.806895 1183 raft_consensus.cc:2272] T c9ce1b27212646e691bad953c9fbe14d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.807478 1041 tablet_service.cc:1505] Processing CreateTablet for tablet f24c0b99574d437a99b273204f105460 (DEFAULT_TABLE table=default.Table_506775b4d1a14bf1b91c9d951c9a88ec [id=9a8a252de32848288c9b3752c7d3feb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.807822 1183 ts_tablet_manager.cc:1916] T c9ce1b27212646e691bad953c9fbe14d P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.808121 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f24c0b99574d437a99b273204f105460. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.808619 519 master_service.cc:954] ReplaceTablet: received request to replace tablet 16713353f2684b9d96c80a3abbcdf74b from {username='slave'} at 127.0.0.1:47430
W20260202 02:52:14.812690 857 leader_election.cc:343] T 6399b63cd8974ee49796f8779f9abc0e P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:14.812903 857 leader_election.cc:304] T 6399b63cd8974ee49796f8779f9abc0e P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:14.813509 1184 raft_consensus.cc:2749] T 6399b63cd8974ee49796f8779f9abc0e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:14.814869 1182 ts_tablet_manager.cc:1929] T c9ce1b27212646e691bad953c9fbe14d P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:14.815042 1182 log.cc:1199] T c9ce1b27212646e691bad953c9fbe14d P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/c9ce1b27212646e691bad953c9fbe14d
I20260202 02:52:14.815470 1182 ts_tablet_manager.cc:1950] T c9ce1b27212646e691bad953c9fbe14d P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:14.816682 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet c9ce1b27212646e691bad953c9fbe14d (table default.Table_506775b4d1a14bf1b91c9d951c9a88ec [id=9a8a252de32848288c9b3752c7d3feb7]) successfully deleted
W20260202 02:52:14.817723 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet add934bf70ae4de8a40066f06fd94795 because the tablet was not found. No further retry: Not found: Tablet not found: add934bf70ae4de8a40066f06fd94795
W20260202 02:52:14.818596 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 72475ea720c6429aa68de9478b374280 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 72475ea720c6429aa68de9478b374280 already in progress: creating tablet
W20260202 02:52:14.819334 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet fde2c1fba78248329f3662dd1036c7a2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fde2c1fba78248329f3662dd1036c7a2 already in progress: creating tablet
I20260202 02:52:14.823045 1182 tablet_replica.cc:333] T 2730d8b6812e487ea0aff4da45d24044 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:14.823414 1182 raft_consensus.cc:2243] T 2730d8b6812e487ea0aff4da45d24044 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:14.823973 1182 raft_consensus.cc:2272] T 2730d8b6812e487ea0aff4da45d24044 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.824687 756 tablet_service.cc:1505] Processing CreateTablet for tablet 251455168e33430aafea8e0de42ca1b8 (DEFAULT_TABLE table=default.Table_86f34199535c451cb54a2e20e70f1149 [id=4f19275f416a425c9b9d100b2a83870a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.825230 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 251455168e33430aafea8e0de42ca1b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.825631 1182 ts_tablet_manager.cc:1916] T 2730d8b6812e487ea0aff4da45d24044 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.826839 522 master_service.cc:954] ReplaceTablet: received request to replace tablet e6b76967e5d3476997fa019c4a8afb91 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:14.826928 901 tablet_service.cc:1505] Processing CreateTablet for tablet add934bf70ae4de8a40066f06fd94795 (DEFAULT_TABLE table=default.Table_9fb69280bd6048d898c8e92f2ef9f61e [id=7cf1e30412cb48ab962f526efee72121]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.827539 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet add934bf70ae4de8a40066f06fd94795. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.828722 751 tablet_service.cc:1505] Processing CreateTablet for tablet add934bf70ae4de8a40066f06fd94795 (DEFAULT_TABLE table=default.Table_9fb69280bd6048d898c8e92f2ef9f61e [id=7cf1e30412cb48ab962f526efee72121]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.829265 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet add934bf70ae4de8a40066f06fd94795. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.832746 899 tablet_service.cc:1505] Processing CreateTablet for tablet 575016c0e9f84e99b962c5f14ffa96cc (DEFAULT_TABLE table=default.Table_a2258ce8a41c4be1b2ee74dc0e31cd84 [id=f2276e1c2ca04a70804053da61dd3447]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.833336 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 575016c0e9f84e99b962c5f14ffa96cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.834022 1044 tablet_service.cc:1505] Processing CreateTablet for tablet ffd8aa2bc6e344b78dce518411997fb5 (DEFAULT_TABLE table=default.Table_4a3ef984ca894b8b8f4301fecb719b5b [id=3ccb047c30974c0f9cbe778971e61644]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.834712 1040 tablet_service.cc:1505] Processing CreateTablet for tablet 575016c0e9f84e99b962c5f14ffa96cc (DEFAULT_TABLE table=default.Table_a2258ce8a41c4be1b2ee74dc0e31cd84 [id=f2276e1c2ca04a70804053da61dd3447]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.835270 1043 tablet_service.cc:1505] Processing CreateTablet for tablet 251455168e33430aafea8e0de42ca1b8 (DEFAULT_TABLE table=default.Table_86f34199535c451cb54a2e20e70f1149 [id=4f19275f416a425c9b9d100b2a83870a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.835826 1042 tablet_service.cc:1505] Processing CreateTablet for tablet add934bf70ae4de8a40066f06fd94795 (DEFAULT_TABLE table=default.Table_9fb69280bd6048d898c8e92f2ef9f61e [id=7cf1e30412cb48ab962f526efee72121]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.836485 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffd8aa2bc6e344b78dce518411997fb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.836822 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 251455168e33430aafea8e0de42ca1b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.837605 898 tablet_service.cc:1505] Processing CreateTablet for tablet 251455168e33430aafea8e0de42ca1b8 (DEFAULT_TABLE table=default.Table_86f34199535c451cb54a2e20e70f1149 [id=4f19275f416a425c9b9d100b2a83870a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:14.838085 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 251455168e33430aafea8e0de42ca1b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.842511 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 575016c0e9f84e99b962c5f14ffa96cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.848642 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet add934bf70ae4de8a40066f06fd94795. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:14.852124 1173 consensus_queue.cc:260] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.853991 1173 ts_tablet_manager.cc:1434] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.059s user 0.003s sys 0.000s
I20260202 02:52:14.854369 1173 tablet_bootstrap.cc:492] T 005cda97ab5c4b70b8fc8ff66a05d996 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
W20260202 02:52:14.859836 523 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 575016c0e9f84e99b962c5f14ffa96cc consensus state has no leader
I20260202 02:52:14.861770 528 master_service.cc:954] ReplaceTablet: received request to replace tablet fc93352b24e345f99d15085e4886c748 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:14.865546 525 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet fc690a7c8e214d1d921488e7f0b1ce8d
I20260202 02:52:14.865890 527 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet 7d2ce50ede9d4d26bf24bd176a7d663d
I20260202 02:52:14.866154 520 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet fc93352b24e345f99d15085e4886c748
I20260202 02:52:14.866962 521 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet 82c080d576694984a2677238b0b7f302
I20260202 02:52:14.867251 519 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 16713353f2684b9d96c80a3abbcdf74b
I20260202 02:52:14.867633 519 catalog_manager.cc:6724] ReplaceTablet: tablet 16713353f2684b9d96c80a3abbcdf74b deleted and replaced by tablet 952d760159844ad08269654264e6381b
I20260202 02:52:14.868691 900 tablet_service.cc:1552] Processing DeleteTablet for tablet 16713353f2684b9d96c80a3abbcdf74b with delete_type TABLET_DATA_DELETED (replaced by tablet 952d760159844ad08269654264e6381b) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:14.869721 524 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 6399b63cd8974ee49796f8779f9abc0e
I20260202 02:52:14.870035 1180 ts_tablet_manager.cc:1929] T a6940c7f38ff496abf3c2dab66da5100 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:14.870190 1180 log.cc:1199] T a6940c7f38ff496abf3c2dab66da5100 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/a6940c7f38ff496abf3c2dab66da5100
I20260202 02:52:14.870585 1180 ts_tablet_manager.cc:1950] T a6940c7f38ff496abf3c2dab66da5100 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:14.871544 1180 tablet_replica.cc:333] T 397ed14bb33e44e59536243d0fa7e459 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:14.871811 900 tablet_service.cc:1552] Processing DeleteTablet for tablet 6399b63cd8974ee49796f8779f9abc0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:14.872179 1180 raft_consensus.cc:2243] T 397ed14bb33e44e59536243d0fa7e459 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:14.872447 1180 raft_consensus.cc:2272] T 397ed14bb33e44e59536243d0fa7e459 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.873358 1180 ts_tablet_manager.cc:1916] T 397ed14bb33e44e59536243d0fa7e459 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.873430 526 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 80eb3013f5354022a2dd78bb294a8497 (table default.Table_25c6d490f243450ab2d9b43ec9cda7bf [id=316b8a276d9b415fafe8511aa7c32fc5]). Expected version 2 got 0
I20260202 02:52:14.875638 1174 tablet_bootstrap.cc:492] T 16713353f2684b9d96c80a3abbcdf74b P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:14.875926 1174 ts_tablet_manager.cc:1403] T 16713353f2684b9d96c80a3abbcdf74b P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.094s user 0.002s sys 0.002s
I20260202 02:52:14.876629 528 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 80eb3013f5354022a2dd78bb294a8497 (table default.Table_25c6d490f243450ab2d9b43ec9cda7bf [id=316b8a276d9b415fafe8511aa7c32fc5]). Expected version 2 got 0
I20260202 02:52:14.876983 1174 raft_consensus.cc:359] T 16713353f2684b9d96c80a3abbcdf74b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.877254 1174 raft_consensus.cc:385] T 16713353f2684b9d96c80a3abbcdf74b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.877377 1174 raft_consensus.cc:740] T 16713353f2684b9d96c80a3abbcdf74b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.877682 1174 consensus_queue.cc:260] T 16713353f2684b9d96c80a3abbcdf74b P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.880645 1182 ts_tablet_manager.cc:1929] T 2730d8b6812e487ea0aff4da45d24044 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:14.881862 1174 ts_tablet_manager.cc:1434] T 16713353f2684b9d96c80a3abbcdf74b P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260202 02:52:14.884071 1182 log.cc:1199] T 2730d8b6812e487ea0aff4da45d24044 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/2730d8b6812e487ea0aff4da45d24044
I20260202 02:52:14.884585 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet a6940c7f38ff496abf3c2dab66da5100 (table default.Table_011de32f5bb648439181d1154b82b2c5 [id=1d7c11d3ee26455e995b30f69a967132]) successfully deleted
I20260202 02:52:14.884691 1182 ts_tablet_manager.cc:1950] T 2730d8b6812e487ea0aff4da45d24044 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:14.886322 1174 tablet_bootstrap.cc:492] T c9ce1b27212646e691bad953c9fbe14d P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:14.886868 1182 tablet_replica.cc:333] T 16713353f2684b9d96c80a3abbcdf74b P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:14.887508 1182 raft_consensus.cc:2243] T 16713353f2684b9d96c80a3abbcdf74b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:14.887959 1182 raft_consensus.cc:2272] T 16713353f2684b9d96c80a3abbcdf74b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.888217 1174 tablet_bootstrap.cc:654] T c9ce1b27212646e691bad953c9fbe14d P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.889180 749 tablet_service.cc:1552] Processing DeleteTablet for tablet fc93352b24e345f99d15085e4886c748 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:14.889521 1182 ts_tablet_manager.cc:1916] T 16713353f2684b9d96c80a3abbcdf74b P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.890105 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 6399b63cd8974ee49796f8779f9abc0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:14.890403 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 16713353f2684b9d96c80a3abbcdf74b with delete_type TABLET_DATA_DELETED (replaced by tablet 952d760159844ad08269654264e6381b) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:14.856310 1173 tablet_bootstrap.cc:654] T 005cda97ab5c4b70b8fc8ff66a05d996 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.856468 1183 ts_tablet_manager.cc:1929] T c9ce1b27212646e691bad953c9fbe14d P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:14.886610 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet fc93352b24e345f99d15085e4886c748 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:14.892161 1183 log.cc:1199] T c9ce1b27212646e691bad953c9fbe14d P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/c9ce1b27212646e691bad953c9fbe14d
I20260202 02:52:14.887420 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet 16713353f2684b9d96c80a3abbcdf74b with delete_type TABLET_DATA_DELETED (replaced by tablet 952d760159844ad08269654264e6381b) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:14.887903 1040 tablet_service.cc:1552] Processing DeleteTablet for tablet 6399b63cd8974ee49796f8779f9abc0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:14.892565 1183 ts_tablet_manager.cc:1950] T c9ce1b27212646e691bad953c9fbe14d P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:14.894044 1183 tablet_replica.cc:333] T 16713353f2684b9d96c80a3abbcdf74b P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:14.894260 1183 raft_consensus.cc:2243] T 16713353f2684b9d96c80a3abbcdf74b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:14.894439 1183 raft_consensus.cc:2272] T 16713353f2684b9d96c80a3abbcdf74b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.895287 1183 ts_tablet_manager.cc:1916] T 16713353f2684b9d96c80a3abbcdf74b P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.896536 899 tablet_service.cc:1552] Processing DeleteTablet for tablet fc93352b24e345f99d15085e4886c748 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:14.900838 1175 tablet_bootstrap.cc:492] T 81f35df38f284f0ca5c3791f3d9d9293 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:14.901166 1175 ts_tablet_manager.cc:1403] T 81f35df38f284f0ca5c3791f3d9d9293 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.207s user 0.052s sys 0.128s
I20260202 02:52:14.901369 1182 ts_tablet_manager.cc:1929] T 16713353f2684b9d96c80a3abbcdf74b P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:14.901562 1182 log.cc:1199] T 16713353f2684b9d96c80a3abbcdf74b P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/16713353f2684b9d96c80a3abbcdf74b
I20260202 02:52:14.901999 1182 ts_tablet_manager.cc:1950] T 16713353f2684b9d96c80a3abbcdf74b P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:14.902760 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet c9ce1b27212646e691bad953c9fbe14d (table default.Table_506775b4d1a14bf1b91c9d951c9a88ec [id=9a8a252de32848288c9b3752c7d3feb7]) successfully deleted
I20260202 02:52:14.903008 1174 tablet_bootstrap.cc:492] T c9ce1b27212646e691bad953c9fbe14d P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:14.902789 1175 raft_consensus.cc:359] T 81f35df38f284f0ca5c3791f3d9d9293 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
W20260202 02:52:14.903090 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet fc93352b24e345f99d15085e4886c748 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fc93352b24e345f99d15085e4886c748 already in progress: creating tablet
I20260202 02:52:14.903174 1174 ts_tablet_manager.cc:1403] T c9ce1b27212646e691bad953c9fbe14d P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.000s
I20260202 02:52:14.903133 1175 raft_consensus.cc:385] T 81f35df38f284f0ca5c3791f3d9d9293 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.903322 1175 raft_consensus.cc:740] T 81f35df38f284f0ca5c3791f3d9d9293 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.903398 1182 tablet_replica.cc:333] T 6399b63cd8974ee49796f8779f9abc0e P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:14.903693 1182 raft_consensus.cc:2243] T 6399b63cd8974ee49796f8779f9abc0e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:14.903909 1182 raft_consensus.cc:2272] T 6399b63cd8974ee49796f8779f9abc0e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.903997 1174 raft_consensus.cc:359] T c9ce1b27212646e691bad953c9fbe14d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.904253 1174 raft_consensus.cc:385] T c9ce1b27212646e691bad953c9fbe14d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.903847 1175 consensus_queue.cc:260] T 81f35df38f284f0ca5c3791f3d9d9293 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.904439 1174 raft_consensus.cc:740] T c9ce1b27212646e691bad953c9fbe14d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.904731 1174 consensus_queue.cc:260] T c9ce1b27212646e691bad953c9fbe14d P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.905406 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_5c9c0f09396a4a0e8b3d676fcb2dbb2c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:14.905575 1175 ts_tablet_manager.cc:1434] T 81f35df38f284f0ca5c3791f3d9d9293 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260202 02:52:14.906060 1182 ts_tablet_manager.cc:1916] T 6399b63cd8974ee49796f8779f9abc0e P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.906847 1174 ts_tablet_manager.cc:1434] T c9ce1b27212646e691bad953c9fbe14d P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260202 02:52:14.907562 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_5e1d4f5a21c14fd2bc84e32d68d065bb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:14.908743 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_7ef6ef5b916e42709217b7c44fb489d0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:14.909555 1174 tablet_bootstrap.cc:492] T 005cda97ab5c4b70b8fc8ff66a05d996 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:14.910066 1175 tablet_bootstrap.cc:492] T fde2c1fba78248329f3662dd1036c7a2 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:14.910498 522 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_506775b4d1a14bf1b91c9d951c9a88ec" } new_table_name: "default.Table_7ff64c2b44d0417ea6445647a80a5b86" modify_external_catalogs: true
I20260202 02:52:14.911715 1174 tablet_bootstrap.cc:654] T 005cda97ab5c4b70b8fc8ff66a05d996 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.911724 1175 tablet_bootstrap.cc:654] T fde2c1fba78248329f3662dd1036c7a2 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.918234 1182 ts_tablet_manager.cc:1929] T 6399b63cd8974ee49796f8779f9abc0e P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:14.918671 1182 log.cc:1199] T 6399b63cd8974ee49796f8779f9abc0e P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/6399b63cd8974ee49796f8779f9abc0e
I20260202 02:52:14.885804 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 2730d8b6812e487ea0aff4da45d24044 (table default.Table_506775b4d1a14bf1b91c9d951c9a88ec [id=9a8a252de32848288c9b3752c7d3feb7]) successfully deleted
W20260202 02:52:14.918874 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5e1d4f5a21c14fd2bc84e32d68d065bb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:14.919196 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 005cda97ab5c4b70b8fc8ff66a05d996 because the tablet was not found. No further retry: Not found: Tablet not found: 005cda97ab5c4b70b8fc8ff66a05d996
I20260202 02:52:14.919649 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 16713353f2684b9d96c80a3abbcdf74b (table default.Table_2475d75250c24b3ea51238f58fd5f32b [id=2cc4567c7bd44745a5851cd8b95b22c4]) successfully deleted
I20260202 02:52:14.919696 1175 tablet_bootstrap.cc:492] T fde2c1fba78248329f3662dd1036c7a2 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
W20260202 02:52:14.920050 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7ef6ef5b916e42709217b7c44fb489d0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.920212 1182 ts_tablet_manager.cc:1950] T 6399b63cd8974ee49796f8779f9abc0e P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
W20260202 02:52:14.921239 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5c9c0f09396a4a0e8b3d676fcb2dbb2c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.922251 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 6399b63cd8974ee49796f8779f9abc0e (table default.Table_04527bd0309f433aa915767172dcd280 [id=dd2c220f5b88440295facca8da0b1752]) successfully deleted
W20260202 02:52:14.922554 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet fc93352b24e345f99d15085e4886c748 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fc93352b24e345f99d15085e4886c748 already in progress: creating tablet
I20260202 02:52:14.922705 1180 ts_tablet_manager.cc:1929] T 397ed14bb33e44e59536243d0fa7e459 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:14.922863 1180 log.cc:1199] T 397ed14bb33e44e59536243d0fa7e459 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/397ed14bb33e44e59536243d0fa7e459
I20260202 02:52:14.923336 1180 ts_tablet_manager.cc:1950] T 397ed14bb33e44e59536243d0fa7e459 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:14.924528 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 397ed14bb33e44e59536243d0fa7e459 (table default.Table_62840640773d47f5889c6a82f84ffb32 [id=583762f37e29416886d3cc6219f7fd45]) successfully deleted
W20260202 02:52:14.925024 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 72475ea720c6429aa68de9478b374280 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 72475ea720c6429aa68de9478b374280 already in progress: creating tablet
W20260202 02:52:14.925261 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 81f35df38f284f0ca5c3791f3d9d9293 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 81f35df38f284f0ca5c3791f3d9d9293 already in progress: creating tablet
W20260202 02:52:14.925472 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 7e81d5780d9647b9baac0066282376ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7e81d5780d9647b9baac0066282376ea already in progress: creating tablet
W20260202 02:52:14.925657 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet add934bf70ae4de8a40066f06fd94795 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet add934bf70ae4de8a40066f06fd94795 already in progress: creating tablet
W20260202 02:52:14.925854 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet e6b76967e5d3476997fa019c4a8afb91 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e6b76967e5d3476997fa019c4a8afb91 already in progress: creating tablet
W20260202 02:52:14.926210 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 005cda97ab5c4b70b8fc8ff66a05d996 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 005cda97ab5c4b70b8fc8ff66a05d996 already in progress: creating tablet
W20260202 02:52:14.926527 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet c2cc2250701d46a7abe7be03b11d45c8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c2cc2250701d46a7abe7be03b11d45c8 already in progress: creating tablet
I20260202 02:52:14.927017 1180 tablet_replica.cc:333] T 9c52693bdd214919b42cee1bbba95ed2 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:14.927230 1180 raft_consensus.cc:2243] T 9c52693bdd214919b42cee1bbba95ed2 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:14.927554 1180 raft_consensus.cc:2272] T 9c52693bdd214919b42cee1bbba95ed2 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.928411 1175 ts_tablet_manager.cc:1403] T fde2c1fba78248329f3662dd1036c7a2 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260202 02:52:14.928449 1180 ts_tablet_manager.cc:1916] T 9c52693bdd214919b42cee1bbba95ed2 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.929288 1175 raft_consensus.cc:359] T fde2c1fba78248329f3662dd1036c7a2 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.929497 1174 tablet_bootstrap.cc:492] T 005cda97ab5c4b70b8fc8ff66a05d996 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:14.929493 1175 raft_consensus.cc:385] T fde2c1fba78248329f3662dd1036c7a2 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.929567 1175 raft_consensus.cc:740] T fde2c1fba78248329f3662dd1036c7a2 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.929796 1174 ts_tablet_manager.cc:1403] T 005cda97ab5c4b70b8fc8ff66a05d996 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.001s
I20260202 02:52:14.929858 1175 consensus_queue.cc:260] T fde2c1fba78248329f3662dd1036c7a2 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.930858 1174 raft_consensus.cc:359] T 005cda97ab5c4b70b8fc8ff66a05d996 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.931146 1174 raft_consensus.cc:385] T 005cda97ab5c4b70b8fc8ff66a05d996 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.931279 1174 raft_consensus.cc:740] T 005cda97ab5c4b70b8fc8ff66a05d996 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.931617 1174 consensus_queue.cc:260] T 005cda97ab5c4b70b8fc8ff66a05d996 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.932955 1175 ts_tablet_manager.cc:1434] T fde2c1fba78248329f3662dd1036c7a2 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260202 02:52:14.935288 1180 ts_tablet_manager.cc:1929] T 9c52693bdd214919b42cee1bbba95ed2 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:14.935484 1180 log.cc:1199] T 9c52693bdd214919b42cee1bbba95ed2 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/9c52693bdd214919b42cee1bbba95ed2
I20260202 02:52:14.935592 1174 ts_tablet_manager.cc:1434] T 005cda97ab5c4b70b8fc8ff66a05d996 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260202 02:52:14.935976 1180 ts_tablet_manager.cc:1950] T 9c52693bdd214919b42cee1bbba95ed2 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:14.936508 1175 tablet_bootstrap.cc:492] T 7e81d5780d9647b9baac0066282376ea P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:14.937405 1180 tablet_replica.cc:333] T 220c3b32f56c478fbca7fbbd6135290c P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:14.937652 1180 raft_consensus.cc:2243] T 220c3b32f56c478fbca7fbbd6135290c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
W20260202 02:52:14.937822 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet add934bf70ae4de8a40066f06fd94795 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet add934bf70ae4de8a40066f06fd94795 already in progress: creating tablet
I20260202 02:52:14.937892 1180 raft_consensus.cc:2272] T 220c3b32f56c478fbca7fbbd6135290c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:14.938102 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 72475ea720c6429aa68de9478b374280 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 72475ea720c6429aa68de9478b374280 already in progress: creating tablet
W20260202 02:52:14.938328 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet fde2c1fba78248329f3662dd1036c7a2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fde2c1fba78248329f3662dd1036c7a2 already in progress: creating tablet
I20260202 02:52:14.938411 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_34c95b3937dd4587a9bd81a7c805f562"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:14.938652 1175 tablet_bootstrap.cc:654] T 7e81d5780d9647b9baac0066282376ea P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.938774 1180 ts_tablet_manager.cc:1916] T 220c3b32f56c478fbca7fbbd6135290c P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:14.938997 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_34c95b3937dd4587a9bd81a7c805f562 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.940860 1174 tablet_bootstrap.cc:492] T 2730d8b6812e487ea0aff4da45d24044 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:14.941711 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_c245b83c1987454fabbe8646d14252d6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:14.942305 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c245b83c1987454fabbe8646d14252d6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.945362 1180 ts_tablet_manager.cc:1929] T 220c3b32f56c478fbca7fbbd6135290c P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:14.945585 1180 log.cc:1199] T 220c3b32f56c478fbca7fbbd6135290c P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/220c3b32f56c478fbca7fbbd6135290c
I20260202 02:52:14.946009 1180 ts_tablet_manager.cc:1950] T 220c3b32f56c478fbca7fbbd6135290c P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:14.947141 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 220c3b32f56c478fbca7fbbd6135290c (table default.Table_c5a4aa6f50fb472c8982feedb30982a9 [id=7af0d6d84db7404ebed9db660a633c12]) successfully deleted
W20260202 02:52:14.947945 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet fc93352b24e345f99d15085e4886c748 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fc93352b24e345f99d15085e4886c748 already in progress: creating tablet
W20260202 02:52:14.948689 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 6399b63cd8974ee49796f8779f9abc0e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6399b63cd8974ee49796f8779f9abc0e already in progress: creating tablet
I20260202 02:52:14.949117 1180 tablet_replica.cc:333] T 16713353f2684b9d96c80a3abbcdf74b P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:14.949005 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_e764fa5da91046ba8293586023f427d5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:14.949378 1180 raft_consensus.cc:2243] T 16713353f2684b9d96c80a3abbcdf74b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:14.949584 1180 raft_consensus.cc:2272] T 16713353f2684b9d96c80a3abbcdf74b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:14.949657 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e764fa5da91046ba8293586023f427d5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.950484 1180 ts_tablet_manager.cc:1916] T 16713353f2684b9d96c80a3abbcdf74b P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.950695 1175 tablet_bootstrap.cc:492] T 7e81d5780d9647b9baac0066282376ea P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:14.950727 1174 tablet_bootstrap.cc:654] T 2730d8b6812e487ea0aff4da45d24044 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.950835 1175 ts_tablet_manager.cc:1403] T 7e81d5780d9647b9baac0066282376ea P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.000s
I20260202 02:52:14.951332 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_f0eb2828eb0a4d968bec535ac68a7e3b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:14.951678 1175 raft_consensus.cc:359] T 7e81d5780d9647b9baac0066282376ea P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.951855 1175 raft_consensus.cc:385] T 7e81d5780d9647b9baac0066282376ea P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.951933 1175 raft_consensus.cc:740] T 7e81d5780d9647b9baac0066282376ea P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
W20260202 02:52:14.951972 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f0eb2828eb0a4d968bec535ac68a7e3b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:14.952180 1175 consensus_queue.cc:260] T 7e81d5780d9647b9baac0066282376ea P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.954890 1175 ts_tablet_manager.cc:1434] T 7e81d5780d9647b9baac0066282376ea P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260202 02:52:14.956422 1175 tablet_bootstrap.cc:492] T c2cc2250701d46a7abe7be03b11d45c8 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:14.957145 1180 ts_tablet_manager.cc:1929] T 16713353f2684b9d96c80a3abbcdf74b P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:14.957283 1180 log.cc:1199] T 16713353f2684b9d96c80a3abbcdf74b P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/16713353f2684b9d96c80a3abbcdf74b
I20260202 02:52:14.957858 1180 ts_tablet_manager.cc:1950] T 16713353f2684b9d96c80a3abbcdf74b P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:14.958642 1173 tablet_bootstrap.cc:492] T 005cda97ab5c4b70b8fc8ff66a05d996 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:14.958832 1173 ts_tablet_manager.cc:1403] T 005cda97ab5c4b70b8fc8ff66a05d996 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.105s user 0.003s sys 0.000s
I20260202 02:52:14.959131 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 16713353f2684b9d96c80a3abbcdf74b (table default.Table_2475d75250c24b3ea51238f58fd5f32b [id=2cc4567c7bd44745a5851cd8b95b22c4]) successfully deleted
I20260202 02:52:14.959635 1173 raft_consensus.cc:359] T 005cda97ab5c4b70b8fc8ff66a05d996 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.959790 1175 tablet_bootstrap.cc:654] T c2cc2250701d46a7abe7be03b11d45c8 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.959867 1173 raft_consensus.cc:385] T 005cda97ab5c4b70b8fc8ff66a05d996 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.959944 1173 raft_consensus.cc:740] T 005cda97ab5c4b70b8fc8ff66a05d996 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.960196 1173 consensus_queue.cc:260] T 005cda97ab5c4b70b8fc8ff66a05d996 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.961932 1173 ts_tablet_manager.cc:1434] T 005cda97ab5c4b70b8fc8ff66a05d996 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260202 02:52:14.962476 1173 tablet_bootstrap.cc:492] T 2730d8b6812e487ea0aff4da45d24044 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:14.964257 1173 tablet_bootstrap.cc:654] T 2730d8b6812e487ea0aff4da45d24044 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.964790 1174 tablet_bootstrap.cc:492] T 2730d8b6812e487ea0aff4da45d24044 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:14.964969 1174 ts_tablet_manager.cc:1403] T 2730d8b6812e487ea0aff4da45d24044 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.006s
I20260202 02:52:14.965772 1174 raft_consensus.cc:359] T 2730d8b6812e487ea0aff4da45d24044 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.965986 1174 raft_consensus.cc:385] T 2730d8b6812e487ea0aff4da45d24044 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.966091 1174 raft_consensus.cc:740] T 2730d8b6812e487ea0aff4da45d24044 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.966320 1174 consensus_queue.cc:260] T 2730d8b6812e487ea0aff4da45d24044 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.966853 1174 ts_tablet_manager.cc:1434] T 2730d8b6812e487ea0aff4da45d24044 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260202 02:52:14.967177 1174 tablet_bootstrap.cc:492] T 6399b63cd8974ee49796f8779f9abc0e P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:14.969317 1174 tablet_bootstrap.cc:654] T 6399b63cd8974ee49796f8779f9abc0e P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:14.970813 522 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f24c0b99574d437a99b273204f105460 consensus state has no leader
I20260202 02:52:14.971666 1175 tablet_bootstrap.cc:492] T c2cc2250701d46a7abe7be03b11d45c8 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:14.971884 1175 ts_tablet_manager.cc:1403] T c2cc2250701d46a7abe7be03b11d45c8 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.000s
I20260202 02:52:14.972209 898 tablet_service.cc:1552] Processing DeleteTablet for tablet 81f35df38f284f0ca5c3791f3d9d9293 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:14.972512 1182 tablet_replica.cc:333] T 81f35df38f284f0ca5c3791f3d9d9293 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:14.972694 1182 raft_consensus.cc:2243] T 81f35df38f284f0ca5c3791f3d9d9293 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:14.972746 1175 raft_consensus.cc:359] T c2cc2250701d46a7abe7be03b11d45c8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.972888 1182 raft_consensus.cc:2272] T 81f35df38f284f0ca5c3791f3d9d9293 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.972967 1175 raft_consensus.cc:385] T c2cc2250701d46a7abe7be03b11d45c8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.973054 1175 raft_consensus.cc:740] T c2cc2250701d46a7abe7be03b11d45c8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.973332 1175 consensus_queue.cc:260] T c2cc2250701d46a7abe7be03b11d45c8 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.973815 1182 ts_tablet_manager.cc:1916] T 81f35df38f284f0ca5c3791f3d9d9293 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.975735 1175 ts_tablet_manager.cc:1434] T c2cc2250701d46a7abe7be03b11d45c8 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260202 02:52:14.976349 1175 tablet_bootstrap.cc:492] T fae7e3941c124cc2871390a0b6471f27 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:14.983637 1182 ts_tablet_manager.cc:1929] T 81f35df38f284f0ca5c3791f3d9d9293 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:14.983896 1182 log.cc:1199] T 81f35df38f284f0ca5c3791f3d9d9293 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/81f35df38f284f0ca5c3791f3d9d9293
I20260202 02:52:14.984452 1182 ts_tablet_manager.cc:1950] T 81f35df38f284f0ca5c3791f3d9d9293 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:14.984721 756 tablet_service.cc:1552] Processing DeleteTablet for tablet c9ce1b27212646e691bad953c9fbe14d with delete_type TABLET_DATA_DELETED (replaced by tablet 2730d8b6812e487ea0aff4da45d24044) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:14.985064 1180 tablet_replica.cc:333] T c9ce1b27212646e691bad953c9fbe14d P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:14.985080 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 005cda97ab5c4b70b8fc8ff66a05d996 with delete_type TABLET_DATA_DELETED (replaced by tablet 4f2a9404f7424dc78b5bd062ebd8cb53) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:14.985268 1175 tablet_bootstrap.cc:654] T fae7e3941c124cc2871390a0b6471f27 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:14.985284 1180 raft_consensus.cc:2243] T c9ce1b27212646e691bad953c9fbe14d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:14.985453 1180 raft_consensus.cc:2272] T c9ce1b27212646e691bad953c9fbe14d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.986371 1180 ts_tablet_manager.cc:1916] T c9ce1b27212646e691bad953c9fbe14d P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.992779 1180 ts_tablet_manager.cc:1929] T c9ce1b27212646e691bad953c9fbe14d P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:14.993049 1180 log.cc:1199] T c9ce1b27212646e691bad953c9fbe14d P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/c9ce1b27212646e691bad953c9fbe14d
I20260202 02:52:14.993505 1180 ts_tablet_manager.cc:1950] T c9ce1b27212646e691bad953c9fbe14d P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:14.995317 1180 tablet_replica.cc:333] T 005cda97ab5c4b70b8fc8ff66a05d996 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:14.995599 1180 raft_consensus.cc:2243] T 005cda97ab5c4b70b8fc8ff66a05d996 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:14.995848 1180 raft_consensus.cc:2272] T 005cda97ab5c4b70b8fc8ff66a05d996 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:14.996712 1180 ts_tablet_manager.cc:1916] T 005cda97ab5c4b70b8fc8ff66a05d996 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:14.997212 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 81f35df38f284f0ca5c3791f3d9d9293 (table default.Table_15c57b6d38ab4f2fb7dca2a405f01058 [id=d54948158c124d00ab0e2b1f43d45cc3]) successfully deleted
I20260202 02:52:14.997334 1175 tablet_bootstrap.cc:492] T fae7e3941c124cc2871390a0b6471f27 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:14.997483 1175 ts_tablet_manager.cc:1403] T fae7e3941c124cc2871390a0b6471f27 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.021s user 0.006s sys 0.000s
I20260202 02:52:14.977672 1040 tablet_service.cc:1552] Processing DeleteTablet for tablet 005cda97ab5c4b70b8fc8ff66a05d996 with delete_type TABLET_DATA_DELETED (replaced by tablet 4f2a9404f7424dc78b5bd062ebd8cb53) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:14.998394 1175 raft_consensus.cc:359] T fae7e3941c124cc2871390a0b6471f27 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.998607 1175 raft_consensus.cc:385] T fae7e3941c124cc2871390a0b6471f27 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:14.998695 1175 raft_consensus.cc:740] T fae7e3941c124cc2871390a0b6471f27 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:14.998991 1175 consensus_queue.cc:260] T fae7e3941c124cc2871390a0b6471f27 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:14.999679 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet c9ce1b27212646e691bad953c9fbe14d (table default.Table_7ff64c2b44d0417ea6445647a80a5b86 [id=9a8a252de32848288c9b3752c7d3feb7]) successfully deleted
I20260202 02:52:15.000823 1174 tablet_bootstrap.cc:492] T 6399b63cd8974ee49796f8779f9abc0e P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:15.000984 1174 ts_tablet_manager.cc:1403] T 6399b63cd8974ee49796f8779f9abc0e P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.034s user 0.008s sys 0.009s
I20260202 02:52:15.000972 1175 ts_tablet_manager.cc:1434] T fae7e3941c124cc2871390a0b6471f27 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260202 02:52:15.001778 1174 raft_consensus.cc:359] T 6399b63cd8974ee49796f8779f9abc0e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.002064 1174 raft_consensus.cc:385] T 6399b63cd8974ee49796f8779f9abc0e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.002190 1174 raft_consensus.cc:740] T 6399b63cd8974ee49796f8779f9abc0e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.002592 1174 consensus_queue.cc:260] T 6399b63cd8974ee49796f8779f9abc0e P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.002924 1175 tablet_bootstrap.cc:492] T 58f09c607cce4ccc82afd75c002ed1a8 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:15.004127 1174 ts_tablet_manager.cc:1434] T 6399b63cd8974ee49796f8779f9abc0e P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260202 02:52:15.004566 1174 tablet_bootstrap.cc:492] T 81f35df38f284f0ca5c3791f3d9d9293 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:15.006202 898 tablet_service.cc:1552] Processing DeleteTablet for tablet 81f35df38f284f0ca5c3791f3d9d9293 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:15.006487 899 tablet_service.cc:1552] Processing DeleteTablet for tablet fde2c1fba78248329f3662dd1036c7a2 with delete_type TABLET_DATA_DELETED (replaced by tablet 23c38ef4e3c840808ec36f859cf6f6de) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:15.006695 903 tablet_service.cc:1552] Processing DeleteTablet for tablet c2cc2250701d46a7abe7be03b11d45c8 with delete_type TABLET_DATA_DELETED (replaced by tablet 0d297b6243c348ee82c146a8339cf6c4) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:15.006940 901 tablet_service.cc:1552] Processing DeleteTablet for tablet 7e81d5780d9647b9baac0066282376ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:15.006705 1175 tablet_bootstrap.cc:654] T 58f09c607cce4ccc82afd75c002ed1a8 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.007450 1182 tablet_replica.cc:333] T fde2c1fba78248329f3662dd1036c7a2 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:15.007648 1182 raft_consensus.cc:2243] T fde2c1fba78248329f3662dd1036c7a2 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:15.007817 1182 raft_consensus.cc:2272] T fde2c1fba78248329f3662dd1036c7a2 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:15.008613 1182 ts_tablet_manager.cc:1916] T fde2c1fba78248329f3662dd1036c7a2 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:15.009549 528 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_d1ead30f822b4a99ab5777a06313d94c" } new_table_name: "default.Table_8cb3cbab1a654f88a07d6d480fc72e52" modify_external_catalogs: true
W20260202 02:52:15.010524 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 81f35df38f284f0ca5c3791f3d9d9293 because the tablet was not found. No further retry: Not found: Tablet not found: 81f35df38f284f0ca5c3791f3d9d9293
I20260202 02:52:15.011276 521 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_2475d75250c24b3ea51238f58fd5f32b" } new_table_name: "default.Table_b6c1fb1e7282492d920385869810b0e0" modify_external_catalogs: true
I20260202 02:52:15.012297 1174 tablet_bootstrap.cc:654] T 81f35df38f284f0ca5c3791f3d9d9293 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.012540 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_9bac792e43464105846e5482edbccacc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:15.013089 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9bac792e43464105846e5482edbccacc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:15.015061 1182 ts_tablet_manager.cc:1929] T fde2c1fba78248329f3662dd1036c7a2 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:15.015221 1182 log.cc:1199] T fde2c1fba78248329f3662dd1036c7a2 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/fde2c1fba78248329f3662dd1036c7a2
I20260202 02:52:15.015606 1182 ts_tablet_manager.cc:1950] T fde2c1fba78248329f3662dd1036c7a2 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:15.016819 1182 tablet_replica.cc:333] T c2cc2250701d46a7abe7be03b11d45c8 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:15.017055 1182 raft_consensus.cc:2243] T c2cc2250701d46a7abe7be03b11d45c8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:15.017268 1182 raft_consensus.cc:2272] T c2cc2250701d46a7abe7be03b11d45c8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:15.017217 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet fde2c1fba78248329f3662dd1036c7a2 (table default.Table_9fd715491a6a4bbc8685138d0194362d [id=98ddcbfb5ad14ae480a54845ec7c45a4]) successfully deleted
I20260202 02:52:15.017681 1180 ts_tablet_manager.cc:1929] T 005cda97ab5c4b70b8fc8ff66a05d996 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:15.017818 1180 log.cc:1199] T 005cda97ab5c4b70b8fc8ff66a05d996 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/005cda97ab5c4b70b8fc8ff66a05d996
I20260202 02:52:15.018177 1180 ts_tablet_manager.cc:1950] T 005cda97ab5c4b70b8fc8ff66a05d996 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:15.019168 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 005cda97ab5c4b70b8fc8ff66a05d996 (table default.Table_0842fd6a82e145e7a98b3fed2c0c6993 [id=4e076b2023be4c888ac1c27babde1607]) successfully deleted
I20260202 02:52:15.019428 1182 ts_tablet_manager.cc:1916] T c2cc2250701d46a7abe7be03b11d45c8 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:15.020102 1183 ts_tablet_manager.cc:1929] T 16713353f2684b9d96c80a3abbcdf74b P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:15.020267 1183 log.cc:1199] T 16713353f2684b9d96c80a3abbcdf74b P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/16713353f2684b9d96c80a3abbcdf74b
I20260202 02:52:15.020653 1183 ts_tablet_manager.cc:1950] T 16713353f2684b9d96c80a3abbcdf74b P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:15.022920 519 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_4a3ef984ca894b8b8f4301fecb719b5b" } new_table_name: "default.Table_ab8d74f30cd14c61b85950a82d0f3787" modify_external_catalogs: true
I20260202 02:52:15.026252 1175 tablet_bootstrap.cc:492] T 58f09c607cce4ccc82afd75c002ed1a8 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:15.026710 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 2730d8b6812e487ea0aff4da45d24044 with delete_type TABLET_DATA_DELETED (replaced by tablet f24c0b99574d437a99b273204f105460) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:15.027033 1180 tablet_replica.cc:333] T 2730d8b6812e487ea0aff4da45d24044 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:15.027069 1175 ts_tablet_manager.cc:1403] T 58f09c607cce4ccc82afd75c002ed1a8 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.024s user 0.006s sys 0.001s
I20260202 02:52:15.027346 1180 raft_consensus.cc:2243] T 2730d8b6812e487ea0aff4da45d24044 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:15.027573 1180 raft_consensus.cc:2272] T 2730d8b6812e487ea0aff4da45d24044 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:15.027923 1175 raft_consensus.cc:359] T 58f09c607cce4ccc82afd75c002ed1a8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.028154 1175 raft_consensus.cc:385] T 58f09c607cce4ccc82afd75c002ed1a8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.028239 1175 raft_consensus.cc:740] T 58f09c607cce4ccc82afd75c002ed1a8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.028661 1180 ts_tablet_manager.cc:1916] T 2730d8b6812e487ea0aff4da45d24044 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:15.030237 520 master_service.cc:954] ReplaceTablet: received request to replace tablet 575016c0e9f84e99b962c5f14ffa96cc from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:15.031085 1175 consensus_queue.cc:260] T 58f09c607cce4ccc82afd75c002ed1a8 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.032092 1174 tablet_bootstrap.cc:492] T 81f35df38f284f0ca5c3791f3d9d9293 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:15.034559 1174 ts_tablet_manager.cc:1403] T 81f35df38f284f0ca5c3791f3d9d9293 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.030s user 0.007s sys 0.002s
I20260202 02:52:15.036568 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 16713353f2684b9d96c80a3abbcdf74b (table default.Table_2475d75250c24b3ea51238f58fd5f32b [id=2cc4567c7bd44745a5851cd8b95b22c4]) successfully deleted
I20260202 02:52:15.037189 1174 raft_consensus.cc:359] T 81f35df38f284f0ca5c3791f3d9d9293 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.037483 1174 raft_consensus.cc:385] T 81f35df38f284f0ca5c3791f3d9d9293 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.037614 1174 raft_consensus.cc:740] T 81f35df38f284f0ca5c3791f3d9d9293 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.037961 1174 consensus_queue.cc:260] T 81f35df38f284f0ca5c3791f3d9d9293 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.039021 1183 tablet_replica.cc:333] T 005cda97ab5c4b70b8fc8ff66a05d996 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:15.039096 1184 raft_consensus.cc:493] T fae7e3941c124cc2871390a0b6471f27 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:15.039225 1183 raft_consensus.cc:2243] T 005cda97ab5c4b70b8fc8ff66a05d996 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:15.039271 1184 raft_consensus.cc:515] T fae7e3941c124cc2871390a0b6471f27 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.039418 1183 raft_consensus.cc:2272] T 005cda97ab5c4b70b8fc8ff66a05d996 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:15.039860 1184 leader_election.cc:290] T fae7e3941c124cc2871390a0b6471f27 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:15.040877 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fae7e3941c124cc2871390a0b6471f27" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:15.041054 1175 ts_tablet_manager.cc:1434] T 58f09c607cce4ccc82afd75c002ed1a8 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.014s user 0.002s sys 0.000s
I20260202 02:52:15.041713 526 master_service.cc:954] ReplaceTablet: received request to replace tablet 4de41396cffd4d32b86c701b538d1e01 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:15.043159 1182 ts_tablet_manager.cc:1929] T c2cc2250701d46a7abe7be03b11d45c8 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:15.043285 1182 log.cc:1199] T c2cc2250701d46a7abe7be03b11d45c8 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/c2cc2250701d46a7abe7be03b11d45c8
I20260202 02:52:15.043299 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fae7e3941c124cc2871390a0b6471f27" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:15.043632 1182 ts_tablet_manager.cc:1950] T c2cc2250701d46a7abe7be03b11d45c8 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
W20260202 02:52:15.044000 859 leader_election.cc:343] T fae7e3941c124cc2871390a0b6471f27 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:15.044508 857 leader_election.cc:343] T fae7e3941c124cc2871390a0b6471f27 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:15.044601 1182 tablet_replica.cc:333] T 7e81d5780d9647b9baac0066282376ea P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:15.044631 857 leader_election.cc:304] T fae7e3941c124cc2871390a0b6471f27 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:15.044782 1182 raft_consensus.cc:2243] T 7e81d5780d9647b9baac0066282376ea P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:15.044927 1182 raft_consensus.cc:2272] T 7e81d5780d9647b9baac0066282376ea P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:15.045192 1175 tablet_bootstrap.cc:492] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:15.045221 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet c2cc2250701d46a7abe7be03b11d45c8 (table default.Table_d210d1530706429197d6294b29f18c87 [id=7c76848b98614fbf8b2950e5d55cc609]) successfully deleted
I20260202 02:52:15.045667 1173 tablet_bootstrap.cc:492] T 2730d8b6812e487ea0aff4da45d24044 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:15.045662 1184 raft_consensus.cc:2749] T fae7e3941c124cc2871390a0b6471f27 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:15.045964 1173 ts_tablet_manager.cc:1403] T 2730d8b6812e487ea0aff4da45d24044 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.084s user 0.002s sys 0.002s
I20260202 02:52:15.046391 1174 ts_tablet_manager.cc:1434] T 81f35df38f284f0ca5c3791f3d9d9293 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.012s user 0.001s sys 0.001s
I20260202 02:52:15.046898 1175 tablet_bootstrap.cc:654] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.046926 1173 raft_consensus.cc:359] T 2730d8b6812e487ea0aff4da45d24044 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.047128 1173 raft_consensus.cc:385] T 2730d8b6812e487ea0aff4da45d24044 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.047216 1173 raft_consensus.cc:740] T 2730d8b6812e487ea0aff4da45d24044 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.047465 1173 consensus_queue.cc:260] T 2730d8b6812e487ea0aff4da45d24044 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.050107 1174 tablet_bootstrap.cc:492] T 7e81d5780d9647b9baac0066282376ea P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:15.051812 1174 tablet_bootstrap.cc:654] T 7e81d5780d9647b9baac0066282376ea P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.052136 1173 ts_tablet_manager.cc:1434] T 2730d8b6812e487ea0aff4da45d24044 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260202 02:52:15.053766 1180 ts_tablet_manager.cc:1929] T 2730d8b6812e487ea0aff4da45d24044 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:15.054920 1180 log.cc:1199] T 2730d8b6812e487ea0aff4da45d24044 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/2730d8b6812e487ea0aff4da45d24044
I20260202 02:52:15.054984 1182 ts_tablet_manager.cc:1916] T 7e81d5780d9647b9baac0066282376ea P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:15.055465 1173 tablet_bootstrap.cc:492] T 6399b63cd8974ee49796f8779f9abc0e P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:15.055474 1180 ts_tablet_manager.cc:1950] T 2730d8b6812e487ea0aff4da45d24044 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:15.056502 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 2730d8b6812e487ea0aff4da45d24044 (table default.Table_7ff64c2b44d0417ea6445647a80a5b86 [id=9a8a252de32848288c9b3752c7d3feb7]) successfully deleted
W20260202 02:52:15.056732 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 6399b63cd8974ee49796f8779f9abc0e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6399b63cd8974ee49796f8779f9abc0e already in progress: creating tablet
I20260202 02:52:15.061633 1175 tablet_bootstrap.cc:492] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:15.061817 1175 ts_tablet_manager.cc:1403] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.003s
I20260202 02:52:15.062698 1175 raft_consensus.cc:359] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.062920 1175 raft_consensus.cc:385] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.063004 1175 raft_consensus.cc:740] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.063313 1175 consensus_queue.cc:260] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.063925 1183 ts_tablet_manager.cc:1916] T 005cda97ab5c4b70b8fc8ff66a05d996 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:15.064069 1175 ts_tablet_manager.cc:1434] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:15.066160 1175 tablet_bootstrap.cc:492] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
W20260202 02:52:15.067683 528 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5986676f5484498da927b4c6da3f47ff has no consensus state
I20260202 02:52:15.069677 1175 tablet_bootstrap.cc:654] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.071074 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 6399b63cd8974ee49796f8779f9abc0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:15.071398 1180 tablet_replica.cc:333] T 6399b63cd8974ee49796f8779f9abc0e P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:15.071585 1180 raft_consensus.cc:2243] T 6399b63cd8974ee49796f8779f9abc0e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:15.071764 1180 raft_consensus.cc:2272] T 6399b63cd8974ee49796f8779f9abc0e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:15.072834 1180 ts_tablet_manager.cc:1916] T 6399b63cd8974ee49796f8779f9abc0e P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:15.073649 1182 ts_tablet_manager.cc:1929] T 7e81d5780d9647b9baac0066282376ea P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:15.074079 1182 log.cc:1199] T 7e81d5780d9647b9baac0066282376ea P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/7e81d5780d9647b9baac0066282376ea
I20260202 02:52:15.074460 1182 ts_tablet_manager.cc:1950] T 7e81d5780d9647b9baac0066282376ea P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:15.075832 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 7e81d5780d9647b9baac0066282376ea (table default.Table_629c3da327a740f684072799e959c2ca [id=30cd87ed1178421bb31ee622ce3254c1]) successfully deleted
I20260202 02:52:15.075959 523 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet fae7e3941c124cc2871390a0b6471f27 (table default.Table_3d26f7a34d0c48379e3cebda66e1ef13 [id=45e6d83589fa493195b06f633e295b00]). Expected version 2 got 0
I20260202 02:52:15.076622 1175 tablet_bootstrap.cc:492] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:15.076990 1175 ts_tablet_manager.cc:1403] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20260202 02:52:15.077801 1175 raft_consensus.cc:359] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.078008 1175 raft_consensus.cc:385] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.078091 1175 raft_consensus.cc:740] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.078081 751 tablet_service.cc:1505] Processing CreateTablet for tablet 73f8db8f30f14d369d2227c9be7c4647 (DEFAULT_TABLE table=default.Table_0276ce9d8aaa44fa832bb88a8b1f9446 [id=a8eac3717bad4d1496acd1c857a510a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.078331 1175 consensus_queue.cc:260] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.078550 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73f8db8f30f14d369d2227c9be7c4647. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.079099 1175 ts_tablet_manager.cc:1434] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:15.081118 1175 tablet_bootstrap.cc:492] T 72475ea720c6429aa68de9478b374280 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:15.082753 1180 ts_tablet_manager.cc:1929] T 6399b63cd8974ee49796f8779f9abc0e P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:15.082989 1180 log.cc:1199] T 6399b63cd8974ee49796f8779f9abc0e P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/6399b63cd8974ee49796f8779f9abc0e
I20260202 02:52:15.085081 1180 ts_tablet_manager.cc:1950] T 6399b63cd8974ee49796f8779f9abc0e P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:15.085906 1175 tablet_bootstrap.cc:654] T 72475ea720c6429aa68de9478b374280 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.086484 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 6399b63cd8974ee49796f8779f9abc0e (table default.Table_04527bd0309f433aa915767172dcd280 [id=dd2c220f5b88440295facca8da0b1752]) successfully deleted
I20260202 02:52:15.087126 1174 tablet_bootstrap.cc:492] T 7e81d5780d9647b9baac0066282376ea P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:15.087383 1174 ts_tablet_manager.cc:1403] T 7e81d5780d9647b9baac0066282376ea P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.037s user 0.010s sys 0.005s
I20260202 02:52:15.089107 901 tablet_service.cc:1505] Processing CreateTablet for tablet 73f8db8f30f14d369d2227c9be7c4647 (DEFAULT_TABLE table=default.Table_0276ce9d8aaa44fa832bb88a8b1f9446 [id=a8eac3717bad4d1496acd1c857a510a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.089563 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73f8db8f30f14d369d2227c9be7c4647. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.089581 1174 raft_consensus.cc:359] T 7e81d5780d9647b9baac0066282376ea P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.090246 1174 raft_consensus.cc:385] T 7e81d5780d9647b9baac0066282376ea P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.090324 1174 raft_consensus.cc:740] T 7e81d5780d9647b9baac0066282376ea P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.090552 1174 consensus_queue.cc:260] T 7e81d5780d9647b9baac0066282376ea P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.092279 899 tablet_service.cc:1505] Processing CreateTablet for tablet 97bd7223d41d40a6bcdc60b1f63abc67 (DEFAULT_TABLE table=default.Table_53eaadb859904d6b95ef0e536019ad79 [id=0ca5aac09bb54feba75b2d3784a69140]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.092764 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97bd7223d41d40a6bcdc60b1f63abc67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.094566 1174 ts_tablet_manager.cc:1434] T 7e81d5780d9647b9baac0066282376ea P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260202 02:52:15.094681 754 tablet_service.cc:1505] Processing CreateTablet for tablet c0937b6910534ddc8a2217f0cbd539c5 (DEFAULT_TABLE table=default.Table_9c6ddf42d084449998f1f7925b0b0c3b [id=46b5d47377474ff9bac2acdb26b08c65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.095137 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0937b6910534ddc8a2217f0cbd539c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.096784 903 tablet_service.cc:1505] Processing CreateTablet for tablet c050a2882f764dda8df2a7df6547f149 (DEFAULT_TABLE table=default.Table_115696f06e984b47bd26eab721b76bf7 [id=5f8923fc0fb84598a1fb7c3a8abbb8cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.097195 903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c050a2882f764dda8df2a7df6547f149. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.098565 1174 tablet_bootstrap.cc:492] T fde2c1fba78248329f3662dd1036c7a2 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:15.100634 1174 tablet_bootstrap.cc:654] T fde2c1fba78248329f3662dd1036c7a2 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.101058 756 tablet_service.cc:1505] Processing CreateTablet for tablet 97bd7223d41d40a6bcdc60b1f63abc67 (DEFAULT_TABLE table=default.Table_53eaadb859904d6b95ef0e536019ad79 [id=0ca5aac09bb54feba75b2d3784a69140]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.101473 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97bd7223d41d40a6bcdc60b1f63abc67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.102435 751 tablet_service.cc:1505] Processing CreateTablet for tablet c050a2882f764dda8df2a7df6547f149 (DEFAULT_TABLE table=default.Table_115696f06e984b47bd26eab721b76bf7 [id=5f8923fc0fb84598a1fb7c3a8abbb8cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.102870 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c050a2882f764dda8df2a7df6547f149. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.104725 749 tablet_service.cc:1505] Processing CreateTablet for tablet 61a9203035f2495f8f99924303ebf120 (DEFAULT_TABLE table=default.Table_6a67d7cee0254eba9a53f4d52a30ee39 [id=36a229921ecc4185adbfdfba372a38ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.105118 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61a9203035f2495f8f99924303ebf120. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.108305 898 tablet_service.cc:1505] Processing CreateTablet for tablet 61a9203035f2495f8f99924303ebf120 (DEFAULT_TABLE table=default.Table_6a67d7cee0254eba9a53f4d52a30ee39 [id=36a229921ecc4185adbfdfba372a38ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.112907 1040 tablet_service.cc:1552] Processing DeleteTablet for tablet 2730d8b6812e487ea0aff4da45d24044 with delete_type TABLET_DATA_DELETED (replaced by tablet f24c0b99574d437a99b273204f105460) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:15.113286 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61a9203035f2495f8f99924303ebf120. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.116489 1040 tablet_service.cc:1505] Processing CreateTablet for tablet 73f8db8f30f14d369d2227c9be7c4647 (DEFAULT_TABLE table=default.Table_0276ce9d8aaa44fa832bb88a8b1f9446 [id=a8eac3717bad4d1496acd1c857a510a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.116932 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73f8db8f30f14d369d2227c9be7c4647. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.124172 900 tablet_service.cc:1505] Processing CreateTablet for tablet c0937b6910534ddc8a2217f0cbd539c5 (DEFAULT_TABLE table=default.Table_9c6ddf42d084449998f1f7925b0b0c3b [id=46b5d47377474ff9bac2acdb26b08c65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.124693 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0937b6910534ddc8a2217f0cbd539c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.135035 1040 tablet_service.cc:1505] Processing CreateTablet for tablet c050a2882f764dda8df2a7df6547f149 (DEFAULT_TABLE table=default.Table_115696f06e984b47bd26eab721b76bf7 [id=5f8923fc0fb84598a1fb7c3a8abbb8cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.135833 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c050a2882f764dda8df2a7df6547f149. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.140051 1174 tablet_bootstrap.cc:492] T fde2c1fba78248329f3662dd1036c7a2 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:15.140484 1174 ts_tablet_manager.cc:1403] T fde2c1fba78248329f3662dd1036c7a2 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.042s user 0.026s sys 0.012s
I20260202 02:52:15.141577 1174 raft_consensus.cc:359] T fde2c1fba78248329f3662dd1036c7a2 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.141925 1174 raft_consensus.cc:385] T fde2c1fba78248329f3662dd1036c7a2 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.142083 1174 raft_consensus.cc:740] T fde2c1fba78248329f3662dd1036c7a2 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.142473 1174 consensus_queue.cc:260] T fde2c1fba78248329f3662dd1036c7a2 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.142738 1041 tablet_service.cc:1505] Processing CreateTablet for tablet 97bd7223d41d40a6bcdc60b1f63abc67 (DEFAULT_TABLE table=default.Table_53eaadb859904d6b95ef0e536019ad79 [id=0ca5aac09bb54feba75b2d3784a69140]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.143180 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97bd7223d41d40a6bcdc60b1f63abc67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.147009 1174 ts_tablet_manager.cc:1434] T fde2c1fba78248329f3662dd1036c7a2 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260202 02:52:15.150578 523 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_a2838c9f404646eea3ee18d69eb32a12" } new_table_name: "default.Table_10aa0dafec0547069eb92ecf12b3c780" modify_external_catalogs: true
I20260202 02:52:15.150704 1044 tablet_service.cc:1505] Processing CreateTablet for tablet c0937b6910534ddc8a2217f0cbd539c5 (DEFAULT_TABLE table=default.Table_9c6ddf42d084449998f1f7925b0b0c3b [id=46b5d47377474ff9bac2acdb26b08c65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.151257 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0937b6910534ddc8a2217f0cbd539c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.152387 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_bb9eeb45b01e4ad3b5a6bc6ef8ab60e9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:15.152974 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bb9eeb45b01e4ad3b5a6bc6ef8ab60e9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:15.154451 1184 raft_consensus.cc:493] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:15.154680 1184 raft_consensus.cc:515] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.154785 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_052de59e60de4473a2d13185440ab818"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:15.155330 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_052de59e60de4473a2d13185440ab818 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:15.155443 1184 leader_election.cc:290] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:15.156751 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2ff6af17a6854451a1d7b904aa4f1f55" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:15.157603 859 leader_election.cc:343] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:15.158449 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2ff6af17a6854451a1d7b904aa4f1f55" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:15.159114 857 leader_election.cc:343] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:15.159256 857 leader_election.cc:304] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:15.159765 1184 raft_consensus.cc:2749] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:15.160476 525 master_service.cc:954] ReplaceTablet: received request to replace tablet 58f09c607cce4ccc82afd75c002ed1a8 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:15.160580 1174 tablet_bootstrap.cc:492] T c2cc2250701d46a7abe7be03b11d45c8 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:15.162222 1041 tablet_service.cc:1505] Processing CreateTablet for tablet 61a9203035f2495f8f99924303ebf120 (DEFAULT_TABLE table=default.Table_6a67d7cee0254eba9a53f4d52a30ee39 [id=36a229921ecc4185adbfdfba372a38ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.162557 1174 tablet_bootstrap.cc:654] T c2cc2250701d46a7abe7be03b11d45c8 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.162739 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61a9203035f2495f8f99924303ebf120. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.168565 1173 tablet_bootstrap.cc:654] T 6399b63cd8974ee49796f8779f9abc0e P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.171481 1174 tablet_bootstrap.cc:492] T c2cc2250701d46a7abe7be03b11d45c8 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:15.171828 1174 ts_tablet_manager.cc:1403] T c2cc2250701d46a7abe7be03b11d45c8 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.008s
I20260202 02:52:15.172821 1174 raft_consensus.cc:359] T c2cc2250701d46a7abe7be03b11d45c8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.173166 1174 raft_consensus.cc:385] T c2cc2250701d46a7abe7be03b11d45c8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.173285 1174 raft_consensus.cc:740] T c2cc2250701d46a7abe7be03b11d45c8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.173662 1174 consensus_queue.cc:260] T c2cc2250701d46a7abe7be03b11d45c8 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.178673 1174 ts_tablet_manager.cc:1434] T c2cc2250701d46a7abe7be03b11d45c8 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260202 02:52:15.180538 751 tablet_service.cc:1505] Processing CreateTablet for tablet c1fc01252d95407e954cfcbf7e715912 (DEFAULT_TABLE table=default.Table_f23c311557a14f4a8c66eaf4af7a8a92 [id=10880639cfc743fcafecc538fa2ecd85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.181113 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1fc01252d95407e954cfcbf7e715912. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.181391 898 tablet_service.cc:1505] Processing CreateTablet for tablet c1fc01252d95407e954cfcbf7e715912 (DEFAULT_TABLE table=default.Table_f23c311557a14f4a8c66eaf4af7a8a92 [id=10880639cfc743fcafecc538fa2ecd85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.181861 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1fc01252d95407e954cfcbf7e715912. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.183131 1174 tablet_bootstrap.cc:492] T e6b76967e5d3476997fa019c4a8afb91 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:15.184665 527 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_f0eb2828eb0a4d968bec535ac68a7e3b" } modify_external_catalogs: true
I20260202 02:52:15.184892 527 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_f0eb2828eb0a4d968bec535ac68a7e3b" } modify_external_catalogs: true
I20260202 02:52:15.185212 1174 tablet_bootstrap.cc:654] T e6b76967e5d3476997fa019c4a8afb91 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.187934 1174 tablet_bootstrap.cc:492] T e6b76967e5d3476997fa019c4a8afb91 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:15.188220 1174 ts_tablet_manager.cc:1403] T e6b76967e5d3476997fa019c4a8afb91 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
W20260202 02:52:15.188934 521 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 952d760159844ad08269654264e6381b consensus state has no leader
I20260202 02:52:15.189244 1174 raft_consensus.cc:359] T e6b76967e5d3476997fa019c4a8afb91 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.189473 1174 raft_consensus.cc:385] T e6b76967e5d3476997fa019c4a8afb91 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.189453 521 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_5e1d4f5a21c14fd2bc84e32d68d065bb" } modify_external_catalogs: true
I20260202 02:52:15.189556 1174 raft_consensus.cc:740] T e6b76967e5d3476997fa019c4a8afb91 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.189646 521 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_5e1d4f5a21c14fd2bc84e32d68d065bb" } modify_external_catalogs: true
I20260202 02:52:15.189836 1174 consensus_queue.cc:260] T e6b76967e5d3476997fa019c4a8afb91 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.192696 524 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_85da33c295954de58e2c6290005ac865" } modify_external_catalogs: true
I20260202 02:52:15.192970 524 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_85da33c295954de58e2c6290005ac865" } modify_external_catalogs: true
I20260202 02:52:15.194591 1174 ts_tablet_manager.cc:1434] T e6b76967e5d3476997fa019c4a8afb91 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260202 02:52:15.196530 1044 tablet_service.cc:1505] Processing CreateTablet for tablet c1fc01252d95407e954cfcbf7e715912 (DEFAULT_TABLE table=default.Table_f23c311557a14f4a8c66eaf4af7a8a92 [id=10880639cfc743fcafecc538fa2ecd85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.197065 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1fc01252d95407e954cfcbf7e715912. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.201999 1174 tablet_bootstrap.cc:492] T 58f09c607cce4ccc82afd75c002ed1a8 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:15.203030 1184 raft_consensus.cc:493] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:15.203248 1184 raft_consensus.cc:515] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.203958 1184 leader_election.cc:290] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:15.204407 1174 tablet_bootstrap.cc:654] T 58f09c607cce4ccc82afd75c002ed1a8 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.204754 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "80eb3013f5354022a2dd78bb294a8497" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:15.204972 1064 raft_consensus.cc:2468] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
I20260202 02:52:15.205468 859 leader_election.cc:304] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:15.206759 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "80eb3013f5354022a2dd78bb294a8497" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:15.207036 777 raft_consensus.cc:2468] T 80eb3013f5354022a2dd78bb294a8497 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
I20260202 02:52:15.207880 1184 raft_consensus.cc:2804] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:15.207995 1184 raft_consensus.cc:493] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:15.208083 1184 raft_consensus.cc:3060] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:15.211558 1184 raft_consensus.cc:515] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.213116 1174 tablet_bootstrap.cc:492] T 58f09c607cce4ccc82afd75c002ed1a8 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:15.213333 1174 ts_tablet_manager.cc:1403] T 58f09c607cce4ccc82afd75c002ed1a8 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.008s
I20260202 02:52:15.214184 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "80eb3013f5354022a2dd78bb294a8497" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:15.214237 1174 raft_consensus.cc:359] T 58f09c607cce4ccc82afd75c002ed1a8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.214453 777 raft_consensus.cc:3060] T 80eb3013f5354022a2dd78bb294a8497 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:15.214543 1174 raft_consensus.cc:385] T 58f09c607cce4ccc82afd75c002ed1a8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.214622 1174 raft_consensus.cc:740] T 58f09c607cce4ccc82afd75c002ed1a8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.214949 1174 consensus_queue.cc:260] T 58f09c607cce4ccc82afd75c002ed1a8 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.218571 777 raft_consensus.cc:2468] T 80eb3013f5354022a2dd78bb294a8497 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
I20260202 02:52:15.219630 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "80eb3013f5354022a2dd78bb294a8497" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:15.219856 1064 raft_consensus.cc:3060] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:15.219873 857 leader_election.cc:304] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters:
I20260202 02:52:15.220829 1184 leader_election.cc:290] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:15.221338 1184 raft_consensus.cc:2804] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:15.221561 1184 raft_consensus.cc:697] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:15.221907 1184 consensus_queue.cc:237] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.224877 1174 ts_tablet_manager.cc:1434] T 58f09c607cce4ccc82afd75c002ed1a8 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.011s user 0.002s sys 0.001s
I20260202 02:52:15.228905 751 tablet_service.cc:1505] Processing CreateTablet for tablet d2d27ceea1bb4122b18f0fcf4d2d1bf4 (DEFAULT_TABLE table=default.Table_a2838c9f404646eea3ee18d69eb32a12 [id=1f9c48c8e2484856a66a0719268f22ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.229454 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2d27ceea1bb4122b18f0fcf4d2d1bf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.230846 1174 tablet_bootstrap.cc:492] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:15.233228 1043 tablet_service.cc:1505] Processing CreateTablet for tablet d2d27ceea1bb4122b18f0fcf4d2d1bf4 (DEFAULT_TABLE table=default.Table_a2838c9f404646eea3ee18d69eb32a12 [id=1f9c48c8e2484856a66a0719268f22ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.233593 749 tablet_service.cc:1505] Processing CreateTablet for tablet 098c5cdfcd0f4c52b6ae4f986f07705a (DEFAULT_TABLE table=default.Table_601ac359670d4a44a24253267b53a875 [id=cae0773031e040a2972ccb42c18f2267]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.233747 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2d27ceea1bb4122b18f0fcf4d2d1bf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.234005 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 098c5cdfcd0f4c52b6ae4f986f07705a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.236821 1042 tablet_service.cc:1505] Processing CreateTablet for tablet 098c5cdfcd0f4c52b6ae4f986f07705a (DEFAULT_TABLE table=default.Table_601ac359670d4a44a24253267b53a875 [id=cae0773031e040a2972ccb42c18f2267]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.237331 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 098c5cdfcd0f4c52b6ae4f986f07705a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.247567 1064 raft_consensus.cc:2468] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
I20260202 02:52:15.251219 1183 ts_tablet_manager.cc:1929] T 005cda97ab5c4b70b8fc8ff66a05d996 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:15.254258 1174 tablet_bootstrap.cc:654] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.259590 751 tablet_service.cc:1505] Processing CreateTablet for tablet aa526a69cf914dd08142074636c956b7 (DEFAULT_TABLE table=default.Table_15539ec1a3fc43aab4083fc71964814e [id=9ef504467a544f299b48499e49adf27f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.260226 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa526a69cf914dd08142074636c956b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.264779 749 tablet_service.cc:1505] Processing CreateTablet for tablet 952d760159844ad08269654264e6381b (DEFAULT_TABLE table=default.Table_b6c1fb1e7282492d920385869810b0e0 [id=2cc4567c7bd44745a5851cd8b95b22c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.265256 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 952d760159844ad08269654264e6381b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.265990 1174 tablet_bootstrap.cc:492] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:15.266188 1174 ts_tablet_manager.cc:1403] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.036s user 0.005s sys 0.001s
I20260202 02:52:15.267228 1174 raft_consensus.cc:359] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.267475 1174 raft_consensus.cc:385] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.267565 1174 raft_consensus.cc:740] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.269508 1174 consensus_queue.cc:260] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.275341 1174 ts_tablet_manager.cc:1434] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.009s user 0.002s sys 0.001s
I20260202 02:52:15.276834 526 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 4de41396cffd4d32b86c701b538d1e01
I20260202 02:52:15.277559 526 catalog_manager.cc:6724] ReplaceTablet: tablet 4de41396cffd4d32b86c701b538d1e01 deleted and replaced by tablet bf74f96bea2e4ab7adec42c3c5207e79
I20260202 02:52:15.278640 520 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 575016c0e9f84e99b962c5f14ffa96cc
I20260202 02:52:15.279232 520 catalog_manager.cc:6724] ReplaceTablet: tablet 575016c0e9f84e99b962c5f14ffa96cc deleted and replaced by tablet ff6f5710de4b4c48948aacdf5be29e8b
W20260202 02:52:15.280053 523 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d2d27ceea1bb4122b18f0fcf4d2d1bf4 consensus state has no leader
I20260202 02:52:15.280763 522 master_service.cc:954] ReplaceTablet: received request to replace tablet 575016c0e9f84e99b962c5f14ffa96cc from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:15.281253 1174 tablet_bootstrap.cc:492] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:15.281509 525 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 58f09c607cce4ccc82afd75c002ed1a8
I20260202 02:52:15.281920 525 catalog_manager.cc:6724] ReplaceTablet: tablet 58f09c607cce4ccc82afd75c002ed1a8 deleted and replaced by tablet 560465c063d94ffaab33f9f79a58a357
I20260202 02:52:15.282114 525 master_service.cc:954] ReplaceTablet: received request to replace tablet 672ea3f731ba42ff8b1ff47db039ed22 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:15.283327 1174 tablet_bootstrap.cc:654] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:15.284145 519 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ffd8aa2bc6e344b78dce518411997fb5 consensus state has no leader
I20260202 02:52:15.284158 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 4de41396cffd4d32b86c701b538d1e01 with delete_type TABLET_DATA_DELETED (replaced by tablet bf74f96bea2e4ab7adec42c3c5207e79) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:15.284518 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 575016c0e9f84e99b962c5f14ffa96cc with delete_type TABLET_DATA_DELETED (replaced by tablet ff6f5710de4b4c48948aacdf5be29e8b) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:15.284718 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 58f09c607cce4ccc82afd75c002ed1a8 with delete_type TABLET_DATA_DELETED (replaced by tablet 560465c063d94ffaab33f9f79a58a357) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:15.285334 1180 tablet_replica.cc:333] T 58f09c607cce4ccc82afd75c002ed1a8 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:15.285528 1180 raft_consensus.cc:2243] T 58f09c607cce4ccc82afd75c002ed1a8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:15.285703 1180 raft_consensus.cc:2272] T 58f09c607cce4ccc82afd75c002ed1a8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:15.286540 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 4de41396cffd4d32b86c701b538d1e01 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4de41396cffd4d32b86c701b538d1e01 already in progress: creating tablet
W20260202 02:52:15.286789 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 575016c0e9f84e99b962c5f14ffa96cc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 575016c0e9f84e99b962c5f14ffa96cc already in progress: creating tablet
I20260202 02:52:15.287662 1180 ts_tablet_manager.cc:1916] T 58f09c607cce4ccc82afd75c002ed1a8 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:15.294003 515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.190:33981, user_credentials={real_user=slave}} blocked reactor thread for 61654.5us
I20260202 02:52:15.294682 1180 ts_tablet_manager.cc:1929] T 58f09c607cce4ccc82afd75c002ed1a8 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:15.294970 1180 log.cc:1199] T 58f09c607cce4ccc82afd75c002ed1a8 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/58f09c607cce4ccc82afd75c002ed1a8
I20260202 02:52:15.295465 1180 ts_tablet_manager.cc:1950] T 58f09c607cce4ccc82afd75c002ed1a8 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:15.296847 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 58f09c607cce4ccc82afd75c002ed1a8 (table default.Table_3b335a0806154f98a907f551596ba353 [id=0da4db61b3fe4cf4846edd5530b9327f]) successfully deleted
I20260202 02:52:15.297636 1184 raft_consensus.cc:493] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:15.297930 1184 raft_consensus.cc:515] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.299786 1184 leader_election.cc:290] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:15.301939 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_cf37f87385ee44cea1ff80cc14050ff2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:15.302742 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cf37f87385ee44cea1ff80cc14050ff2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:15.304188 528 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_f23c311557a14f4a8c66eaf4af7a8a92" } modify_external_catalogs: true
I20260202 02:52:15.304430 528 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_f23c311557a14f4a8c66eaf4af7a8a92" } modify_external_catalogs: true
I20260202 02:52:15.305914 523 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_9fd715491a6a4bbc8685138d0194362d" } modify_external_catalogs: true
I20260202 02:52:15.306118 523 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_9fd715491a6a4bbc8685138d0194362d" } modify_external_catalogs: true
I20260202 02:52:15.307673 519 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_2ed2b2d4b5dc4198bb62ee0e72f0e6ab" } new_table_name: "default.Table_5eefa56618e84f8d92f8d87132ccd7d4" modify_external_catalogs: true
I20260202 02:52:15.312042 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 81f35df38f284f0ca5c3791f3d9d9293 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:15.312435 1180 tablet_replica.cc:333] T 81f35df38f284f0ca5c3791f3d9d9293 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:15.312623 1180 raft_consensus.cc:2243] T 81f35df38f284f0ca5c3791f3d9d9293 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:15.312831 1180 raft_consensus.cc:2272] T 81f35df38f284f0ca5c3791f3d9d9293 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:15.312944 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_950ad480ee834bbdbd74eaaccccb42ec"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:15.313293 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 7e81d5780d9647b9baac0066282376ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:42950
W20260202 02:52:15.313567 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_950ad480ee834bbdbd74eaaccccb42ec in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:15.314050 1180 ts_tablet_manager.cc:1916] T 81f35df38f284f0ca5c3791f3d9d9293 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:15.321659 1180 ts_tablet_manager.cc:1929] T 81f35df38f284f0ca5c3791f3d9d9293 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:15.321919 1180 log.cc:1199] T 81f35df38f284f0ca5c3791f3d9d9293 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/81f35df38f284f0ca5c3791f3d9d9293
I20260202 02:52:15.322311 756 tablet_service.cc:1505] Processing CreateTablet for tablet 5986676f5484498da927b4c6da3f47ff (DEFAULT_TABLE table=default.Table_8cb3cbab1a654f88a07d6d480fc72e52 [id=d8485f23f63b493781bea557dbe8a0c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.322861 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5986676f5484498da927b4c6da3f47ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.325839 754 tablet_service.cc:1505] Processing CreateTablet for tablet 23c38ef4e3c840808ec36f859cf6f6de (DEFAULT_TABLE table=default.Table_9fd715491a6a4bbc8685138d0194362d [id=98ddcbfb5ad14ae480a54845ec7c45a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.326241 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23c38ef4e3c840808ec36f859cf6f6de. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:15.327692 857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 107918us
I20260202 02:52:15.329439 751 tablet_service.cc:1505] Processing CreateTablet for tablet 0d297b6243c348ee82c146a8339cf6c4 (DEFAULT_TABLE table=default.Table_d210d1530706429197d6294b29f18c87 [id=7c76848b98614fbf8b2950e5d55cc609]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.329902 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d297b6243c348ee82c146a8339cf6c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.330013 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e6b76967e5d3476997fa019c4a8afb91" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:15.330260 777 raft_consensus.cc:2468] T e6b76967e5d3476997fa019c4a8afb91 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
I20260202 02:52:15.331882 897 tablet_service.cc:1505] Processing CreateTablet for tablet 952d760159844ad08269654264e6381b (DEFAULT_TABLE table=default.Table_b6c1fb1e7282492d920385869810b0e0 [id=2cc4567c7bd44745a5851cd8b95b22c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.332525 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 952d760159844ad08269654264e6381b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.332681 749 tablet_service.cc:1505] Processing CreateTablet for tablet 4f2a9404f7424dc78b5bd062ebd8cb53 (DEFAULT_TABLE table=default.Table_0842fd6a82e145e7a98b3fed2c0c6993 [id=4e076b2023be4c888ac1c27babde1607]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.333088 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f2a9404f7424dc78b5bd062ebd8cb53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.335299 896 tablet_service.cc:1552] Processing DeleteTablet for tablet 58f09c607cce4ccc82afd75c002ed1a8 with delete_type TABLET_DATA_DELETED (replaced by tablet 560465c063d94ffaab33f9f79a58a357) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:15.335644 1182 tablet_replica.cc:333] T 58f09c607cce4ccc82afd75c002ed1a8 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:15.335839 1182 raft_consensus.cc:2243] T 58f09c607cce4ccc82afd75c002ed1a8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:15.336024 1182 raft_consensus.cc:2272] T 58f09c607cce4ccc82afd75c002ed1a8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:15.337343 901 tablet_service.cc:1505] Processing CreateTablet for tablet aa526a69cf914dd08142074636c956b7 (DEFAULT_TABLE table=default.Table_15539ec1a3fc43aab4083fc71964814e [id=9ef504467a544f299b48499e49adf27f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.337536 900 tablet_service.cc:1505] Processing CreateTablet for tablet 098c5cdfcd0f4c52b6ae4f986f07705a (DEFAULT_TABLE table=default.Table_601ac359670d4a44a24253267b53a875 [id=cae0773031e040a2972ccb42c18f2267]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.337774 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa526a69cf914dd08142074636c956b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.338327 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 098c5cdfcd0f4c52b6ae4f986f07705a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.340512 903 tablet_service.cc:1552] Processing DeleteTablet for tablet 4de41396cffd4d32b86c701b538d1e01 with delete_type TABLET_DATA_DELETED (replaced by tablet bf74f96bea2e4ab7adec42c3c5207e79) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:15.340790 899 tablet_service.cc:1552] Processing DeleteTablet for tablet 575016c0e9f84e99b962c5f14ffa96cc with delete_type TABLET_DATA_DELETED (replaced by tablet ff6f5710de4b4c48948aacdf5be29e8b) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:15.341058 1209 raft_consensus.cc:493] T 7e81d5780d9647b9baac0066282376ea P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:15.341207 1209 raft_consensus.cc:515] T 7e81d5780d9647b9baac0066282376ea P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.341936 1209 leader_election.cc:290] T 7e81d5780d9647b9baac0066282376ea P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:15.342159 897 tablet_service.cc:1505] Processing CreateTablet for tablet 4f2a9404f7424dc78b5bd062ebd8cb53 (DEFAULT_TABLE table=default.Table_0842fd6a82e145e7a98b3fed2c0c6993 [id=4e076b2023be4c888ac1c27babde1607]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.342862 899 tablet_service.cc:1505] Processing CreateTablet for tablet 5986676f5484498da927b4c6da3f47ff (DEFAULT_TABLE table=default.Table_8cb3cbab1a654f88a07d6d480fc72e52 [id=d8485f23f63b493781bea557dbe8a0c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.343137 903 tablet_service.cc:1505] Processing CreateTablet for tablet 0d297b6243c348ee82c146a8339cf6c4 (DEFAULT_TABLE table=default.Table_d210d1530706429197d6294b29f18c87 [id=7c76848b98614fbf8b2950e5d55cc609]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.342639 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f2a9404f7424dc78b5bd062ebd8cb53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.343672 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5986676f5484498da927b4c6da3f47ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.343904 1182 ts_tablet_manager.cc:1916] T 58f09c607cce4ccc82afd75c002ed1a8 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:15.349746 898 tablet_service.cc:1505] Processing CreateTablet for tablet d2d27ceea1bb4122b18f0fcf4d2d1bf4 (DEFAULT_TABLE table=default.Table_a2838c9f404646eea3ee18d69eb32a12 [id=1f9c48c8e2484856a66a0719268f22ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.350237 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2d27ceea1bb4122b18f0fcf4d2d1bf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.342941 857 leader_election.cc:304] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters:
I20260202 02:52:15.351056 903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d297b6243c348ee82c146a8339cf6c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.351696 1269 raft_consensus.cc:2804] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:15.351835 1269 raft_consensus.cc:493] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:15.351991 1269 raft_consensus.cc:3060] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:15.359218 1183 log.cc:1199] T 005cda97ab5c4b70b8fc8ff66a05d996 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/005cda97ab5c4b70b8fc8ff66a05d996
I20260202 02:52:15.359740 1183 ts_tablet_manager.cc:1950] T 005cda97ab5c4b70b8fc8ff66a05d996 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:15.261829 1173 tablet_bootstrap.cc:492] T 6399b63cd8974ee49796f8779f9abc0e P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:15.360594 1173 ts_tablet_manager.cc:1403] T 6399b63cd8974ee49796f8779f9abc0e P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.305s user 0.003s sys 0.003s
I20260202 02:52:15.361544 1173 raft_consensus.cc:359] T 6399b63cd8974ee49796f8779f9abc0e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.361861 1173 raft_consensus.cc:385] T 6399b63cd8974ee49796f8779f9abc0e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.361981 1173 raft_consensus.cc:740] T 6399b63cd8974ee49796f8779f9abc0e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.362327 1173 consensus_queue.cc:260] T 6399b63cd8974ee49796f8779f9abc0e P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.364761 1269 raft_consensus.cc:515] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.365561 1269 leader_election.cc:290] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:15.366164 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e6b76967e5d3476997fa019c4a8afb91" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
W20260202 02:52:15.366863 859 leader_election.cc:343] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:15.367241 714 leader_election.cc:343] T 7e81d5780d9647b9baac0066282376ea P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 7e81d5780d9647b9baac0066282376ea
I20260202 02:52:15.367583 1173 ts_tablet_manager.cc:1434] T 6399b63cd8974ee49796f8779f9abc0e P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260202 02:52:15.368486 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e6b76967e5d3476997fa019c4a8afb91" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:15.368716 777 raft_consensus.cc:3060] T e6b76967e5d3476997fa019c4a8afb91 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:15.369215 1173 tablet_bootstrap.cc:492] T 81f35df38f284f0ca5c3791f3d9d9293 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:15.296502 1040 tablet_service.cc:1552] Processing DeleteTablet for tablet 4de41396cffd4d32b86c701b538d1e01 with delete_type TABLET_DATA_DELETED (replaced by tablet bf74f96bea2e4ab7adec42c3c5207e79) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:15.297139 1041 tablet_service.cc:1505] Processing CreateTablet for tablet aa526a69cf914dd08142074636c956b7 (DEFAULT_TABLE table=default.Table_15539ec1a3fc43aab4083fc71964814e [id=9ef504467a544f299b48499e49adf27f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.299432 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e6b76967e5d3476997fa019c4a8afb91" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:15.328236 1042 tablet_service.cc:1552] Processing DeleteTablet for tablet 575016c0e9f84e99b962c5f14ffa96cc with delete_type TABLET_DATA_DELETED (replaced by tablet ff6f5710de4b4c48948aacdf5be29e8b) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:15.329697 1043 tablet_service.cc:1552] Processing DeleteTablet for tablet 58f09c607cce4ccc82afd75c002ed1a8 with delete_type TABLET_DATA_DELETED (replaced by tablet 560465c063d94ffaab33f9f79a58a357) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:15.342705 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7e81d5780d9647b9baac0066282376ea" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:15.371639 715 leader_election.cc:343] T 7e81d5780d9647b9baac0066282376ea P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:15.371694 777 raft_consensus.cc:2468] T e6b76967e5d3476997fa019c4a8afb91 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
I20260202 02:52:15.371786 715 leader_election.cc:304] T 7e81d5780d9647b9baac0066282376ea P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
W20260202 02:52:15.372347 859 leader_election.cc:343] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:15.372697 1209 raft_consensus.cc:2749] T 7e81d5780d9647b9baac0066282376ea P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:15.372645 857 leader_election.cc:304] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:15.346714 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 952d760159844ad08269654264e6381b (DEFAULT_TABLE table=default.Table_b6c1fb1e7282492d920385869810b0e0 [id=2cc4567c7bd44745a5851cd8b95b22c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.373171 1182 ts_tablet_manager.cc:1929] T 58f09c607cce4ccc82afd75c002ed1a8 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:15.373266 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 952d760159844ad08269654264e6381b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.373306 1182 log.cc:1199] T 58f09c607cce4ccc82afd75c002ed1a8 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/58f09c607cce4ccc82afd75c002ed1a8
I20260202 02:52:15.373745 1182 ts_tablet_manager.cc:1950] T 58f09c607cce4ccc82afd75c002ed1a8 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:15.347812 1039 tablet_service.cc:1505] Processing CreateTablet for tablet 5986676f5484498da927b4c6da3f47ff (DEFAULT_TABLE table=default.Table_8cb3cbab1a654f88a07d6d480fc72e52 [id=d8485f23f63b493781bea557dbe8a0c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.374857 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5986676f5484498da927b4c6da3f47ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.375818 1180 ts_tablet_manager.cc:1950] T 81f35df38f284f0ca5c3791f3d9d9293 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:15.376724 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 58f09c607cce4ccc82afd75c002ed1a8 (table default.Table_3b335a0806154f98a907f551596ba353 [id=0da4db61b3fe4cf4846edd5530b9327f]) successfully deleted
I20260202 02:52:15.376891 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 81f35df38f284f0ca5c3791f3d9d9293 (table default.Table_15c57b6d38ab4f2fb7dca2a405f01058 [id=d54948158c124d00ab0e2b1f43d45cc3]) successfully deleted
W20260202 02:52:15.377466 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 4de41396cffd4d32b86c701b538d1e01 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4de41396cffd4d32b86c701b538d1e01 already in progress: creating tablet
W20260202 02:52:15.377871 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 575016c0e9f84e99b962c5f14ffa96cc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 575016c0e9f84e99b962c5f14ffa96cc already in progress: creating tablet
I20260202 02:52:15.349447 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 4f2a9404f7424dc78b5bd062ebd8cb53 (DEFAULT_TABLE table=default.Table_0842fd6a82e145e7a98b3fed2c0c6993 [id=4e076b2023be4c888ac1c27babde1607]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.380842 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f2a9404f7424dc78b5bd062ebd8cb53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.380899 1180 tablet_replica.cc:333] T 7e81d5780d9647b9baac0066282376ea P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:15.381096 1180 raft_consensus.cc:2243] T 7e81d5780d9647b9baac0066282376ea P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:15.381271 1180 raft_consensus.cc:2272] T 7e81d5780d9647b9baac0066282376ea P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:15.382221 1180 ts_tablet_manager.cc:1916] T 7e81d5780d9647b9baac0066282376ea P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:15.383931 1269 raft_consensus.cc:2804] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:15.349599 1038 tablet_service.cc:1505] Processing CreateTablet for tablet 0d297b6243c348ee82c146a8339cf6c4 (DEFAULT_TABLE table=default.Table_d210d1530706429197d6294b29f18c87 [id=7c76848b98614fbf8b2950e5d55cc609]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.384774 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d297b6243c348ee82c146a8339cf6c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.385210 1269 raft_consensus.cc:697] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:15.385519 1269 consensus_queue.cc:237] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.390672 1180 ts_tablet_manager.cc:1929] T 7e81d5780d9647b9baac0066282376ea P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:15.390836 1180 log.cc:1199] T 7e81d5780d9647b9baac0066282376ea P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/7e81d5780d9647b9baac0066282376ea
I20260202 02:52:15.391187 1180 ts_tablet_manager.cc:1950] T 7e81d5780d9647b9baac0066282376ea P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:15.392532 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 7e81d5780d9647b9baac0066282376ea (table default.Table_629c3da327a740f684072799e959c2ca [id=30cd87ed1178421bb31ee622ce3254c1]) successfully deleted
I20260202 02:52:15.418785 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa526a69cf914dd08142074636c956b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.433429 1183 tablet_replica.cc:333] T 2730d8b6812e487ea0aff4da45d24044 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:15.433779 1183 raft_consensus.cc:2243] T 2730d8b6812e487ea0aff4da45d24044 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:15.434024 1183 raft_consensus.cc:2272] T 2730d8b6812e487ea0aff4da45d24044 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:15.434900 1183 ts_tablet_manager.cc:1916] T 2730d8b6812e487ea0aff4da45d24044 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:15.439123 1173 tablet_bootstrap.cc:654] T 81f35df38f284f0ca5c3791f3d9d9293 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.439435 1175 tablet_bootstrap.cc:492] T 72475ea720c6429aa68de9478b374280 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:15.439635 1175 ts_tablet_manager.cc:1403] T 72475ea720c6429aa68de9478b374280 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.359s user 0.186s sys 0.145s
I20260202 02:52:15.440599 1175 raft_consensus.cc:359] T 72475ea720c6429aa68de9478b374280 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.440948 1175 raft_consensus.cc:385] T 72475ea720c6429aa68de9478b374280 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.441071 1175 raft_consensus.cc:740] T 72475ea720c6429aa68de9478b374280 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.441332 1175 consensus_queue.cc:260] T 72475ea720c6429aa68de9478b374280 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.442194 1175 ts_tablet_manager.cc:1434] T 72475ea720c6429aa68de9478b374280 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260202 02:52:15.442574 1175 tablet_bootstrap.cc:492] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:15.445510 1175 tablet_bootstrap.cc:654] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.450874 527 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet 24e3a18a62cb4d32acd9c376934c2d8d
I20260202 02:52:15.452927 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_563264f6196a4d6195a257c4ae9c9397"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:15.453600 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_563264f6196a4d6195a257c4ae9c9397 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:15.454830 521 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet 3c1b4afd0ada4b6cba0c0138ded6cdd8
I20260202 02:52:15.455219 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_983bf3e0c5424c2588a0e754af1a5687"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:15.455744 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_983bf3e0c5424c2588a0e754af1a5687 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:15.462848 1209 raft_consensus.cc:493] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:15.463063 1209 raft_consensus.cc:515] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.463781 1209 leader_election.cc:290] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:15.464118 897 tablet_service.cc:1505] Processing CreateTablet for tablet 23c38ef4e3c840808ec36f859cf6f6de (DEFAULT_TABLE table=default.Table_9fd715491a6a4bbc8685138d0194362d [id=98ddcbfb5ad14ae480a54845ec7c45a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.464644 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23c38ef4e3c840808ec36f859cf6f6de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.464630 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fae7e3941c124cc2871390a0b6471f27" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:15.464974 1037 tablet_service.cc:1552] Processing DeleteTablet for tablet 6399b63cd8974ee49796f8779f9abc0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:15.465440 715 leader_election.cc:343] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:15.466190 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 23c38ef4e3c840808ec36f859cf6f6de (DEFAULT_TABLE table=default.Table_9fd715491a6a4bbc8685138d0194362d [id=98ddcbfb5ad14ae480a54845ec7c45a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.466655 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23c38ef4e3c840808ec36f859cf6f6de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.469051 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fae7e3941c124cc2871390a0b6471f27" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:15.469296 923 raft_consensus.cc:2468] T fae7e3941c124cc2871390a0b6471f27 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:15.469864 714 leader_election.cc:304] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:15.470491 1209 raft_consensus.cc:2804] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:15.470613 1209 raft_consensus.cc:493] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:15.470688 1209 raft_consensus.cc:3060] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:15.472272 1175 tablet_bootstrap.cc:492] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:15.472471 1175 ts_tablet_manager.cc:1403] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.030s user 0.006s sys 0.004s
I20260202 02:52:15.473124 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 005cda97ab5c4b70b8fc8ff66a05d996 (table default.Table_0842fd6a82e145e7a98b3fed2c0c6993 [id=4e076b2023be4c888ac1c27babde1607]) successfully deleted
I20260202 02:52:15.473308 1175 raft_consensus.cc:359] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.473354 1209 raft_consensus.cc:515] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.473498 1175 raft_consensus.cc:385] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.473558 1175 raft_consensus.cc:740] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.473817 1175 consensus_queue.cc:260] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.474589 1175 ts_tablet_manager.cc:1434] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:15.474905 1175 tablet_bootstrap.cc:492] T 4de41396cffd4d32b86c701b538d1e01 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:15.475677 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fae7e3941c124cc2871390a0b6471f27" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:15.475868 923 raft_consensus.cc:3060] T fae7e3941c124cc2871390a0b6471f27 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:15.476256 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fae7e3941c124cc2871390a0b6471f27" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:15.476809 524 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 672ea3f731ba42ff8b1ff47db039ed22
W20260202 02:52:15.476804 715 leader_election.cc:343] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:15.478019 1209 leader_election.cc:290] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:15.478561 923 raft_consensus.cc:2468] T fae7e3941c124cc2871390a0b6471f27 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:15.478588 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_4336d55393794a208c76f0a580420596"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:15.479045 714 leader_election.cc:304] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
W20260202 02:52:15.479164 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4336d55393794a208c76f0a580420596 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:15.480103 1209 raft_consensus.cc:2804] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:15.480918 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_98d65bff8bd24989b88007d79e20c2ff"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:15.481560 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_98d65bff8bd24989b88007d79e20c2ff in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:15.482858 528 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet c1fc01252d95407e954cfcbf7e715912
I20260202 02:52:15.483669 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_08912fa0817a47e6a48aa728d44438cf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:15.484313 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_08912fa0817a47e6a48aa728d44438cf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:15.485706 523 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 23c38ef4e3c840808ec36f859cf6f6de
I20260202 02:52:15.486485 523 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_15539ec1a3fc43aab4083fc71964814e" } new_table_name: "default.Table_9451b9f5148f4d9aab65983dea967e32" modify_external_catalogs: true
W20260202 02:52:15.488044 519 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2ff6af17a6854451a1d7b904aa4f1f55 consensus state has no leader
I20260202 02:52:15.488588 519 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_2ae489b852d84e119c265efad1950d4b" } new_table_name: "default.Table_e70b044a267b465d94fede069f2f0df6" modify_external_catalogs: true
I20260202 02:52:15.488849 1175 tablet_bootstrap.cc:654] T 4de41396cffd4d32b86c701b538d1e01 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.490070 1209 raft_consensus.cc:697] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:15.490275 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_63bb1b19c34a430e94ad0d7bd9bbe795"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:15.490453 1209 consensus_queue.cc:237] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
W20260202 02:52:15.490844 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_63bb1b19c34a430e94ad0d7bd9bbe795 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:15.493774 1175 tablet_bootstrap.cc:492] T 4de41396cffd4d32b86c701b538d1e01 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:15.493938 1175 ts_tablet_manager.cc:1403] T 4de41396cffd4d32b86c701b538d1e01 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.000s
I20260202 02:52:15.493947 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 672ea3f731ba42ff8b1ff47db039ed22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:15.494242 751 tablet_service.cc:1552] Processing DeleteTablet for tablet c1fc01252d95407e954cfcbf7e715912 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:15.494477 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 23c38ef4e3c840808ec36f859cf6f6de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:15.494783 897 tablet_service.cc:1552] Processing DeleteTablet for tablet 672ea3f731ba42ff8b1ff47db039ed22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:15.494841 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 672ea3f731ba42ff8b1ff47db039ed22 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 672ea3f731ba42ff8b1ff47db039ed22 already in progress: creating tablet
I20260202 02:52:15.494992 901 tablet_service.cc:1552] Processing DeleteTablet for tablet c1fc01252d95407e954cfcbf7e715912 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:15.495041 903 tablet_service.cc:1552] Processing DeleteTablet for tablet 23c38ef4e3c840808ec36f859cf6f6de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:15.495103 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet c1fc01252d95407e954cfcbf7e715912 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c1fc01252d95407e954cfcbf7e715912 already in progress: creating tablet
W20260202 02:52:15.495752 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 23c38ef4e3c840808ec36f859cf6f6de because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 23c38ef4e3c840808ec36f859cf6f6de already in progress: creating tablet
I20260202 02:52:15.496769 901 tablet_service.cc:1552] Processing DeleteTablet for tablet 58f09c607cce4ccc82afd75c002ed1a8 with delete_type TABLET_DATA_DELETED (replaced by tablet 560465c063d94ffaab33f9f79a58a357) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:15.496822 903 tablet_service.cc:1552] Processing DeleteTablet for tablet e6b76967e5d3476997fa019c4a8afb91 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:15.497207 1182 tablet_replica.cc:333] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:15.497412 1182 raft_consensus.cc:2243] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:15.497673 1182 pending_rounds.cc:70] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5: Trying to abort 1 pending ops.
I20260202 02:52:15.497562 1175 raft_consensus.cc:359] T 4de41396cffd4d32b86c701b538d1e01 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.497764 1182 pending_rounds.cc:77] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923012146335744 op_type: NO_OP noop_request { }
I20260202 02:52:15.497956 1182 raft_consensus.cc:2889] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
W20260202 02:52:15.498014 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 672ea3f731ba42ff8b1ff47db039ed22 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 672ea3f731ba42ff8b1ff47db039ed22 already in progress: creating tablet
I20260202 02:52:15.498085 1182 raft_consensus.cc:2272] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:15.498240 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet c1fc01252d95407e954cfcbf7e715912 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c1fc01252d95407e954cfcbf7e715912 already in progress: creating tablet
I20260202 02:52:15.498338 520 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet fae7e3941c124cc2871390a0b6471f27 (table default.Table_3d26f7a34d0c48379e3cebda66e1ef13 [id=45e6d83589fa493195b06f633e295b00]). Expected version 2 got 0
W20260202 02:52:15.498476 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 23c38ef4e3c840808ec36f859cf6f6de because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 23c38ef4e3c840808ec36f859cf6f6de already in progress: creating tablet
W20260202 02:52:15.498983 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 58f09c607cce4ccc82afd75c002ed1a8 because the tablet was not found. No further retry: Not found: Tablet not found: 58f09c607cce4ccc82afd75c002ed1a8
I20260202 02:52:15.499017 1175 raft_consensus.cc:385] T 4de41396cffd4d32b86c701b538d1e01 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.499122 1175 raft_consensus.cc:740] T 4de41396cffd4d32b86c701b538d1e01 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.500249 1182 ts_tablet_manager.cc:1916] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:15.500473 754 tablet_service.cc:1552] Processing DeleteTablet for tablet c2cc2250701d46a7abe7be03b11d45c8 with delete_type TABLET_DATA_DELETED (replaced by tablet 0d297b6243c348ee82c146a8339cf6c4) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:15.500770 1180 tablet_replica.cc:333] T c2cc2250701d46a7abe7be03b11d45c8 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:15.500947 1180 raft_consensus.cc:2243] T c2cc2250701d46a7abe7be03b11d45c8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:15.501132 1180 raft_consensus.cc:2272] T c2cc2250701d46a7abe7be03b11d45c8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:15.501500 751 tablet_service.cc:1552] Processing DeleteTablet for tablet e6b76967e5d3476997fa019c4a8afb91 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:15.501753 1175 consensus_queue.cc:260] T 4de41396cffd4d32b86c701b538d1e01 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.502009 1180 ts_tablet_manager.cc:1916] T c2cc2250701d46a7abe7be03b11d45c8 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:15.502280 751 tablet_service.cc:1552] Processing DeleteTablet for tablet fde2c1fba78248329f3662dd1036c7a2 with delete_type TABLET_DATA_DELETED (replaced by tablet 23c38ef4e3c840808ec36f859cf6f6de) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:15.502795 1175 ts_tablet_manager.cc:1434] T 4de41396cffd4d32b86c701b538d1e01 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.009s user 0.003s sys 0.000s
I20260202 02:52:15.503077 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 672ea3f731ba42ff8b1ff47db039ed22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:15.503319 1175 tablet_bootstrap.cc:492] T fc93352b24e345f99d15085e4886c748 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:15.505649 1175 tablet_bootstrap.cc:654] T fc93352b24e345f99d15085e4886c748 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.514292 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 23c38ef4e3c840808ec36f859cf6f6de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:15.514916 1180 ts_tablet_manager.cc:1929] T c2cc2250701d46a7abe7be03b11d45c8 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:15.515059 1180 log.cc:1199] T c2cc2250701d46a7abe7be03b11d45c8 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/c2cc2250701d46a7abe7be03b11d45c8
I20260202 02:52:15.515439 1180 ts_tablet_manager.cc:1950] T c2cc2250701d46a7abe7be03b11d45c8 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:15.516494 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet c2cc2250701d46a7abe7be03b11d45c8 (table default.Table_d210d1530706429197d6294b29f18c87 [id=7c76848b98614fbf8b2950e5d55cc609]) successfully deleted
I20260202 02:52:15.517916 1182 ts_tablet_manager.cc:1929] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:15.518297 1180 tablet_replica.cc:333] T e6b76967e5d3476997fa019c4a8afb91 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:15.518581 1180 raft_consensus.cc:2243] T e6b76967e5d3476997fa019c4a8afb91 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:15.518815 1180 raft_consensus.cc:2272] T e6b76967e5d3476997fa019c4a8afb91 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:15.520068 1182 log.cc:1199] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/e6b76967e5d3476997fa019c4a8afb91
I20260202 02:52:15.520664 1182 ts_tablet_manager.cc:1950] T e6b76967e5d3476997fa019c4a8afb91 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:15.519776 1180 ts_tablet_manager.cc:1916] T e6b76967e5d3476997fa019c4a8afb91 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:15.521031 1175 tablet_bootstrap.cc:492] T fc93352b24e345f99d15085e4886c748 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:15.523844 1175 ts_tablet_manager.cc:1403] T fc93352b24e345f99d15085e4886c748 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.004s
I20260202 02:52:15.524616 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet e6b76967e5d3476997fa019c4a8afb91 (table default.Table_e6d3c2dc7ad1416c949573796c5dc398 [id=a9429c7c44f144baa985c0731279243e]) successfully deleted
I20260202 02:52:15.524755 1175 raft_consensus.cc:359] T fc93352b24e345f99d15085e4886c748 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.524986 1175 raft_consensus.cc:385] T fc93352b24e345f99d15085e4886c748 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.525056 1175 raft_consensus.cc:740] T fc93352b24e345f99d15085e4886c748 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.525280 1175 consensus_queue.cc:260] T fc93352b24e345f99d15085e4886c748 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.526121 1175 ts_tablet_manager.cc:1434] T fc93352b24e345f99d15085e4886c748 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:15.526563 1175 tablet_bootstrap.cc:492] T 13309b20a96240b5aa9fbb691cbed981 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:15.528735 1175 tablet_bootstrap.cc:654] T 13309b20a96240b5aa9fbb691cbed981 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.531847 1037 tablet_service.cc:1552] Processing DeleteTablet for tablet c1fc01252d95407e954cfcbf7e715912 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:15.537950 1173 tablet_bootstrap.cc:492] T 81f35df38f284f0ca5c3791f3d9d9293 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:15.538224 1173 ts_tablet_manager.cc:1403] T 81f35df38f284f0ca5c3791f3d9d9293 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.169s user 0.002s sys 0.002s
I20260202 02:52:15.539495 1173 raft_consensus.cc:359] T 81f35df38f284f0ca5c3791f3d9d9293 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.539791 1173 raft_consensus.cc:385] T 81f35df38f284f0ca5c3791f3d9d9293 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.539938 1173 raft_consensus.cc:740] T 81f35df38f284f0ca5c3791f3d9d9293 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.540285 1173 consensus_queue.cc:260] T 81f35df38f284f0ca5c3791f3d9d9293 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.540992 1174 tablet_bootstrap.cc:492] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:15.541177 1174 ts_tablet_manager.cc:1403] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.260s user 0.024s sys 0.076s
I20260202 02:52:15.541909 1175 tablet_bootstrap.cc:492] T 13309b20a96240b5aa9fbb691cbed981 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:15.542057 1175 ts_tablet_manager.cc:1403] T 13309b20a96240b5aa9fbb691cbed981 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.001s
I20260202 02:52:15.541971 1174 raft_consensus.cc:359] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.542167 1174 raft_consensus.cc:385] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.542241 1174 raft_consensus.cc:740] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.542527 1174 consensus_queue.cc:260] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.542843 1175 raft_consensus.cc:359] T 13309b20a96240b5aa9fbb691cbed981 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.543193 1175 raft_consensus.cc:385] T 13309b20a96240b5aa9fbb691cbed981 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.543277 1175 raft_consensus.cc:740] T 13309b20a96240b5aa9fbb691cbed981 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.543511 1175 consensus_queue.cc:260] T 13309b20a96240b5aa9fbb691cbed981 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.544976 1175 ts_tablet_manager.cc:1434] T 13309b20a96240b5aa9fbb691cbed981 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260202 02:52:15.545578 1173 ts_tablet_manager.cc:1434] T 81f35df38f284f0ca5c3791f3d9d9293 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260202 02:52:15.545694 1175 tablet_bootstrap.cc:492] T ffd8aa2bc6e344b78dce518411997fb5 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:15.546372 1174 ts_tablet_manager.cc:1434] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260202 02:52:15.548296 1175 tablet_bootstrap.cc:654] T ffd8aa2bc6e344b78dce518411997fb5 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.548938 1173 tablet_bootstrap.cc:492] T 7e81d5780d9647b9baac0066282376ea P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:15.551291 1173 tablet_bootstrap.cc:654] T 7e81d5780d9647b9baac0066282376ea P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.553200 1174 tablet_bootstrap.cc:492] T 72475ea720c6429aa68de9478b374280 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:15.555229 1174 tablet_bootstrap.cc:654] T 72475ea720c6429aa68de9478b374280 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.557159 520 master_service.cc:954] ReplaceTablet: received request to replace tablet f24c0b99574d437a99b273204f105460 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:15.557997 1180 ts_tablet_manager.cc:1929] T e6b76967e5d3476997fa019c4a8afb91 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:15.558220 1180 log.cc:1199] T e6b76967e5d3476997fa019c4a8afb91 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/e6b76967e5d3476997fa019c4a8afb91
I20260202 02:52:15.558602 1180 ts_tablet_manager.cc:1950] T e6b76967e5d3476997fa019c4a8afb91 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:15.559300 1269 raft_consensus.cc:493] T 4de41396cffd4d32b86c701b538d1e01 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:15.559491 1269 raft_consensus.cc:515] T 4de41396cffd4d32b86c701b538d1e01 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.559785 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet e6b76967e5d3476997fa019c4a8afb91 (table default.Table_e6d3c2dc7ad1416c949573796c5dc398 [id=a9429c7c44f144baa985c0731279243e]) successfully deleted
I20260202 02:52:15.560221 1269 leader_election.cc:290] T 4de41396cffd4d32b86c701b538d1e01 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:15.560818 1180 tablet_replica.cc:333] T fde2c1fba78248329f3662dd1036c7a2 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:15.560973 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4de41396cffd4d32b86c701b538d1e01" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:15.561866 1180 raft_consensus.cc:2243] T fde2c1fba78248329f3662dd1036c7a2 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
W20260202 02:52:15.561982 857 leader_election.cc:343] T 4de41396cffd4d32b86c701b538d1e01 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:15.562247 1180 raft_consensus.cc:2272] T fde2c1fba78248329f3662dd1036c7a2 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:15.563107 1180 ts_tablet_manager.cc:1916] T fde2c1fba78248329f3662dd1036c7a2 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:15.563282 1175 tablet_bootstrap.cc:492] T ffd8aa2bc6e344b78dce518411997fb5 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:15.563433 1175 ts_tablet_manager.cc:1403] T ffd8aa2bc6e344b78dce518411997fb5 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.008s
I20260202 02:52:15.564252 1175 raft_consensus.cc:359] T ffd8aa2bc6e344b78dce518411997fb5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.564492 1175 raft_consensus.cc:385] T ffd8aa2bc6e344b78dce518411997fb5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.564569 1175 raft_consensus.cc:740] T ffd8aa2bc6e344b78dce518411997fb5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.564817 1175 consensus_queue.cc:260] T ffd8aa2bc6e344b78dce518411997fb5 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.565220 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_7ff64c2b44d0417ea6445647a80a5b86" } new_table_name: "default.Table_f1e417bf19b94bc491c3937276c669fd" modify_external_catalogs: true
I20260202 02:52:15.566107 1174 tablet_bootstrap.cc:492] T 72475ea720c6429aa68de9478b374280 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:15.566260 1174 ts_tablet_manager.cc:1403] T 72475ea720c6429aa68de9478b374280 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.001s
I20260202 02:52:15.567058 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_38d2944c79ed4c609e92afac1acb0e5f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:15.567080 1174 raft_consensus.cc:359] T 72475ea720c6429aa68de9478b374280 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.567260 1174 raft_consensus.cc:385] T 72475ea720c6429aa68de9478b374280 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.567338 1174 raft_consensus.cc:740] T 72475ea720c6429aa68de9478b374280 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.567564 1174 consensus_queue.cc:260] T 72475ea720c6429aa68de9478b374280 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.568219 1174 ts_tablet_manager.cc:1434] T 72475ea720c6429aa68de9478b374280 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:15.568554 1174 tablet_bootstrap.cc:492] T 4de41396cffd4d32b86c701b538d1e01 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
W20260202 02:52:15.569517 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_38d2944c79ed4c609e92afac1acb0e5f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:15.570842 1174 tablet_bootstrap.cc:654] T 4de41396cffd4d32b86c701b538d1e01 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.572099 1175 ts_tablet_manager.cc:1434] T ffd8aa2bc6e344b78dce518411997fb5 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.009s user 0.003s sys 0.000s
I20260202 02:52:15.572741 1175 tablet_bootstrap.cc:492] T 672ea3f731ba42ff8b1ff47db039ed22 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:15.575179 1175 tablet_bootstrap.cc:654] T 672ea3f731ba42ff8b1ff47db039ed22 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.579231 1180 ts_tablet_manager.cc:1929] T fde2c1fba78248329f3662dd1036c7a2 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:15.579384 1180 log.cc:1199] T fde2c1fba78248329f3662dd1036c7a2 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/fde2c1fba78248329f3662dd1036c7a2
I20260202 02:52:15.579753 1180 ts_tablet_manager.cc:1950] T fde2c1fba78248329f3662dd1036c7a2 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:15.580958 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet fde2c1fba78248329f3662dd1036c7a2 (table default.Table_9fd715491a6a4bbc8685138d0194362d [id=98ddcbfb5ad14ae480a54845ec7c45a4]) successfully deleted
I20260202 02:52:15.582463 1174 tablet_bootstrap.cc:492] T 4de41396cffd4d32b86c701b538d1e01 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:15.582641 1174 ts_tablet_manager.cc:1403] T 4de41396cffd4d32b86c701b538d1e01 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.001s
I20260202 02:52:15.583496 1174 raft_consensus.cc:359] T 4de41396cffd4d32b86c701b538d1e01 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.583736 1174 raft_consensus.cc:385] T 4de41396cffd4d32b86c701b538d1e01 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.583822 1174 raft_consensus.cc:740] T 4de41396cffd4d32b86c701b538d1e01 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.584216 1174 consensus_queue.cc:260] T 4de41396cffd4d32b86c701b538d1e01 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.588544 1174 ts_tablet_manager.cc:1434] T 4de41396cffd4d32b86c701b538d1e01 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260202 02:52:15.592852 1174 tablet_bootstrap.cc:492] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:15.594905 1174 tablet_bootstrap.cc:654] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.561683 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4de41396cffd4d32b86c701b538d1e01" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:15.598446 859 leader_election.cc:343] T 4de41396cffd4d32b86c701b538d1e01 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:15.598606 859 leader_election.cc:304] T 4de41396cffd4d32b86c701b538d1e01 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:15.599031 1269 raft_consensus.cc:2749] T 4de41396cffd4d32b86c701b538d1e01 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:15.584554 1037 tablet_service.cc:1552] Processing DeleteTablet for tablet 81f35df38f284f0ca5c3791f3d9d9293 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:15.600867 1175 tablet_bootstrap.cc:492] T 672ea3f731ba42ff8b1ff47db039ed22 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:15.601061 1175 ts_tablet_manager.cc:1403] T 672ea3f731ba42ff8b1ff47db039ed22 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.028s user 0.004s sys 0.016s
I20260202 02:52:15.601959 1175 raft_consensus.cc:359] T 672ea3f731ba42ff8b1ff47db039ed22 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.602176 1175 raft_consensus.cc:385] T 672ea3f731ba42ff8b1ff47db039ed22 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.602281 1175 raft_consensus.cc:740] T 672ea3f731ba42ff8b1ff47db039ed22 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.602672 1175 consensus_queue.cc:260] T 672ea3f731ba42ff8b1ff47db039ed22 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.603590 1175 ts_tablet_manager.cc:1434] T 672ea3f731ba42ff8b1ff47db039ed22 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260202 02:52:15.604436 1175 tablet_bootstrap.cc:492] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:15.604357 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_14ef599b5af14b839c634ea81482dd18"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:15.604979 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_14ef599b5af14b839c634ea81482dd18 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:15.606544 528 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_e764fa5da91046ba8293586023f427d5" } new_table_name: "default.Table_22489ecf56264d93993407ba7d0e6b55" modify_external_catalogs: true
I20260202 02:52:15.607198 1175 tablet_bootstrap.cc:654] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:15.607939 523 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet aa526a69cf914dd08142074636c956b7 consensus state has no leader
I20260202 02:52:15.607992 1174 tablet_bootstrap.cc:492] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:15.608165 1174 ts_tablet_manager.cc:1403] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.006s
I20260202 02:52:15.608222 523 master_service.cc:954] ReplaceTablet: received request to replace tablet add934bf70ae4de8a40066f06fd94795 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:15.608779 525 master_service.cc:954] ReplaceTablet: received request to replace tablet 4de41396cffd4d32b86c701b538d1e01 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:15.609128 1174 raft_consensus.cc:359] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.609341 1174 raft_consensus.cc:385] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.609426 1174 raft_consensus.cc:740] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.609709 1174 consensus_queue.cc:260] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.611945 523 master_service.cc:954] ReplaceTablet: received request to replace tablet 13309b20a96240b5aa9fbb691cbed981 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:15.615221 1175 tablet_bootstrap.cc:492] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:15.615422 1175 ts_tablet_manager.cc:1403] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.007s
I20260202 02:52:15.615869 1174 ts_tablet_manager.cc:1434] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260202 02:52:15.616225 1175 raft_consensus.cc:359] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.616474 1175 raft_consensus.cc:385] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.616559 1175 raft_consensus.cc:740] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.616807 1175 consensus_queue.cc:260] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.617619 1175 ts_tablet_manager.cc:1434] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:15.617957 1175 tablet_bootstrap.cc:492] T 251455168e33430aafea8e0de42ca1b8 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
W20260202 02:52:15.618448 527 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f24c0b99574d437a99b273204f105460 consensus state has no leader
I20260202 02:52:15.620178 1174 tablet_bootstrap.cc:492] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:15.620178 1175 tablet_bootstrap.cc:654] T 251455168e33430aafea8e0de42ca1b8 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.622324 1174 tablet_bootstrap.cc:654] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.593287 1183 ts_tablet_manager.cc:1929] T 2730d8b6812e487ea0aff4da45d24044 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:15.623857 1183 log.cc:1199] T 2730d8b6812e487ea0aff4da45d24044 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/2730d8b6812e487ea0aff4da45d24044
I20260202 02:52:15.624341 1183 ts_tablet_manager.cc:1950] T 2730d8b6812e487ea0aff4da45d24044 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:15.626546 1183 tablet_replica.cc:333] T 6399b63cd8974ee49796f8779f9abc0e P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:15.626827 1183 raft_consensus.cc:2243] T 6399b63cd8974ee49796f8779f9abc0e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:15.627094 1183 raft_consensus.cc:2272] T 6399b63cd8974ee49796f8779f9abc0e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:15.627947 1183 ts_tablet_manager.cc:1916] T 6399b63cd8974ee49796f8779f9abc0e P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:15.630404 1175 tablet_bootstrap.cc:492] T 251455168e33430aafea8e0de42ca1b8 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:15.630607 1175 ts_tablet_manager.cc:1403] T 251455168e33430aafea8e0de42ca1b8 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260202 02:52:15.631510 1175 raft_consensus.cc:359] T 251455168e33430aafea8e0de42ca1b8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.631742 1175 raft_consensus.cc:385] T 251455168e33430aafea8e0de42ca1b8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.631839 1175 raft_consensus.cc:740] T 251455168e33430aafea8e0de42ca1b8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.632186 1175 consensus_queue.cc:260] T 251455168e33430aafea8e0de42ca1b8 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.633792 1175 ts_tablet_manager.cc:1434] T 251455168e33430aafea8e0de42ca1b8 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260202 02:52:15.635524 1175 tablet_bootstrap.cc:492] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:15.636695 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 2730d8b6812e487ea0aff4da45d24044 (table default.Table_f1e417bf19b94bc491c3937276c669fd [id=9a8a252de32848288c9b3752c7d3feb7]) successfully deleted
W20260202 02:52:15.637115 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 4de41396cffd4d32b86c701b538d1e01 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4de41396cffd4d32b86c701b538d1e01 already in progress: creating tablet
I20260202 02:52:15.637446 1175 tablet_bootstrap.cc:654] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:15.639117 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 575016c0e9f84e99b962c5f14ffa96cc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 575016c0e9f84e99b962c5f14ffa96cc already in progress: creating tablet
W20260202 02:52:15.639495 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 58f09c607cce4ccc82afd75c002ed1a8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 58f09c607cce4ccc82afd75c002ed1a8 already in progress: creating tablet
I20260202 02:52:15.639822 1174 tablet_bootstrap.cc:492] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:15.639935 1175 tablet_bootstrap.cc:492] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:15.640000 1174 ts_tablet_manager.cc:1403] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.020s user 0.009s sys 0.008s
I20260202 02:52:15.640151 1175 ts_tablet_manager.cc:1403] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260202 02:52:15.641062 1175 raft_consensus.cc:359] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.641458 1174 raft_consensus.cc:359] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.641635 1174 raft_consensus.cc:385] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.641705 1174 raft_consensus.cc:740] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.641986 1174 consensus_queue.cc:260] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.641263 1175 raft_consensus.cc:385] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.642452 1175 raft_consensus.cc:740] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.642735 1175 consensus_queue.cc:260] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.644155 1175 ts_tablet_manager.cc:1434] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260202 02:52:15.646060 1175 tablet_bootstrap.cc:492] T f24c0b99574d437a99b273204f105460 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:15.648094 1175 tablet_bootstrap.cc:654] T f24c0b99574d437a99b273204f105460 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.650210 1174 ts_tablet_manager.cc:1434] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.010s user 0.000s sys 0.002s
I20260202 02:52:15.652678 1174 tablet_bootstrap.cc:492] T 13309b20a96240b5aa9fbb691cbed981 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:15.654996 1174 tablet_bootstrap.cc:654] T 13309b20a96240b5aa9fbb691cbed981 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.656270 1175 tablet_bootstrap.cc:492] T f24c0b99574d437a99b273204f105460 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:15.656482 1175 ts_tablet_manager.cc:1403] T f24c0b99574d437a99b273204f105460 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.001s
I20260202 02:52:15.657593 1175 raft_consensus.cc:359] T f24c0b99574d437a99b273204f105460 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.657820 1175 raft_consensus.cc:385] T f24c0b99574d437a99b273204f105460 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.657912 1175 raft_consensus.cc:740] T f24c0b99574d437a99b273204f105460 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.658167 1175 consensus_queue.cc:260] T f24c0b99574d437a99b273204f105460 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.659686 1175 ts_tablet_manager.cc:1434] T f24c0b99574d437a99b273204f105460 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260202 02:52:15.660699 1175 tablet_bootstrap.cc:492] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:15.663153 1175 tablet_bootstrap.cc:654] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.666744 1173 tablet_bootstrap.cc:492] T 7e81d5780d9647b9baac0066282376ea P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:15.667744 1173 ts_tablet_manager.cc:1403] T 7e81d5780d9647b9baac0066282376ea P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.119s user 0.003s sys 0.001s
I20260202 02:52:15.668823 1173 raft_consensus.cc:359] T 7e81d5780d9647b9baac0066282376ea P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.669118 1173 raft_consensus.cc:385] T 7e81d5780d9647b9baac0066282376ea P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.669236 1173 raft_consensus.cc:740] T 7e81d5780d9647b9baac0066282376ea P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.669582 1173 consensus_queue.cc:260] T 7e81d5780d9647b9baac0066282376ea P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.670482 1174 tablet_bootstrap.cc:492] T 13309b20a96240b5aa9fbb691cbed981 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:15.670614 1175 tablet_bootstrap.cc:492] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:15.670746 1174 ts_tablet_manager.cc:1403] T 13309b20a96240b5aa9fbb691cbed981 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.008s
I20260202 02:52:15.670776 1175 ts_tablet_manager.cc:1403] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260202 02:52:15.671571 1175 raft_consensus.cc:359] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.671593 1174 raft_consensus.cc:359] T 13309b20a96240b5aa9fbb691cbed981 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.671800 1175 raft_consensus.cc:385] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.671877 1175 raft_consensus.cc:740] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.671917 1174 raft_consensus.cc:385] T 13309b20a96240b5aa9fbb691cbed981 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.672102 1174 raft_consensus.cc:740] T 13309b20a96240b5aa9fbb691cbed981 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.672185 1175 consensus_queue.cc:260] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.672462 1174 consensus_queue.cc:260] T 13309b20a96240b5aa9fbb691cbed981 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.673444 1175 ts_tablet_manager.cc:1434] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260202 02:52:15.674739 1174 ts_tablet_manager.cc:1434] T 13309b20a96240b5aa9fbb691cbed981 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260202 02:52:15.674741 1175 tablet_bootstrap.cc:492] T c050a2882f764dda8df2a7df6547f149 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:15.677031 1174 tablet_bootstrap.cc:492] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:15.677145 1175 tablet_bootstrap.cc:654] T c050a2882f764dda8df2a7df6547f149 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.679906 1174 tablet_bootstrap.cc:654] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:15.680541 514 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet aa526a69cf914dd08142074636c956b7 (table default.Table_9451b9f5148f4d9aab65983dea967e32 [id=9ef504467a544f299b48499e49adf27f], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet aa526a69cf914dd08142074636c956b7 consensus state has no leader
I20260202 02:52:15.681876 1173 ts_tablet_manager.cc:1434] T 7e81d5780d9647b9baac0066282376ea P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.014s user 0.002s sys 0.000s
I20260202 02:52:15.682497 1175 tablet_bootstrap.cc:492] T c050a2882f764dda8df2a7df6547f149 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:15.682662 1175 ts_tablet_manager.cc:1403] T c050a2882f764dda8df2a7df6547f149 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260202 02:52:15.683763 1175 raft_consensus.cc:359] T c050a2882f764dda8df2a7df6547f149 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.683975 1175 raft_consensus.cc:385] T c050a2882f764dda8df2a7df6547f149 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.684075 1175 raft_consensus.cc:740] T c050a2882f764dda8df2a7df6547f149 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.684319 1175 consensus_queue.cc:260] T c050a2882f764dda8df2a7df6547f149 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.685889 1175 ts_tablet_manager.cc:1434] T c050a2882f764dda8df2a7df6547f149 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260202 02:52:15.685830 522 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_34c95b3937dd4587a9bd81a7c805f562" } modify_external_catalogs: true
I20260202 02:52:15.686033 522 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_34c95b3937dd4587a9bd81a7c805f562" } modify_external_catalogs: true
I20260202 02:52:15.686214 1175 tablet_bootstrap.cc:492] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:15.686437 1174 tablet_bootstrap.cc:492] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:15.686611 1174 ts_tablet_manager.cc:1403] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
W20260202 02:52:15.687052 528 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 415d14533fd54f20adb00455a7bcad5f has no consensus state
I20260202 02:52:15.687510 521 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_9bac792e43464105846e5482edbccacc" } modify_external_catalogs: true
I20260202 02:52:15.687654 521 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_9bac792e43464105846e5482edbccacc" } modify_external_catalogs: true
I20260202 02:52:15.688042 1174 raft_consensus.cc:359] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.688246 1174 raft_consensus.cc:385] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.688310 1174 raft_consensus.cc:740] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.688608 1174 consensus_queue.cc:260] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.688863 1175 tablet_bootstrap.cc:654] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.689283 1037 tablet_service.cc:1552] Processing DeleteTablet for tablet 7e81d5780d9647b9baac0066282376ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:15.691069 1174 ts_tablet_manager.cc:1434] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260202 02:52:15.693461 1174 tablet_bootstrap.cc:492] T fc93352b24e345f99d15085e4886c748 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:15.695467 1174 tablet_bootstrap.cc:654] T fc93352b24e345f99d15085e4886c748 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.696740 524 catalog_manager.cc:5654] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130). New cstate: current_term: 1 leader_uuid: "1521a6d262b7451bbef8debe970dd6b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } }
I20260202 02:52:15.697620 524 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 80eb3013f5354022a2dd78bb294a8497 (table default.Table_25c6d490f243450ab2d9b43ec9cda7bf [id=316b8a276d9b415fafe8511aa7c32fc5]). Expected version 2 got 0
I20260202 02:52:15.697868 1175 tablet_bootstrap.cc:492] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:15.698170 1175 ts_tablet_manager.cc:1403] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
W20260202 02:52:15.698745 519 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 938813ee0a8a40b6bf0e537e3ec1a0d4 consensus state has no leader
I20260202 02:52:15.699424 1178 raft_consensus.cc:493] T 81f35df38f284f0ca5c3791f3d9d9293 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:15.699374 1175 raft_consensus.cc:359] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.699606 1175 raft_consensus.cc:385] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.699674 1175 raft_consensus.cc:740] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.699599 1178 raft_consensus.cc:515] T 81f35df38f284f0ca5c3791f3d9d9293 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.699973 1175 consensus_queue.cc:260] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.700258 1178 leader_election.cc:290] T 81f35df38f284f0ca5c3791f3d9d9293 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:15.702165 999 leader_election.cc:343] T 81f35df38f284f0ca5c3791f3d9d9293 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 81f35df38f284f0ca5c3791f3d9d9293
I20260202 02:52:15.702283 1175 ts_tablet_manager.cc:1434] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
W20260202 02:52:15.702382 998 leader_election.cc:343] T 81f35df38f284f0ca5c3791f3d9d9293 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Not found: Tablet not found: 81f35df38f284f0ca5c3791f3d9d9293
I20260202 02:52:15.702507 998 leader_election.cc:304] T 81f35df38f284f0ca5c3791f3d9d9293 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:15.703374 1178 raft_consensus.cc:2749] T 81f35df38f284f0ca5c3791f3d9d9293 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:15.703449 1175 tablet_bootstrap.cc:492] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:15.705264 1174 tablet_bootstrap.cc:492] T fc93352b24e345f99d15085e4886c748 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:15.705389 1175 tablet_bootstrap.cc:654] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.705432 1174 ts_tablet_manager.cc:1403] T fc93352b24e345f99d15085e4886c748 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.002s
I20260202 02:52:15.706276 1174 raft_consensus.cc:359] T fc93352b24e345f99d15085e4886c748 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.706578 1174 raft_consensus.cc:385] T fc93352b24e345f99d15085e4886c748 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.707420 1174 raft_consensus.cc:740] T fc93352b24e345f99d15085e4886c748 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.707728 1174 consensus_queue.cc:260] T fc93352b24e345f99d15085e4886c748 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.708657 526 catalog_manager.cc:5654] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f442ef85029461091de847118c34c0d (127.0.104.129). New cstate: current_term: 1 leader_uuid: "1f442ef85029461091de847118c34c0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } }
I20260202 02:52:15.709117 526 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet fae7e3941c124cc2871390a0b6471f27 (table default.Table_3d26f7a34d0c48379e3cebda66e1ef13 [id=45e6d83589fa493195b06f633e295b00]). Expected version 2 got 0
I20260202 02:52:15.710538 1174 ts_tablet_manager.cc:1434] T fc93352b24e345f99d15085e4886c748 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260202 02:52:15.713210 1174 tablet_bootstrap.cc:492] T 575016c0e9f84e99b962c5f14ffa96cc P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:15.713559 1037 tablet_service.cc:1505] Processing CreateTablet for tablet ea040c03420c4b1b98897f39160b4130 (DEFAULT_TABLE table=default.Table_9bac792e43464105846e5482edbccacc [id=0a3c23955dde47d9982c4659d9a73e30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.714063 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea040c03420c4b1b98897f39160b4130. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.716794 519 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_b6c1fb1e7282492d920385869810b0e0" } modify_external_catalogs: true
I20260202 02:52:15.717056 1174 tablet_bootstrap.cc:654] T 575016c0e9f84e99b962c5f14ffa96cc P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.717026 519 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_b6c1fb1e7282492d920385869810b0e0" } modify_external_catalogs: true
I20260202 02:52:15.718729 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_9232ab3d803b45a8b491947852fc8a98"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:15.719230 1175 tablet_bootstrap.cc:492] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
W20260202 02:52:15.719288 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9232ab3d803b45a8b491947852fc8a98 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:15.719484 1269 raft_consensus.cc:493] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:15.719617 1269 raft_consensus.cc:515] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.720396 1269 leader_election.cc:290] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:15.721145 527 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_53eaadb859904d6b95ef0e536019ad79" } modify_external_catalogs: true
I20260202 02:52:15.721280 1175 ts_tablet_manager.cc:1403] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.016s user 0.012s sys 0.003s
I20260202 02:52:15.721341 527 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_53eaadb859904d6b95ef0e536019ad79" } modify_external_catalogs: true
I20260202 02:52:15.721340 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "938813ee0a8a40b6bf0e537e3ec1a0d4" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:15.722129 777 raft_consensus.cc:2468] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
I20260202 02:52:15.722124 1175 raft_consensus.cc:359] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.722292 1175 raft_consensus.cc:385] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.722640 857 leader_election.cc:304] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters:
I20260202 02:52:15.722892 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "938813ee0a8a40b6bf0e537e3ec1a0d4" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:15.722970 1269 raft_consensus.cc:2804] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:15.723086 1269 raft_consensus.cc:493] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:15.723167 1269 raft_consensus.cc:3060] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:15.723711 1174 tablet_bootstrap.cc:492] T 575016c0e9f84e99b962c5f14ffa96cc P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:15.723887 1174 ts_tablet_manager.cc:1403] T 575016c0e9f84e99b962c5f14ffa96cc P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260202 02:52:15.723925 898 tablet_service.cc:1505] Processing CreateTablet for tablet ea040c03420c4b1b98897f39160b4130 (DEFAULT_TABLE table=default.Table_9bac792e43464105846e5482edbccacc [id=0a3c23955dde47d9982c4659d9a73e30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.724401 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea040c03420c4b1b98897f39160b4130. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.724684 1174 raft_consensus.cc:359] T 575016c0e9f84e99b962c5f14ffa96cc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.724891 1174 raft_consensus.cc:385] T 575016c0e9f84e99b962c5f14ffa96cc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.724958 1174 raft_consensus.cc:740] T 575016c0e9f84e99b962c5f14ffa96cc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.725247 1174 consensus_queue.cc:260] T 575016c0e9f84e99b962c5f14ffa96cc P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.726037 1174 ts_tablet_manager.cc:1434] T 575016c0e9f84e99b962c5f14ffa96cc P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:15.725935 1269 raft_consensus.cc:515] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.726342 1174 tablet_bootstrap.cc:492] T add934bf70ae4de8a40066f06fd94795 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
W20260202 02:52:15.727444 859 leader_election.cc:343] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:15.727458 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "938813ee0a8a40b6bf0e537e3ec1a0d4" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:15.727655 777 raft_consensus.cc:3060] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:15.728237 1269 leader_election.cc:290] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:15.728628 1175 raft_consensus.cc:740] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.728904 1175 consensus_queue.cc:260] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.729789 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "938813ee0a8a40b6bf0e537e3ec1a0d4" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:15.730199 1175 ts_tablet_manager.cc:1434] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260202 02:52:15.730237 777 raft_consensus.cc:2468] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
W20260202 02:52:15.730376 859 leader_election.cc:343] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:15.730705 857 leader_election.cc:304] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:15.732002 1269 raft_consensus.cc:2804] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:15.732172 1269 raft_consensus.cc:697] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:15.732463 1269 consensus_queue.cc:237] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.732753 1174 tablet_bootstrap.cc:654] T add934bf70ae4de8a40066f06fd94795 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.735054 1175 tablet_bootstrap.cc:492] T c0937b6910534ddc8a2217f0cbd539c5 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:15.737064 1175 tablet_bootstrap.cc:654] T c0937b6910534ddc8a2217f0cbd539c5 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.744087 751 tablet_service.cc:1505] Processing CreateTablet for tablet ea040c03420c4b1b98897f39160b4130 (DEFAULT_TABLE table=default.Table_9bac792e43464105846e5482edbccacc [id=0a3c23955dde47d9982c4659d9a73e30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.744643 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea040c03420c4b1b98897f39160b4130. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.745689 1175 tablet_bootstrap.cc:492] T c0937b6910534ddc8a2217f0cbd539c5 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:15.745877 1175 ts_tablet_manager.cc:1403] T c0937b6910534ddc8a2217f0cbd539c5 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.007s
I20260202 02:52:15.746778 1175 raft_consensus.cc:359] T c0937b6910534ddc8a2217f0cbd539c5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.747013 1175 raft_consensus.cc:385] T c0937b6910534ddc8a2217f0cbd539c5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.747116 1175 raft_consensus.cc:740] T c0937b6910534ddc8a2217f0cbd539c5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.747372 1175 consensus_queue.cc:260] T c0937b6910534ddc8a2217f0cbd539c5 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.748939 1175 ts_tablet_manager.cc:1434] T c0937b6910534ddc8a2217f0cbd539c5 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260202 02:52:15.750278 1175 tablet_bootstrap.cc:492] T c1fc01252d95407e954cfcbf7e715912 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:15.750464 1178 raft_consensus.cc:493] T 7e81d5780d9647b9baac0066282376ea P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:15.750685 1178 raft_consensus.cc:515] T 7e81d5780d9647b9baac0066282376ea P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.751358 1178 leader_election.cc:290] T 7e81d5780d9647b9baac0066282376ea P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:15.752216 999 leader_election.cc:343] T 7e81d5780d9647b9baac0066282376ea P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 7e81d5780d9647b9baac0066282376ea
W20260202 02:52:15.752403 998 leader_election.cc:343] T 7e81d5780d9647b9baac0066282376ea P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Not found: Tablet not found: 7e81d5780d9647b9baac0066282376ea
I20260202 02:52:15.752513 998 leader_election.cc:304] T 7e81d5780d9647b9baac0066282376ea P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:15.752877 1175 tablet_bootstrap.cc:654] T c1fc01252d95407e954cfcbf7e715912 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.752957 1178 raft_consensus.cc:2749] T 7e81d5780d9647b9baac0066282376ea P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:15.758932 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 724d47241f0342768fb1321ef37c3a7b (DEFAULT_TABLE table=default.Table_5c9c0f09396a4a0e8b3d676fcb2dbb2c [id=6a47f828bd9c4d7aa7d385e4a1dda9ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.759516 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 724d47241f0342768fb1321ef37c3a7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.761492 1175 tablet_bootstrap.cc:492] T c1fc01252d95407e954cfcbf7e715912 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:15.761678 1175 ts_tablet_manager.cc:1403] T c1fc01252d95407e954cfcbf7e715912 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260202 02:52:15.762620 1175 raft_consensus.cc:359] T c1fc01252d95407e954cfcbf7e715912 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.762837 1175 raft_consensus.cc:385] T c1fc01252d95407e954cfcbf7e715912 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.762926 1175 raft_consensus.cc:740] T c1fc01252d95407e954cfcbf7e715912 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.763242 1175 consensus_queue.cc:260] T c1fc01252d95407e954cfcbf7e715912 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.764952 1175 ts_tablet_manager.cc:1434] T c1fc01252d95407e954cfcbf7e715912 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260202 02:52:15.766242 1175 tablet_bootstrap.cc:492] T 952d760159844ad08269654264e6381b P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:15.768990 1175 tablet_bootstrap.cc:654] T 952d760159844ad08269654264e6381b P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.778553 1175 tablet_bootstrap.cc:492] T 952d760159844ad08269654264e6381b P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:15.778770 1175 ts_tablet_manager.cc:1403] T 952d760159844ad08269654264e6381b P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.005s
I20260202 02:52:15.779397 1173 tablet_bootstrap.cc:492] T e6b76967e5d3476997fa019c4a8afb91 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:15.779628 1175 raft_consensus.cc:359] T 952d760159844ad08269654264e6381b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.779861 1175 raft_consensus.cc:385] T 952d760159844ad08269654264e6381b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.779942 1175 raft_consensus.cc:740] T 952d760159844ad08269654264e6381b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.780222 1175 consensus_queue.cc:260] T 952d760159844ad08269654264e6381b P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.781581 1175 ts_tablet_manager.cc:1434] T 952d760159844ad08269654264e6381b P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260202 02:52:15.781744 1173 tablet_bootstrap.cc:654] T e6b76967e5d3476997fa019c4a8afb91 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.781948 1175 tablet_bootstrap.cc:492] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:15.785355 1175 tablet_bootstrap.cc:654] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.789176 523 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 13309b20a96240b5aa9fbb691cbed981
I20260202 02:52:15.789758 523 catalog_manager.cc:6724] ReplaceTablet: tablet 13309b20a96240b5aa9fbb691cbed981 deleted and replaced by tablet c064ea0dade840b4840a5715c2d83f69
I20260202 02:52:15.791435 523 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_f1e417bf19b94bc491c3937276c669fd" } modify_external_catalogs: true
I20260202 02:52:15.791646 523 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_f1e417bf19b94bc491c3937276c669fd" } modify_external_catalogs: true
I20260202 02:52:15.795379 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet 13309b20a96240b5aa9fbb691cbed981 with delete_type TABLET_DATA_DELETED (replaced by tablet c064ea0dade840b4840a5715c2d83f69) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:15.796896 525 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_14ef599b5af14b839c634ea81482dd18" } new_table_name: "default.Table_42ecaddd58b64765aef12dda7d1e0fcc" modify_external_catalogs: true
I20260202 02:52:15.797562 1175 tablet_bootstrap.cc:492] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:15.797736 1175 ts_tablet_manager.cc:1403] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.004s
I20260202 02:52:15.798702 1175 raft_consensus.cc:359] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.798910 1175 raft_consensus.cc:385] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.798995 1175 raft_consensus.cc:740] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.799230 1175 consensus_queue.cc:260] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.800143 1175 ts_tablet_manager.cc:1434] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:15.800505 1175 tablet_bootstrap.cc:492] T 5986676f5484498da927b4c6da3f47ff P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:15.802083 524 master_service.cc:954] ReplaceTablet: received request to replace tablet aa526a69cf914dd08142074636c956b7 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:15.804162 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 13309b20a96240b5aa9fbb691cbed981 with delete_type TABLET_DATA_DELETED (replaced by tablet c064ea0dade840b4840a5715c2d83f69) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:15.804512 903 tablet_service.cc:1552] Processing DeleteTablet for tablet 72475ea720c6429aa68de9478b374280 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:15.804591 1180 tablet_replica.cc:333] T 13309b20a96240b5aa9fbb691cbed981 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:15.804705 901 tablet_service.cc:1552] Processing DeleteTablet for tablet 4de41396cffd4d32b86c701b538d1e01 with delete_type TABLET_DATA_DELETED (replaced by tablet bf74f96bea2e4ab7adec42c3c5207e79) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:15.804797 1182 tablet_replica.cc:333] T 72475ea720c6429aa68de9478b374280 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:15.804819 1180 raft_consensus.cc:2243] T 13309b20a96240b5aa9fbb691cbed981 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:15.804999 1180 raft_consensus.cc:2272] T 13309b20a96240b5aa9fbb691cbed981 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:15.805008 1182 raft_consensus.cc:2243] T 72475ea720c6429aa68de9478b374280 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:15.805195 1182 raft_consensus.cc:2272] T 72475ea720c6429aa68de9478b374280 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:15.805449 903 tablet_service.cc:1552] Processing DeleteTablet for tablet e6b76967e5d3476997fa019c4a8afb91 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:15.805811 1180 ts_tablet_manager.cc:1916] T 13309b20a96240b5aa9fbb691cbed981 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:15.806490 1175 tablet_bootstrap.cc:654] T 5986676f5484498da927b4c6da3f47ff P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.809124 1182 ts_tablet_manager.cc:1916] T 72475ea720c6429aa68de9478b374280 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:15.809077 1246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260202 02:52:15.809495 1246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260202 02:52:15.809850 1246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260202 02:52:15.814736 1175 tablet_bootstrap.cc:492] T 5986676f5484498da927b4c6da3f47ff P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:15.814958 1175 ts_tablet_manager.cc:1403] T 5986676f5484498da927b4c6da3f47ff P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.003s
I20260202 02:52:15.815805 1175 raft_consensus.cc:359] T 5986676f5484498da927b4c6da3f47ff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.816017 1175 raft_consensus.cc:385] T 5986676f5484498da927b4c6da3f47ff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.816115 1175 raft_consensus.cc:740] T 5986676f5484498da927b4c6da3f47ff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.816398 1175 consensus_queue.cc:260] T 5986676f5484498da927b4c6da3f47ff P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.817842 1175 ts_tablet_manager.cc:1434] T 5986676f5484498da927b4c6da3f47ff P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260202 02:52:15.818889 1175 tablet_bootstrap.cc:492] T 0d297b6243c348ee82c146a8339cf6c4 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:15.821225 1180 ts_tablet_manager.cc:1929] T 13309b20a96240b5aa9fbb691cbed981 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:15.821506 1180 log.cc:1199] T 13309b20a96240b5aa9fbb691cbed981 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/13309b20a96240b5aa9fbb691cbed981
I20260202 02:52:15.822654 1182 ts_tablet_manager.cc:1929] T 72475ea720c6429aa68de9478b374280 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:15.822804 1182 log.cc:1199] T 72475ea720c6429aa68de9478b374280 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/72475ea720c6429aa68de9478b374280
I20260202 02:52:15.823226 1180 ts_tablet_manager.cc:1950] T 13309b20a96240b5aa9fbb691cbed981 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:15.823266 1182 ts_tablet_manager.cc:1950] T 72475ea720c6429aa68de9478b374280 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:15.824455 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 72475ea720c6429aa68de9478b374280 (table default.Table_09d8f825c4c546839c0a820f1ddf0047 [id=ce957713e7fb49928d3fe57c3682f26f]) successfully deleted
I20260202 02:52:15.824525 1182 tablet_replica.cc:333] T 4de41396cffd4d32b86c701b538d1e01 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:15.824746 1182 raft_consensus.cc:2243] T 4de41396cffd4d32b86c701b538d1e01 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:15.824751 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 13309b20a96240b5aa9fbb691cbed981 (table default.Table_5fa327d665484811934aa3b11188a76d [id=f7c5b24c065443d89a2b1f9c34409c9d]) successfully deleted
I20260202 02:52:15.824949 1182 raft_consensus.cc:2272] T 4de41396cffd4d32b86c701b538d1e01 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:15.826092 1246 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260202 02:52:15.826207 1246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260202 02:52:15.826288 1246 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260202 02:52:15.826352 1246 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260202 02:52:15.827105 1175 tablet_bootstrap.cc:654] T 0d297b6243c348ee82c146a8339cf6c4 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.828866 1182 ts_tablet_manager.cc:1916] T 4de41396cffd4d32b86c701b538d1e01 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:15.831014 903 tablet_service.cc:1505] Processing CreateTablet for tablet 724d47241f0342768fb1321ef37c3a7b (DEFAULT_TABLE table=default.Table_5c9c0f09396a4a0e8b3d676fcb2dbb2c [id=6a47f828bd9c4d7aa7d385e4a1dda9ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.831535 903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 724d47241f0342768fb1321ef37c3a7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.832710 751 tablet_service.cc:1505] Processing CreateTablet for tablet 724d47241f0342768fb1321ef37c3a7b (DEFAULT_TABLE table=default.Table_5c9c0f09396a4a0e8b3d676fcb2dbb2c [id=6a47f828bd9c4d7aa7d385e4a1dda9ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.833143 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 724d47241f0342768fb1321ef37c3a7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.837397 1041 tablet_service.cc:1505] Processing CreateTablet for tablet bd7b0a0fcb7645fdb9edfeccb3d18268 (DEFAULT_TABLE table=default.Table_7ef6ef5b916e42709217b7c44fb489d0 [id=1ffcc11561524fe8a7b09a7ddc92d5b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.837944 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd7b0a0fcb7645fdb9edfeccb3d18268. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.841238 1182 ts_tablet_manager.cc:1929] T 4de41396cffd4d32b86c701b538d1e01 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:15.841419 1182 log.cc:1199] T 4de41396cffd4d32b86c701b538d1e01 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/4de41396cffd4d32b86c701b538d1e01
I20260202 02:52:15.841899 1182 ts_tablet_manager.cc:1950] T 4de41396cffd4d32b86c701b538d1e01 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:15.843185 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 4de41396cffd4d32b86c701b538d1e01 (table default.Table_d142ff2cb10e491997bf0e302d6a9fa5 [id=a0b8442c02974d6596db002c9292be0e]) successfully deleted
W20260202 02:52:15.843528 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet e6b76967e5d3476997fa019c4a8afb91 because the tablet was not found. No further retry: Not found: Tablet not found: e6b76967e5d3476997fa019c4a8afb91
I20260202 02:52:15.845528 1246 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:45945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-2/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.190:33981,127.0.104.189:46271,127.0.104.188:39077
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.188:39077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.104.188
--webserver_port=35523
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3722ed49c19ce54c7feae2fa944e32b1d4ed8d2d
build type FASTDEBUG
built by None at 02 Feb 2026 02:43:23 UTC on 24a791456cd2
build id 10304
ASAN enabled
I20260202 02:52:15.847947 1246 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260202 02:52:15.850525 1246 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20260202 02:52:15.851058 898 tablet_service.cc:1552] Processing DeleteTablet for tablet 13309b20a96240b5aa9fbb691cbed981 with delete_type TABLET_DATA_DELETED (replaced by tablet c064ea0dade840b4840a5715c2d83f69) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:15.851490 1182 tablet_replica.cc:333] T 13309b20a96240b5aa9fbb691cbed981 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:15.851774 1182 raft_consensus.cc:2243] T 13309b20a96240b5aa9fbb691cbed981 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:15.852030 1182 raft_consensus.cc:2272] T 13309b20a96240b5aa9fbb691cbed981 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:15.853055 1182 ts_tablet_manager.cc:1916] T 13309b20a96240b5aa9fbb691cbed981 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:15.860153 522 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet bf5ca99708044482970cb854d2eced18
I20260202 02:52:15.862321 898 tablet_service.cc:1505] Processing CreateTablet for tablet bd7b0a0fcb7645fdb9edfeccb3d18268 (DEFAULT_TABLE table=default.Table_7ef6ef5b916e42709217b7c44fb489d0 [id=1ffcc11561524fe8a7b09a7ddc92d5b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.863157 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd7b0a0fcb7645fdb9edfeccb3d18268. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.864843 754 tablet_service.cc:1552] Processing DeleteTablet for tablet bf5ca99708044482970cb854d2eced18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:15.864972 756 tablet_service.cc:1505] Processing CreateTablet for tablet bd7b0a0fcb7645fdb9edfeccb3d18268 (DEFAULT_TABLE table=default.Table_7ef6ef5b916e42709217b7c44fb489d0 [id=1ffcc11561524fe8a7b09a7ddc92d5b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.865397 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd7b0a0fcb7645fdb9edfeccb3d18268. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.866369 1044 tablet_service.cc:1505] Processing CreateTablet for tablet bf5ca99708044482970cb854d2eced18 (DEFAULT_TABLE table=default.Table_34c95b3937dd4587a9bd81a7c805f562 [id=8227929c83a441af86babfb3cff579bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.866812 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf5ca99708044482970cb854d2eced18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.868141 903 tablet_service.cc:1552] Processing DeleteTablet for tablet bf5ca99708044482970cb854d2eced18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:15.868570 1182 ts_tablet_manager.cc:1929] T 13309b20a96240b5aa9fbb691cbed981 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:15.868703 1182 log.cc:1199] T 13309b20a96240b5aa9fbb691cbed981 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/13309b20a96240b5aa9fbb691cbed981
I20260202 02:52:15.868669 749 tablet_service.cc:1505] Processing CreateTablet for tablet bf5ca99708044482970cb854d2eced18 (DEFAULT_TABLE table=default.Table_34c95b3937dd4587a9bd81a7c805f562 [id=8227929c83a441af86babfb3cff579bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.869055 1182 ts_tablet_manager.cc:1950] T 13309b20a96240b5aa9fbb691cbed981 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:15.869524 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf5ca99708044482970cb854d2eced18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.870398 521 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet ea040c03420c4b1b98897f39160b4130
I20260202 02:52:15.870841 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 13309b20a96240b5aa9fbb691cbed981 (table default.Table_5fa327d665484811934aa3b11188a76d [id=f7c5b24c065443d89a2b1f9c34409c9d]) successfully deleted
I20260202 02:52:15.871263 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_51cfbe7668e344b1bcca61f0ab935341"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:15.871616 903 tablet_service.cc:1552] Processing DeleteTablet for tablet ea040c03420c4b1b98897f39160b4130 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:15.871865 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_51cfbe7668e344b1bcca61f0ab935341 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:15.873178 520 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet f24c0b99574d437a99b273204f105460
I20260202 02:52:15.873643 520 catalog_manager.cc:6724] ReplaceTablet: tablet f24c0b99574d437a99b273204f105460 deleted and replaced by tablet bc1539e5c8aa4d6e98f687728b891530
I20260202 02:52:15.874006 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_ef11a1ba065d483c9f7862345ab1aedb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:15.874605 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ef11a1ba065d483c9f7862345ab1aedb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:15.875916 1039 tablet_service.cc:1552] Processing DeleteTablet for tablet ea040c03420c4b1b98897f39160b4130 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:15.876194 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet f24c0b99574d437a99b273204f105460 with delete_type TABLET_DATA_DELETED (replaced by tablet bc1539e5c8aa4d6e98f687728b891530) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:15.876849 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet bf5ca99708044482970cb854d2eced18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:15.877719 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_fe76707af2ff46d1b44c46cef9e88be4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:15.878311 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fe76707af2ff46d1b44c46cef9e88be4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:15.879997 513 catalog_manager.cc:4952] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet bf5ca99708044482970cb854d2eced18 because the tablet was not found. No further retry: Not found: Tablet not found: bf5ca99708044482970cb854d2eced18
I20260202 02:52:15.880254 754 tablet_service.cc:1552] Processing DeleteTablet for tablet ea040c03420c4b1b98897f39160b4130 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:15.880555 750 tablet_service.cc:1552] Processing DeleteTablet for tablet f24c0b99574d437a99b273204f105460 with delete_type TABLET_DATA_DELETED (replaced by tablet bc1539e5c8aa4d6e98f687728b891530) from {username='slave'} at 127.0.0.1:42950
W20260202 02:52:15.882283 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet ea040c03420c4b1b98897f39160b4130 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ea040c03420c4b1b98897f39160b4130 already in progress: creating tablet
W20260202 02:52:15.882683 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet f24c0b99574d437a99b273204f105460 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f24c0b99574d437a99b273204f105460 already in progress: creating tablet
W20260202 02:52:15.884053 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet bf5ca99708044482970cb854d2eced18 because the tablet was not found. No further retry: Not found: Tablet not found: bf5ca99708044482970cb854d2eced18
W20260202 02:52:15.884284 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet ea040c03420c4b1b98897f39160b4130 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ea040c03420c4b1b98897f39160b4130 already in progress: creating tablet
I20260202 02:52:15.884400 898 tablet_service.cc:1552] Processing DeleteTablet for tablet f24c0b99574d437a99b273204f105460 with delete_type TABLET_DATA_DELETED (replaced by tablet bc1539e5c8aa4d6e98f687728b891530) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:15.884735 1182 tablet_replica.cc:333] T f24c0b99574d437a99b273204f105460 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:15.885051 1182 raft_consensus.cc:2243] T f24c0b99574d437a99b273204f105460 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:15.885320 1182 raft_consensus.cc:2272] T f24c0b99574d437a99b273204f105460 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:15.886287 1182 ts_tablet_manager.cc:1916] T f24c0b99574d437a99b273204f105460 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:15.894874 1182 ts_tablet_manager.cc:1929] T f24c0b99574d437a99b273204f105460 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:15.895463 1269 raft_consensus.cc:493] T c1fc01252d95407e954cfcbf7e715912 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:15.897307 1269 raft_consensus.cc:515] T c1fc01252d95407e954cfcbf7e715912 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.898108 1269 leader_election.cc:290] T c1fc01252d95407e954cfcbf7e715912 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:15.899170 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c1fc01252d95407e954cfcbf7e715912" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:15.899535 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c1fc01252d95407e954cfcbf7e715912" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:15.900103 857 leader_election.cc:343] T c1fc01252d95407e954cfcbf7e715912 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:15.900478 859 leader_election.cc:343] T c1fc01252d95407e954cfcbf7e715912 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:15.900591 859 leader_election.cc:304] T c1fc01252d95407e954cfcbf7e715912 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:15.901069 1269 raft_consensus.cc:2749] T c1fc01252d95407e954cfcbf7e715912 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:15.899273 1182 log.cc:1199] T f24c0b99574d437a99b273204f105460 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/f24c0b99574d437a99b273204f105460
I20260202 02:52:15.901877 1182 ts_tablet_manager.cc:1950] T f24c0b99574d437a99b273204f105460 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:15.903738 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet f24c0b99574d437a99b273204f105460 (table default.Table_f1e417bf19b94bc491c3937276c669fd [id=9a8a252de32848288c9b3752c7d3feb7]) successfully deleted
I20260202 02:52:15.905062 1173 tablet_bootstrap.cc:492] T e6b76967e5d3476997fa019c4a8afb91 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:15.905241 1173 ts_tablet_manager.cc:1403] T e6b76967e5d3476997fa019c4a8afb91 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.126s user 0.003s sys 0.000s
I20260202 02:52:15.906137 1173 raft_consensus.cc:359] T e6b76967e5d3476997fa019c4a8afb91 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.906343 1173 raft_consensus.cc:385] T e6b76967e5d3476997fa019c4a8afb91 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.906442 1173 raft_consensus.cc:740] T e6b76967e5d3476997fa019c4a8afb91 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.906781 1173 consensus_queue.cc:260] T e6b76967e5d3476997fa019c4a8afb91 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.915010 898 tablet_service.cc:1505] Processing CreateTablet for tablet bf5ca99708044482970cb854d2eced18 (DEFAULT_TABLE table=default.Table_34c95b3937dd4587a9bd81a7c805f562 [id=8227929c83a441af86babfb3cff579bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.915645 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf5ca99708044482970cb854d2eced18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.926030 749 tablet_service.cc:1505] Processing CreateTablet for tablet cb0dcacdf0f1419da0b25f20cac1d62d (DEFAULT_TABLE table=default.Table_c245b83c1987454fabbe8646d14252d6 [id=08498a654b6a4a9ca3f968d70daad9b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.926420 1044 tablet_service.cc:1505] Processing CreateTablet for tablet cb0dcacdf0f1419da0b25f20cac1d62d (DEFAULT_TABLE table=default.Table_c245b83c1987454fabbe8646d14252d6 [id=08498a654b6a4a9ca3f968d70daad9b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.926537 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb0dcacdf0f1419da0b25f20cac1d62d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.926790 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb0dcacdf0f1419da0b25f20cac1d62d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.934310 898 tablet_service.cc:1505] Processing CreateTablet for tablet cb0dcacdf0f1419da0b25f20cac1d62d (DEFAULT_TABLE table=default.Table_c245b83c1987454fabbe8646d14252d6 [id=08498a654b6a4a9ca3f968d70daad9b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.934891 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb0dcacdf0f1419da0b25f20cac1d62d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.941454 1183 ts_tablet_manager.cc:1929] T 6399b63cd8974ee49796f8779f9abc0e P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:15.941610 1183 log.cc:1199] T 6399b63cd8974ee49796f8779f9abc0e P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/6399b63cd8974ee49796f8779f9abc0e
I20260202 02:52:15.941974 1183 ts_tablet_manager.cc:1950] T 6399b63cd8974ee49796f8779f9abc0e P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:15.944556 1041 tablet_service.cc:1505] Processing CreateTablet for tablet 560465c063d94ffaab33f9f79a58a357 (DEFAULT_TABLE table=default.Table_3b335a0806154f98a907f551596ba353 [id=0da4db61b3fe4cf4846edd5530b9327f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.944782 749 tablet_service.cc:1505] Processing CreateTablet for tablet 560465c063d94ffaab33f9f79a58a357 (DEFAULT_TABLE table=default.Table_3b335a0806154f98a907f551596ba353 [id=0da4db61b3fe4cf4846edd5530b9327f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.945024 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 560465c063d94ffaab33f9f79a58a357. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.945223 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 560465c063d94ffaab33f9f79a58a357. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.946822 1038 tablet_service.cc:1505] Processing CreateTablet for tablet bf74f96bea2e4ab7adec42c3c5207e79 (DEFAULT_TABLE table=default.Table_d142ff2cb10e491997bf0e302d6a9fa5 [id=a0b8442c02974d6596db002c9292be0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.947235 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf74f96bea2e4ab7adec42c3c5207e79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.952710 903 tablet_service.cc:1505] Processing CreateTablet for tablet 560465c063d94ffaab33f9f79a58a357 (DEFAULT_TABLE table=default.Table_3b335a0806154f98a907f551596ba353 [id=0da4db61b3fe4cf4846edd5530b9327f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.953280 903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 560465c063d94ffaab33f9f79a58a357. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.956841 901 tablet_service.cc:1505] Processing CreateTablet for tablet bf74f96bea2e4ab7adec42c3c5207e79 (DEFAULT_TABLE table=default.Table_d142ff2cb10e491997bf0e302d6a9fa5 [id=a0b8442c02974d6596db002c9292be0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.957347 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf74f96bea2e4ab7adec42c3c5207e79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.962052 1209 raft_consensus.cc:493] T 72475ea720c6429aa68de9478b374280 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:15.962273 1209 raft_consensus.cc:515] T 72475ea720c6429aa68de9478b374280 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.962963 1209 leader_election.cc:290] T 72475ea720c6429aa68de9478b374280 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:15.963634 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "72475ea720c6429aa68de9478b374280" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:15.964344 715 leader_election.cc:343] T 72475ea720c6429aa68de9478b374280 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:15.965425 714 leader_election.cc:343] T 72475ea720c6429aa68de9478b374280 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 72475ea720c6429aa68de9478b374280
I20260202 02:52:15.965546 714 leader_election.cc:304] T 72475ea720c6429aa68de9478b374280 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:15.966075 1209 raft_consensus.cc:2749] T 72475ea720c6429aa68de9478b374280 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:15.972429 749 tablet_service.cc:1505] Processing CreateTablet for tablet bf74f96bea2e4ab7adec42c3c5207e79 (DEFAULT_TABLE table=default.Table_d142ff2cb10e491997bf0e302d6a9fa5 [id=a0b8442c02974d6596db002c9292be0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.973038 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf74f96bea2e4ab7adec42c3c5207e79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.975723 1173 ts_tablet_manager.cc:1434] T e6b76967e5d3476997fa019c4a8afb91 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.070s user 0.003s sys 0.000s
I20260202 02:52:15.977840 1173 tablet_bootstrap.cc:492] T fde2c1fba78248329f3662dd1036c7a2 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
W20260202 02:52:15.979417 513 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.188:39077: connect: Connection refused (error 111) [suppressed 8 similar messages]
I20260202 02:52:15.979808 1173 tablet_bootstrap.cc:654] T fde2c1fba78248329f3662dd1036c7a2 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.985772 1174 tablet_bootstrap.cc:492] T add934bf70ae4de8a40066f06fd94795 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:15.985949 1175 tablet_bootstrap.cc:492] T 0d297b6243c348ee82c146a8339cf6c4 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:15.985965 1174 ts_tablet_manager.cc:1403] T add934bf70ae4de8a40066f06fd94795 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.260s user 0.076s sys 0.107s
I20260202 02:52:15.986100 1175 ts_tablet_manager.cc:1403] T 0d297b6243c348ee82c146a8339cf6c4 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.167s user 0.030s sys 0.033s
I20260202 02:52:15.986931 1209 raft_consensus.cc:493] T 4de41396cffd4d32b86c701b538d1e01 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:15.986883 1174 raft_consensus.cc:359] T add934bf70ae4de8a40066f06fd94795 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.986874 1175 raft_consensus.cc:359] T 0d297b6243c348ee82c146a8339cf6c4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.987073 1175 raft_consensus.cc:385] T 0d297b6243c348ee82c146a8339cf6c4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.987084 1174 raft_consensus.cc:385] T add934bf70ae4de8a40066f06fd94795 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:15.987160 1175 raft_consensus.cc:740] T 0d297b6243c348ee82c146a8339cf6c4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.987175 1174 raft_consensus.cc:740] T add934bf70ae4de8a40066f06fd94795 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:15.987124 1209 raft_consensus.cc:515] T 4de41396cffd4d32b86c701b538d1e01 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.987387 1175 consensus_queue.cc:260] T 0d297b6243c348ee82c146a8339cf6c4 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:15.987448 1174 consensus_queue.cc:260] T add934bf70ae4de8a40066f06fd94795 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:15.987907 1175 ts_tablet_manager.cc:1434] T 0d297b6243c348ee82c146a8339cf6c4 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:15.987941 1174 ts_tablet_manager.cc:1434] T add934bf70ae4de8a40066f06fd94795 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:15.988230 1175 tablet_bootstrap.cc:492] T aa526a69cf914dd08142074636c956b7 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:15.990249 1174 tablet_bootstrap.cc:492] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:15.991470 1175 tablet_bootstrap.cc:654] T aa526a69cf914dd08142074636c956b7 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.993584 1209 leader_election.cc:290] T 4de41396cffd4d32b86c701b538d1e01 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:15.994364 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4de41396cffd4d32b86c701b538d1e01" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:15.994958 715 leader_election.cc:343] T 4de41396cffd4d32b86c701b538d1e01 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:15.995028 1038 tablet_service.cc:1505] Processing CreateTablet for tablet 761f5a51335a4de5977f5f7fef0d4260 (DEFAULT_TABLE table=default.Table_bb9eeb45b01e4ad3b5a6bc6ef8ab60e9 [id=b40664207c364307ba6f5ade490a21d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.995476 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 761f5a51335a4de5977f5f7fef0d4260. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:15.995821 1174 tablet_bootstrap.cc:654] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:15.995697 901 tablet_service.cc:1505] Processing CreateTablet for tablet 0b411e8fab074aa88f8380e3f5694340 (DEFAULT_TABLE table=default.Table_052de59e60de4473a2d13185440ab818 [id=cd07f9746ec94d9ebe956a7ca0d19b1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:15.996447 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b411e8fab074aa88f8380e3f5694340. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.000146 1209 raft_consensus.cc:493] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:16.000450 1209 raft_consensus.cc:515] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.001168 1209 leader_election.cc:290] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:16.001250 714 leader_election.cc:343] T 4de41396cffd4d32b86c701b538d1e01 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 4de41396cffd4d32b86c701b538d1e01
I20260202 02:52:16.001420 714 leader_election.cc:304] T 4de41396cffd4d32b86c701b538d1e01 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:16.002476 1269 raft_consensus.cc:493] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:16.002561 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "672ea3f731ba42ff8b1ff47db039ed22" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:16.002683 1269 raft_consensus.cc:515] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:16.003090 715 leader_election.cc:343] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.003183 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 0b411e8fab074aa88f8380e3f5694340 (DEFAULT_TABLE table=default.Table_052de59e60de4473a2d13185440ab818 [id=cd07f9746ec94d9ebe956a7ca0d19b1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.003394 1269 leader_election.cc:290] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:16.003569 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b411e8fab074aa88f8380e3f5694340. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.004417 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "575016c0e9f84e99b962c5f14ffa96cc" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:16.005100 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "672ea3f731ba42ff8b1ff47db039ed22" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:16.005324 923 raft_consensus.cc:2468] T 672ea3f731ba42ff8b1ff47db039ed22 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:16.005327 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "575016c0e9f84e99b962c5f14ffa96cc" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:16.005439 1174 tablet_bootstrap.cc:492] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:16.005554 777 raft_consensus.cc:2468] T 575016c0e9f84e99b962c5f14ffa96cc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
I20260202 02:52:16.005609 1174 ts_tablet_manager.cc:1403] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.001s
W20260202 02:52:16.006335 859 leader_election.cc:343] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.006454 1174 raft_consensus.cc:359] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.006644 1174 raft_consensus.cc:385] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.006718 1174 raft_consensus.cc:740] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.006986 1174 consensus_queue.cc:260] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.007364 1277 raft_consensus.cc:2749] T 4de41396cffd4d32b86c701b538d1e01 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:16.007539 1174 ts_tablet_manager.cc:1434] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:16.007575 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 6399b63cd8974ee49796f8779f9abc0e (table default.Table_04527bd0309f433aa915767172dcd280 [id=dd2c220f5b88440295facca8da0b1752]) successfully deleted
I20260202 02:52:16.007791 857 leader_election.cc:304] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:16.008356 714 leader_election.cc:304] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
W20260202 02:52:16.008308 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 672ea3f731ba42ff8b1ff47db039ed22 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 672ea3f731ba42ff8b1ff47db039ed22 already in progress: creating tablet
I20260202 02:52:16.008735 1277 raft_consensus.cc:2804] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:16.008843 1277 raft_consensus.cc:493] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:16.008929 1277 raft_consensus.cc:3060] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:16.009016 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 23c38ef4e3c840808ec36f859cf6f6de because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 23c38ef4e3c840808ec36f859cf6f6de already in progress: creating tablet
W20260202 02:52:16.009577 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet c1fc01252d95407e954cfcbf7e715912 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c1fc01252d95407e954cfcbf7e715912 already in progress: creating tablet
I20260202 02:52:16.010010 1183 tablet_replica.cc:333] T 81f35df38f284f0ca5c3791f3d9d9293 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:16.010215 1183 raft_consensus.cc:2243] T 81f35df38f284f0ca5c3791f3d9d9293 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:16.010383 1269 raft_consensus.cc:2804] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:16.010439 1183 raft_consensus.cc:2272] T 81f35df38f284f0ca5c3791f3d9d9293 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:16.010495 1269 raft_consensus.cc:493] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:16.010608 1269 raft_consensus.cc:3060] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:16.011235 1183 ts_tablet_manager.cc:1916] T 81f35df38f284f0ca5c3791f3d9d9293 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:16.011297 1174 tablet_bootstrap.cc:492] T f24c0b99574d437a99b273204f105460 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:16.013119 1174 tablet_bootstrap.cc:654] T f24c0b99574d437a99b273204f105460 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.013340 1269 raft_consensus.cc:515] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.015741 1174 tablet_bootstrap.cc:492] T f24c0b99574d437a99b273204f105460 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:16.015903 1174 ts_tablet_manager.cc:1403] T f24c0b99574d437a99b273204f105460 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260202 02:52:16.016870 1174 raft_consensus.cc:359] T f24c0b99574d437a99b273204f105460 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.017115 1174 raft_consensus.cc:385] T f24c0b99574d437a99b273204f105460 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.017218 1174 raft_consensus.cc:740] T f24c0b99574d437a99b273204f105460 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.017503 1174 consensus_queue.cc:260] T f24c0b99574d437a99b273204f105460 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.018069 1174 ts_tablet_manager.cc:1434] T f24c0b99574d437a99b273204f105460 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:16.019199 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "575016c0e9f84e99b962c5f14ffa96cc" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
W20260202 02:52:16.019491 525 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet af9a1836e5214730a467e86f61165d8a has no consensus state
W20260202 02:52:16.019804 859 leader_election.cc:343] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.020699 1174 tablet_bootstrap.cc:492] T 73f8db8f30f14d369d2227c9be7c4647 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:16.021306 1209 raft_consensus.cc:493] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:16.021484 1209 raft_consensus.cc:515] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:16.022251 1209 leader_election.cc:290] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:16.022886 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ffd8aa2bc6e344b78dce518411997fb5" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:16.023092 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ffd8aa2bc6e344b78dce518411997fb5" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:16.023329 923 raft_consensus.cc:2468] T ffd8aa2bc6e344b78dce518411997fb5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
W20260202 02:52:16.023516 715 leader_election.cc:343] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.024461 714 leader_election.cc:304] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:16.025771 1321 raft_consensus.cc:2804] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:16.025962 1277 raft_consensus.cc:515] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.027880 1041 tablet_service.cc:1505] Processing CreateTablet for tablet 415d14533fd54f20adb00455a7bcad5f (DEFAULT_TABLE table=default.Table_22489ecf56264d93993407ba7d0e6b55 [id=7c19fad4b9554c0682a01bbd3f73d198]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.028012 1277 leader_election.cc:290] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:16.028357 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 415d14533fd54f20adb00455a7bcad5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.028365 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "672ea3f731ba42ff8b1ff47db039ed22" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:16.028707 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "672ea3f731ba42ff8b1ff47db039ed22" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:16.028916 923 raft_consensus.cc:3060] T 672ea3f731ba42ff8b1ff47db039ed22 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:16.028950 715 leader_election.cc:343] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.029601 1269 leader_election.cc:290] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:16.029717 751 tablet_service.cc:1505] Processing CreateTablet for tablet 0b411e8fab074aa88f8380e3f5694340 (DEFAULT_TABLE table=default.Table_052de59e60de4473a2d13185440ab818 [id=cd07f9746ec94d9ebe956a7ca0d19b1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.030004 749 tablet_service.cc:1505] Processing CreateTablet for tablet 761f5a51335a4de5977f5f7fef0d4260 (DEFAULT_TABLE table=default.Table_bb9eeb45b01e4ad3b5a6bc6ef8ab60e9 [id=b40664207c364307ba6f5ade490a21d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.030112 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b411e8fab074aa88f8380e3f5694340. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.030368 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 761f5a51335a4de5977f5f7fef0d4260. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.031437 923 raft_consensus.cc:2468] T 672ea3f731ba42ff8b1ff47db039ed22 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:16.032060 1175 tablet_bootstrap.cc:492] T aa526a69cf914dd08142074636c956b7 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:16.031960 714 leader_election.cc:304] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:16.032235 1175 ts_tablet_manager.cc:1403] T aa526a69cf914dd08142074636c956b7 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.044s user 0.007s sys 0.003s
I20260202 02:52:16.032498 1277 raft_consensus.cc:2804] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:16.033063 1175 raft_consensus.cc:359] T aa526a69cf914dd08142074636c956b7 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.025913 1321 raft_consensus.cc:493] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:16.033264 1175 raft_consensus.cc:385] T aa526a69cf914dd08142074636c956b7 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.026712 1174 tablet_bootstrap.cc:654] T 73f8db8f30f14d369d2227c9be7c4647 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.033337 1175 raft_consensus.cc:740] T aa526a69cf914dd08142074636c956b7 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.033479 1321 raft_consensus.cc:3060] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:16.033619 1175 consensus_queue.cc:260] T aa526a69cf914dd08142074636c956b7 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.026825 1209 raft_consensus.cc:493] T f24c0b99574d437a99b273204f105460 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:16.027356 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "575016c0e9f84e99b962c5f14ffa96cc" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:16.034394 1175 ts_tablet_manager.cc:1434] T aa526a69cf914dd08142074636c956b7 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:16.034888 1175 tablet_bootstrap.cc:492] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:16.036620 1175 tablet_bootstrap.cc:654] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.037734 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_f52d623ce2cb43bf9598994d5298752f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:16.038334 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f52d623ce2cb43bf9598994d5298752f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:16.045262 777 raft_consensus.cc:3060] T 575016c0e9f84e99b962c5f14ffa96cc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:16.045850 1209 raft_consensus.cc:515] T f24c0b99574d437a99b273204f105460 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.047422 524 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet aa526a69cf914dd08142074636c956b7
I20260202 02:52:16.047946 524 catalog_manager.cc:6724] ReplaceTablet: tablet aa526a69cf914dd08142074636c956b7 deleted and replaced by tablet c184d44d7f434b39a997a7387e5d3de0
I20260202 02:52:16.048949 901 tablet_service.cc:1552] Processing DeleteTablet for tablet aa526a69cf914dd08142074636c956b7 with delete_type TABLET_DATA_DELETED (replaced by tablet c184d44d7f434b39a997a7387e5d3de0) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:16.049248 1182 tablet_replica.cc:333] T aa526a69cf914dd08142074636c956b7 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:16.049476 1182 raft_consensus.cc:2243] T aa526a69cf914dd08142074636c956b7 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:16.049656 1182 raft_consensus.cc:2272] T aa526a69cf914dd08142074636c956b7 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:16.050102 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f24c0b99574d437a99b273204f105460" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:16.050618 1182 ts_tablet_manager.cc:1916] T aa526a69cf914dd08142074636c956b7 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:16.051728 715 leader_election.cc:343] T f24c0b99574d437a99b273204f105460 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.051856 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet aa526a69cf914dd08142074636c956b7 with delete_type TABLET_DATA_DELETED (replaced by tablet c184d44d7f434b39a997a7387e5d3de0) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:16.053120 1209 leader_election.cc:290] T f24c0b99574d437a99b273204f105460 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:16.053246 777 raft_consensus.cc:2468] T 575016c0e9f84e99b962c5f14ffa96cc P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
W20260202 02:52:16.053682 714 leader_election.cc:343] T f24c0b99574d437a99b273204f105460 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: f24c0b99574d437a99b273204f105460
I20260202 02:52:16.054333 714 leader_election.cc:304] T f24c0b99574d437a99b273204f105460 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:16.054765 1277 raft_consensus.cc:697] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:16.055087 756 tablet_service.cc:1552] Processing DeleteTablet for tablet aa526a69cf914dd08142074636c956b7 with delete_type TABLET_DATA_DELETED (replaced by tablet c184d44d7f434b39a997a7387e5d3de0) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:16.055469 1209 raft_consensus.cc:2749] T f24c0b99574d437a99b273204f105460 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:16.055859 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet aa526a69cf914dd08142074636c956b7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aa526a69cf914dd08142074636c956b7 already in progress: creating tablet
I20260202 02:52:16.058909 1321 raft_consensus.cc:515] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:16.059190 1182 ts_tablet_manager.cc:1929] T aa526a69cf914dd08142074636c956b7 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:16.059404 1182 log.cc:1199] T aa526a69cf914dd08142074636c956b7 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/aa526a69cf914dd08142074636c956b7
I20260202 02:52:16.059870 1182 ts_tablet_manager.cc:1950] T aa526a69cf914dd08142074636c956b7 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:16.059841 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet e6b76967e5d3476997fa019c4a8afb91 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:16.060060 1322 raft_consensus.cc:493] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:16.060181 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ffd8aa2bc6e344b78dce518411997fb5" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:16.060202 1322 raft_consensus.cc:515] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.060520 923 raft_consensus.cc:3060] T ffd8aa2bc6e344b78dce518411997fb5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:16.061465 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "251455168e33430aafea8e0de42ca1b8" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:16.063436 923 raft_consensus.cc:2468] T ffd8aa2bc6e344b78dce518411997fb5 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:16.063905 1044 tablet_service.cc:1505] Processing CreateTablet for tablet ff6f5710de4b4c48948aacdf5be29e8b (DEFAULT_TABLE table=default.Table_4720e764353243a495245b92e5b913fb [id=f2276e1c2ca04a70804053da61dd3447]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.064381 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff6f5710de4b4c48948aacdf5be29e8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.064570 750 tablet_service.cc:1505] Processing CreateTablet for tablet ff6f5710de4b4c48948aacdf5be29e8b (DEFAULT_TABLE table=default.Table_4720e764353243a495245b92e5b913fb [id=f2276e1c2ca04a70804053da61dd3447]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.064949 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff6f5710de4b4c48948aacdf5be29e8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.067175 756 tablet_service.cc:1505] Processing CreateTablet for tablet 415d14533fd54f20adb00455a7bcad5f (DEFAULT_TABLE table=default.Table_22489ecf56264d93993407ba7d0e6b55 [id=7c19fad4b9554c0682a01bbd3f73d198]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.067653 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 415d14533fd54f20adb00455a7bcad5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.069592 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_8dc3d64cfa464a00864bf7850006f1f0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:16.070192 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8dc3d64cfa464a00864bf7850006f1f0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:16.071832 715 leader_election.cc:343] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.081173 521 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_08912fa0817a47e6a48aa728d44438cf" } new_table_name: "default.Table_2669ca9662b942c08bcbfd7641cfe2e3" modify_external_catalogs: true
I20260202 02:52:16.082237 1174 tablet_bootstrap.cc:492] T 73f8db8f30f14d369d2227c9be7c4647 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:16.082433 1174 ts_tablet_manager.cc:1403] T 73f8db8f30f14d369d2227c9be7c4647 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.064s user 0.015s sys 0.005s
I20260202 02:52:16.083287 1174 raft_consensus.cc:359] T 73f8db8f30f14d369d2227c9be7c4647 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.083994 1175 tablet_bootstrap.cc:492] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:16.084192 1175 ts_tablet_manager.cc:1403] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.049s user 0.008s sys 0.000s
I20260202 02:52:16.085039 1175 raft_consensus.cc:359] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:16.085301 1175 raft_consensus.cc:385] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.085449 1175 raft_consensus.cc:740] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.085422 1269 consensus_queue.cc:1048] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:16.085805 1175 consensus_queue.cc:260] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:16.086675 1175 ts_tablet_manager.cc:1434] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:16.087031 1175 tablet_bootstrap.cc:492] T 23c38ef4e3c840808ec36f859cf6f6de P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:16.088825 1175 tablet_bootstrap.cc:654] T 23c38ef4e3c840808ec36f859cf6f6de P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.090462 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet aa526a69cf914dd08142074636c956b7 (table default.Table_9451b9f5148f4d9aab65983dea967e32 [id=9ef504467a544f299b48499e49adf27f]) successfully deleted
I20260202 02:52:16.091550 857 leader_election.cc:304] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:16.091951 714 leader_election.cc:304] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters:
I20260202 02:52:16.092429 1324 raft_consensus.cc:2804] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:16.092607 1324 raft_consensus.cc:697] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:16.092941 1324 consensus_queue.cc:237] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:16.094930 1269 raft_consensus.cc:2804] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:16.095093 1269 raft_consensus.cc:697] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:16.095350 1269 consensus_queue.cc:237] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.096133 1174 raft_consensus.cc:385] T 73f8db8f30f14d369d2227c9be7c4647 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.096239 1174 raft_consensus.cc:740] T 73f8db8f30f14d369d2227c9be7c4647 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.096607 898 tablet_service.cc:1505] Processing CreateTablet for tablet 761f5a51335a4de5977f5f7fef0d4260 (DEFAULT_TABLE table=default.Table_bb9eeb45b01e4ad3b5a6bc6ef8ab60e9 [id=b40664207c364307ba6f5ade490a21d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.096941 1174 consensus_queue.cc:260] T 73f8db8f30f14d369d2227c9be7c4647 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.097249 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 761f5a51335a4de5977f5f7fef0d4260. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.097126 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_78e75899f83c46f09c66dc84192f5438"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:16.097821 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_78e75899f83c46f09c66dc84192f5438 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:16.098919 1174 ts_tablet_manager.cc:1434] T 73f8db8f30f14d369d2227c9be7c4647 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.016s user 0.002s sys 0.000s
I20260202 02:52:16.099344 901 tablet_service.cc:1505] Processing CreateTablet for tablet 415d14533fd54f20adb00455a7bcad5f (DEFAULT_TABLE table=default.Table_22489ecf56264d93993407ba7d0e6b55 [id=7c19fad4b9554c0682a01bbd3f73d198]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.099349 1277 consensus_queue.cc:237] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.099751 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 415d14533fd54f20adb00455a7bcad5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.100494 903 tablet_service.cc:1505] Processing CreateTablet for tablet ff6f5710de4b4c48948aacdf5be29e8b (DEFAULT_TABLE table=default.Table_4720e764353243a495245b92e5b913fb [id=f2276e1c2ca04a70804053da61dd3447]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.100907 903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff6f5710de4b4c48948aacdf5be29e8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.109863 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_5e366ca4b9744597add5ca93f7fe7850"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:16.110505 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5e366ca4b9744597add5ca93f7fe7850 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:16.112124 1174 tablet_bootstrap.cc:492] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:16.113996 1174 tablet_bootstrap.cc:654] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.114849 1173 tablet_bootstrap.cc:492] T fde2c1fba78248329f3662dd1036c7a2 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:16.115098 1173 ts_tablet_manager.cc:1403] T fde2c1fba78248329f3662dd1036c7a2 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.137s user 0.001s sys 0.003s
I20260202 02:52:16.119163 522 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 251455168e33430aafea8e0de42ca1b8 (table default.Table_86f34199535c451cb54a2e20e70f1149 [id=4f19275f416a425c9b9d100b2a83870a]). Expected version 1 got 0
I20260202 02:52:16.119624 522 catalog_manager.cc:5654] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130). New cstate: current_term: 1 leader_uuid: "1521a6d262b7451bbef8debe970dd6b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: HEALTHY } } }
I20260202 02:52:16.120029 522 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 938813ee0a8a40b6bf0e537e3ec1a0d4 (table default.Table_e70b044a267b465d94fede069f2f0df6 [id=1229c6d37b994a669d3363fdf6f8c6c8]). Expected version 1 got 0
I20260202 02:52:16.120316 522 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet ffd8aa2bc6e344b78dce518411997fb5 (table default.Table_ab8d74f30cd14c61b85950a82d0f3787 [id=3ccb047c30974c0f9cbe778971e61644]). Expected version 3 got 0
I20260202 02:52:16.121186 527 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 97bd7223d41d40a6bcdc60b1f63abc67
I20260202 02:52:16.121995 519 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 952d760159844ad08269654264e6381b
I20260202 02:52:16.122586 519 master_service.cc:954] ReplaceTablet: received request to replace tablet 4f2a9404f7424dc78b5bd062ebd8cb53 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:16.122792 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 952d760159844ad08269654264e6381b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:16.122892 523 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet bc1539e5c8aa4d6e98f687728b891530
W20260202 02:52:16.123416 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 952d760159844ad08269654264e6381b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 952d760159844ad08269654264e6381b already in progress: creating tablet
I20260202 02:52:16.124472 901 tablet_service.cc:1552] Processing DeleteTablet for tablet 97bd7223d41d40a6bcdc60b1f63abc67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:16.124712 903 tablet_service.cc:1552] Processing DeleteTablet for tablet 952d760159844ad08269654264e6381b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:16.124707 750 tablet_service.cc:1552] Processing DeleteTablet for tablet 97bd7223d41d40a6bcdc60b1f63abc67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:16.124948 1182 tablet_replica.cc:333] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:16.125144 1182 raft_consensus.cc:2243] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:16.125365 1182 raft_consensus.cc:2272] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:16.125754 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 97bd7223d41d40a6bcdc60b1f63abc67 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 97bd7223d41d40a6bcdc60b1f63abc67 already in progress: creating tablet
I20260202 02:52:16.126242 1182 ts_tablet_manager.cc:1916] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:16.126466 1322 leader_election.cc:290] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:16.127050 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet 952d760159844ad08269654264e6381b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:16.128571 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "251455168e33430aafea8e0de42ca1b8" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:16.128829 923 raft_consensus.cc:2468] T 251455168e33430aafea8e0de42ca1b8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:16.129271 714 leader_election.cc:304] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:16.130296 1322 raft_consensus.cc:2804] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:16.130465 1322 raft_consensus.cc:493] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:16.130604 1322 raft_consensus.cc:3060] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:16.132061 1183 ts_tablet_manager.cc:1929] T 81f35df38f284f0ca5c3791f3d9d9293 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:16.132215 1183 log.cc:1199] T 81f35df38f284f0ca5c3791f3d9d9293 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/81f35df38f284f0ca5c3791f3d9d9293
I20260202 02:52:16.132436 1175 tablet_bootstrap.cc:492] T 23c38ef4e3c840808ec36f859cf6f6de P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:16.132651 1175 ts_tablet_manager.cc:1403] T 23c38ef4e3c840808ec36f859cf6f6de P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.046s user 0.006s sys 0.000s
I20260202 02:52:16.132959 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet 97bd7223d41d40a6bcdc60b1f63abc67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:16.133529 1175 raft_consensus.cc:359] T 23c38ef4e3c840808ec36f859cf6f6de P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:16.133736 1175 raft_consensus.cc:385] T 23c38ef4e3c840808ec36f859cf6f6de P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.133816 1175 raft_consensus.cc:740] T 23c38ef4e3c840808ec36f859cf6f6de P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.133740 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_950ad480ee834bbdbd74eaaccccb42ec" } new_table_name: "default.Table_c9d046b019d94f46bee8e3fd28d253ca" modify_external_catalogs: true
I20260202 02:52:16.134054 1175 consensus_queue.cc:260] T 23c38ef4e3c840808ec36f859cf6f6de P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:16.135216 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_ae49654353a64141b6d7f0e6afe13d9d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:16.135748 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ae49654353a64141b6d7f0e6afe13d9d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:16.136710 1182 ts_tablet_manager.cc:1929] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:16.136853 1182 log.cc:1199] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/97bd7223d41d40a6bcdc60b1f63abc67
I20260202 02:52:16.137204 1182 ts_tablet_manager.cc:1950] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:16.138113 1182 tablet_replica.cc:333] T 952d760159844ad08269654264e6381b P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:16.138502 1175 ts_tablet_manager.cc:1434] T 23c38ef4e3c840808ec36f859cf6f6de P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260202 02:52:16.140125 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 97bd7223d41d40a6bcdc60b1f63abc67 (table default.Table_53eaadb859904d6b95ef0e536019ad79 [id=0ca5aac09bb54feba75b2d3784a69140]) successfully deleted
I20260202 02:52:16.140367 1175 tablet_bootstrap.cc:492] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:16.142211 1175 tablet_bootstrap.cc:654] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.143277 1173 raft_consensus.cc:359] T fde2c1fba78248329f3662dd1036c7a2 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:16.143527 1173 raft_consensus.cc:385] T fde2c1fba78248329f3662dd1036c7a2 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.143877 1173 raft_consensus.cc:740] T fde2c1fba78248329f3662dd1036c7a2 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.144152 1173 consensus_queue.cc:260] T fde2c1fba78248329f3662dd1036c7a2 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:16.145001 1173 ts_tablet_manager.cc:1434] T fde2c1fba78248329f3662dd1036c7a2 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.030s user 0.002s sys 0.000s
I20260202 02:52:16.146695 1182 raft_consensus.cc:2243] T 952d760159844ad08269654264e6381b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:16.146929 1182 raft_consensus.cc:2272] T 952d760159844ad08269654264e6381b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:16.147455 1321 leader_election.cc:290] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:16.147801 1269 raft_consensus.cc:493] T c0937b6910534ddc8a2217f0cbd539c5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:16.147838 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ffd8aa2bc6e344b78dce518411997fb5" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:16.147976 1269 raft_consensus.cc:515] T c0937b6910534ddc8a2217f0cbd539c5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.148636 1182 ts_tablet_manager.cc:1916] T 952d760159844ad08269654264e6381b P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:16.148870 1269 leader_election.cc:290] T c0937b6910534ddc8a2217f0cbd539c5 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:16.149477 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c0937b6910534ddc8a2217f0cbd539c5" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:16.149475 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c0937b6910534ddc8a2217f0cbd539c5" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:16.150009 857 leader_election.cc:343] T c0937b6910534ddc8a2217f0cbd539c5 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:16.150265 715 leader_election.cc:343] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:16.150645 859 leader_election.cc:343] T c0937b6910534ddc8a2217f0cbd539c5 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.150828 859 leader_election.cc:304] T c0937b6910534ddc8a2217f0cbd539c5 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
W20260202 02:52:16.154249 1334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260202 02:52:16.154896 1269 raft_consensus.cc:2749] T c0937b6910534ddc8a2217f0cbd539c5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:16.155795 1322 raft_consensus.cc:515] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.155905 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_dcf32fa66b524c84877c4668489c9962"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:16.156052 1314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:52:16.156553 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dcf32fa66b524c84877c4668489c9962 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:16.157378 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "251455168e33430aafea8e0de42ca1b8" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:16.157609 923 raft_consensus.cc:3060] T 251455168e33430aafea8e0de42ca1b8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:16.158488 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "251455168e33430aafea8e0de42ca1b8" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
W20260202 02:52:16.159140 715 leader_election.cc:343] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.159410 1269 raft_consensus.cc:493] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:16.159611 1269 raft_consensus.cc:515] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.160550 923 raft_consensus.cc:2468] T 251455168e33430aafea8e0de42ca1b8 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:16.160750 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "73f8db8f30f14d369d2227c9be7c4647" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:16.161063 714 leader_election.cc:304] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
W20260202 02:52:16.161391 859 leader_election.cc:343] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.161461 1321 raft_consensus.cc:2804] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:16.161625 1321 raft_consensus.cc:697] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:16.161867 1269 leader_election.cc:290] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:16.162323 1321 consensus_queue.cc:237] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.163022 1175 tablet_bootstrap.cc:492] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:16.163189 1175 ts_tablet_manager.cc:1403] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.024s user 0.004s sys 0.000s
I20260202 02:52:16.165123 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "73f8db8f30f14d369d2227c9be7c4647" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:16.165347 777 raft_consensus.cc:2468] T 73f8db8f30f14d369d2227c9be7c4647 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
I20260202 02:52:16.165830 857 leader_election.cc:304] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:16.166070 1175 raft_consensus.cc:359] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.166237 1175 raft_consensus.cc:385] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.166321 1175 raft_consensus.cc:740] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.166586 777 raft_consensus.cc:1275] T 80eb3013f5354022a2dd78bb294a8497 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Refusing update from remote peer 1521a6d262b7451bbef8debe970dd6b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260202 02:52:16.166550 1175 consensus_queue.cc:260] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.166817 1269 raft_consensus.cc:2804] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:16.166885 1269 raft_consensus.cc:493] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:16.166970 1269 raft_consensus.cc:3060] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:16.169756 1269 raft_consensus.cc:515] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.170470 1269 leader_election.cc:290] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:16.171305 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "73f8db8f30f14d369d2227c9be7c4647" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:16.171525 777 raft_consensus.cc:3060] T 73f8db8f30f14d369d2227c9be7c4647 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:16.171980 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "73f8db8f30f14d369d2227c9be7c4647" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
W20260202 02:52:16.172518 859 leader_election.cc:343] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.174321 777 raft_consensus.cc:2468] T 73f8db8f30f14d369d2227c9be7c4647 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
I20260202 02:52:16.145357 1173 tablet_bootstrap.cc:492] T fae7e3941c124cc2871390a0b6471f27 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:16.175990 524 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_98d65bff8bd24989b88007d79e20c2ff" } new_table_name: "default.Table_3919d090261e4b90b0672f2b863588b2" modify_external_catalogs: true
I20260202 02:52:16.176739 857 leader_election.cc:304] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:16.177035 1184 consensus_queue.cc:1048] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:16.177199 1269 raft_consensus.cc:2804] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:16.178634 1175 ts_tablet_manager.cc:1434] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.015s user 0.002s sys 0.000s
I20260202 02:52:16.178954 1175 tablet_bootstrap.cc:492] T ea040c03420c4b1b98897f39160b4130 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:16.180433 1322 leader_election.cc:290] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:16.180694 1175 tablet_bootstrap.cc:654] T ea040c03420c4b1b98897f39160b4130 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.181668 1269 raft_consensus.cc:697] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:16.181998 1269 consensus_queue.cc:237] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.183127 1174 tablet_bootstrap.cc:492] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:16.183391 1174 ts_tablet_manager.cc:1403] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.071s user 0.007s sys 0.021s
I20260202 02:52:16.186302 1182 ts_tablet_manager.cc:1929] T 952d760159844ad08269654264e6381b P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:16.185951 1174 raft_consensus.cc:359] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.186448 1174 raft_consensus.cc:385] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.186547 1174 raft_consensus.cc:740] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.187068 1174 consensus_queue.cc:260] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.187628 1174 ts_tablet_manager.cc:1434] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260202 02:52:16.187975 1174 tablet_bootstrap.cc:492] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
W20260202 02:52:16.188429 521 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 583e547d0a1943f09f4efe82070fc777 has no consensus state
I20260202 02:52:16.189695 1182 log.cc:1199] T 952d760159844ad08269654264e6381b P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/952d760159844ad08269654264e6381b
I20260202 02:52:16.190101 1182 ts_tablet_manager.cc:1950] T 952d760159844ad08269654264e6381b P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:16.190802 1174 tablet_bootstrap.cc:654] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.189136 1175 tablet_bootstrap.cc:492] T ea040c03420c4b1b98897f39160b4130 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:16.191133 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 952d760159844ad08269654264e6381b (table default.Table_b6c1fb1e7282492d920385869810b0e0 [id=2cc4567c7bd44745a5851cd8b95b22c4]) successfully deleted
I20260202 02:52:16.191197 1175 ts_tablet_manager.cc:1403] T ea040c03420c4b1b98897f39160b4130 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.000s
I20260202 02:52:16.194356 1175 raft_consensus.cc:359] T ea040c03420c4b1b98897f39160b4130 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:16.194651 1175 raft_consensus.cc:385] T ea040c03420c4b1b98897f39160b4130 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.194870 1175 raft_consensus.cc:740] T ea040c03420c4b1b98897f39160b4130 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.195429 1175 consensus_queue.cc:260] T ea040c03420c4b1b98897f39160b4130 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:16.196849 1175 ts_tablet_manager.cc:1434] T ea040c03420c4b1b98897f39160b4130 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260202 02:52:16.198292 521 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 80eb3013f5354022a2dd78bb294a8497 (table default.Table_25c6d490f243450ab2d9b43ec9cda7bf [id=316b8a276d9b415fafe8511aa7c32fc5]). Expected version 2 got 0
I20260202 02:52:16.199153 521 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_9c6ddf42d084449998f1f7925b0b0c3b" } new_table_name: "default.Table_e0149b31be0c49d19600de4849d6577e" modify_external_catalogs: true
I20260202 02:52:16.200182 1175 tablet_bootstrap.cc:492] T 724d47241f0342768fb1321ef37c3a7b P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:16.201655 1184 raft_consensus.cc:493] T c050a2882f764dda8df2a7df6547f149 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:16.201802 1184 raft_consensus.cc:515] T c050a2882f764dda8df2a7df6547f149 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.202461 1184 leader_election.cc:290] T c050a2882f764dda8df2a7df6547f149 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:16.203125 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c050a2882f764dda8df2a7df6547f149" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:16.203684 857 leader_election.cc:343] T c050a2882f764dda8df2a7df6547f149 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.203720 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c050a2882f764dda8df2a7df6547f149" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:16.204941 859 leader_election.cc:343] T c050a2882f764dda8df2a7df6547f149 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.205123 859 leader_election.cc:304] T c050a2882f764dda8df2a7df6547f149 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:16.205667 1184 raft_consensus.cc:2749] T c050a2882f764dda8df2a7df6547f149 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:16.206580 520 master_service.cc:954] ReplaceTablet: received request to replace tablet c1fc01252d95407e954cfcbf7e715912 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:16.204948 1175 tablet_bootstrap.cc:654] T 724d47241f0342768fb1321ef37c3a7b P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.209309 1183 ts_tablet_manager.cc:1950] T 81f35df38f284f0ca5c3791f3d9d9293 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:16.210326 1183 tablet_replica.cc:333] T 7e81d5780d9647b9baac0066282376ea P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:16.210495 1183 raft_consensus.cc:2243] T 7e81d5780d9647b9baac0066282376ea P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:16.210671 1183 raft_consensus.cc:2272] T 7e81d5780d9647b9baac0066282376ea P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:16.211201 1183 ts_tablet_manager.cc:1916] T 7e81d5780d9647b9baac0066282376ea P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:16.211944 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 81f35df38f284f0ca5c3791f3d9d9293 (table default.Table_15c57b6d38ab4f2fb7dca2a405f01058 [id=d54948158c124d00ab0e2b1f43d45cc3]) successfully deleted
I20260202 02:52:16.212212 1174 tablet_bootstrap.cc:492] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:16.212426 1174 ts_tablet_manager.cc:1403] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.025s user 0.010s sys 0.000s
I20260202 02:52:16.213088 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet fde2c1fba78248329f3662dd1036c7a2 with delete_type TABLET_DATA_DELETED (replaced by tablet 23c38ef4e3c840808ec36f859cf6f6de) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:16.213209 1174 raft_consensus.cc:359] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.213418 1174 raft_consensus.cc:385] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.213510 1174 raft_consensus.cc:740] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.213757 1174 consensus_queue.cc:260] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.214242 1174 ts_tablet_manager.cc:1434] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:16.214641 1174 tablet_bootstrap.cc:492] T 61a9203035f2495f8f99924303ebf120 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:16.218181 1174 tablet_bootstrap.cc:654] T 61a9203035f2495f8f99924303ebf120 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.223197 1173 tablet_bootstrap.cc:654] T fae7e3941c124cc2871390a0b6471f27 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.225754 526 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 80eb3013f5354022a2dd78bb294a8497 (table default.Table_25c6d490f243450ab2d9b43ec9cda7bf [id=316b8a276d9b415fafe8511aa7c32fc5]). Expected version 2 got 0
I20260202 02:52:16.226225 1175 tablet_bootstrap.cc:492] T 724d47241f0342768fb1321ef37c3a7b P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:16.226495 526 master_service.cc:954] ReplaceTablet: received request to replace tablet add934bf70ae4de8a40066f06fd94795 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:16.226989 1175 ts_tablet_manager.cc:1403] T 724d47241f0342768fb1321ef37c3a7b P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.027s user 0.008s sys 0.005s
I20260202 02:52:16.227236 1174 tablet_bootstrap.cc:492] T 61a9203035f2495f8f99924303ebf120 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:16.227392 1174 ts_tablet_manager.cc:1403] T 61a9203035f2495f8f99924303ebf120 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.002s
I20260202 02:52:16.227946 1175 raft_consensus.cc:359] T 724d47241f0342768fb1321ef37c3a7b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:16.228143 1175 raft_consensus.cc:385] T 724d47241f0342768fb1321ef37c3a7b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.228225 1175 raft_consensus.cc:740] T 724d47241f0342768fb1321ef37c3a7b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.228137 1174 raft_consensus.cc:359] T 61a9203035f2495f8f99924303ebf120 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.228368 1174 raft_consensus.cc:385] T 61a9203035f2495f8f99924303ebf120 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.228458 1174 raft_consensus.cc:740] T 61a9203035f2495f8f99924303ebf120 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.228472 1175 consensus_queue.cc:260] T 724d47241f0342768fb1321ef37c3a7b P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:16.228681 1174 consensus_queue.cc:260] T 61a9203035f2495f8f99924303ebf120 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.229321 1174 ts_tablet_manager.cc:1434] T 61a9203035f2495f8f99924303ebf120 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260202 02:52:16.229636 1174 tablet_bootstrap.cc:492] T c050a2882f764dda8df2a7df6547f149 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:16.232162 1174 tablet_bootstrap.cc:654] T c050a2882f764dda8df2a7df6547f149 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.232949 1175 ts_tablet_manager.cc:1434] T 724d47241f0342768fb1321ef37c3a7b P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260202 02:52:16.233476 528 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet 2ff6af17a6854451a1d7b904aa4f1f55 (table default.Table_5eefa56618e84f8d92f8d87132ccd7d4 [id=b4ab2ba86e1a49338a98d855ded99f5d]). Expected version 1 got 0
I20260202 02:52:16.233716 528 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet ffd8aa2bc6e344b78dce518411997fb5 (table default.Table_ab8d74f30cd14c61b85950a82d0f3787 [id=3ccb047c30974c0f9cbe778971e61644]). Expected version 3 got 0
I20260202 02:52:16.233894 528 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet 938813ee0a8a40b6bf0e537e3ec1a0d4 (table default.Table_e70b044a267b465d94fede069f2f0df6 [id=1229c6d37b994a669d3363fdf6f8c6c8]). Expected version 1 got 0
I20260202 02:52:16.233880 1321 raft_consensus.cc:493] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:16.234025 1321 raft_consensus.cc:515] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.234407 1175 tablet_bootstrap.cc:492] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:16.234694 1321 leader_election.cc:290] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:16.235778 714 leader_election.cc:343] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 97bd7223d41d40a6bcdc60b1f63abc67
I20260202 02:52:16.237576 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "97bd7223d41d40a6bcdc60b1f63abc67" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:16.238179 715 leader_election.cc:343] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.238341 715 leader_election.cc:304] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:16.238788 1321 raft_consensus.cc:2749] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:16.239233 1175 tablet_bootstrap.cc:654] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.239573 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 575016c0e9f84e99b962c5f14ffa96cc with delete_type TABLET_DATA_DELETED (replaced by tablet ff6f5710de4b4c48948aacdf5be29e8b) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:16.239871 1180 tablet_replica.cc:333] T 575016c0e9f84e99b962c5f14ffa96cc P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:16.240068 1180 raft_consensus.cc:2243] T 575016c0e9f84e99b962c5f14ffa96cc P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:16.240270 1180 raft_consensus.cc:2272] T 575016c0e9f84e99b962c5f14ffa96cc P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:16.240891 1180 ts_tablet_manager.cc:1916] T 575016c0e9f84e99b962c5f14ffa96cc P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:16.244625 1269 raft_consensus.cc:493] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:16.244663 750 tablet_service.cc:1552] Processing DeleteTablet for tablet 72475ea720c6429aa68de9478b374280 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:16.244848 1269 raft_consensus.cc:515] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.245760 1269 leader_election.cc:290] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:16.246865 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "61a9203035f2495f8f99924303ebf120" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:16.247077 526 master_service.cc:954] ReplaceTablet: received request to replace tablet 952d760159844ad08269654264e6381b from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:16.247131 779 raft_consensus.cc:2468] T 61a9203035f2495f8f99924303ebf120 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
I20260202 02:52:16.247423 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "61a9203035f2495f8f99924303ebf120" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:16.247567 857 leader_election.cc:304] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters:
W20260202 02:52:16.248077 859 leader_election.cc:343] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.248194 756 tablet_service.cc:1552] Processing DeleteTablet for tablet fc93352b24e345f99d15085e4886c748 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:16.248430 1269 raft_consensus.cc:2804] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:16.248538 1269 raft_consensus.cc:493] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:16.248620 1269 raft_consensus.cc:3060] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:16.250003 1321 consensus_queue.cc:1048] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260202 02:52:16.253271 1180 ts_tablet_manager.cc:1929] T 575016c0e9f84e99b962c5f14ffa96cc P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:16.253474 1269 raft_consensus.cc:515] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.253755 1180 log.cc:1199] T 575016c0e9f84e99b962c5f14ffa96cc P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/575016c0e9f84e99b962c5f14ffa96cc
I20260202 02:52:16.254173 1180 ts_tablet_manager.cc:1950] T 575016c0e9f84e99b962c5f14ffa96cc P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:16.255456 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 575016c0e9f84e99b962c5f14ffa96cc (table default.Table_4720e764353243a495245b92e5b913fb [id=f2276e1c2ca04a70804053da61dd3447]) successfully deleted
I20260202 02:52:16.255959 1180 tablet_replica.cc:333] T 72475ea720c6429aa68de9478b374280 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:16.256301 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "61a9203035f2495f8f99924303ebf120" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:16.256371 1269 leader_election.cc:290] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:16.256896 859 leader_election.cc:343] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.256898 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "61a9203035f2495f8f99924303ebf120" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:16.257112 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 4de41396cffd4d32b86c701b538d1e01 with delete_type TABLET_DATA_DELETED (replaced by tablet bf74f96bea2e4ab7adec42c3c5207e79) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:16.257378 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 672ea3f731ba42ff8b1ff47db039ed22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:16.257138 779 raft_consensus.cc:3060] T 61a9203035f2495f8f99924303ebf120 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:16.261245 779 raft_consensus.cc:2468] T 61a9203035f2495f8f99924303ebf120 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
I20260202 02:52:16.261935 857 leader_election.cc:304] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:16.262472 1269 raft_consensus.cc:2804] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:16.262636 1269 raft_consensus.cc:697] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:16.262943 1269 consensus_queue.cc:237] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.264104 1175 tablet_bootstrap.cc:492] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:16.265262 1175 ts_tablet_manager.cc:1403] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.031s user 0.001s sys 0.011s
I20260202 02:52:16.266589 1175 raft_consensus.cc:359] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:16.256909 1180 raft_consensus.cc:2243] T 72475ea720c6429aa68de9478b374280 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:16.268159 1175 raft_consensus.cc:385] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.268741 1175 raft_consensus.cc:740] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.268766 1180 raft_consensus.cc:2272] T 72475ea720c6429aa68de9478b374280 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:16.269443 1175 consensus_queue.cc:260] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:16.269593 1180 ts_tablet_manager.cc:1916] T 72475ea720c6429aa68de9478b374280 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:16.270967 1175 ts_tablet_manager.cc:1434] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260202 02:52:16.276592 1175 tablet_bootstrap.cc:492] T bf5ca99708044482970cb854d2eced18 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:16.281278 1175 tablet_bootstrap.cc:654] T bf5ca99708044482970cb854d2eced18 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.282361 528 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_42ecaddd58b64765aef12dda7d1e0fcc" } modify_external_catalogs: true
I20260202 02:52:16.282590 528 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_42ecaddd58b64765aef12dda7d1e0fcc" } modify_external_catalogs: true
I20260202 02:52:16.283107 520 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_5c9c0f09396a4a0e8b3d676fcb2dbb2c" } modify_external_catalogs: true
W20260202 02:52:16.283237 715 consensus_peers.cc:597] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:16.283252 520 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_5c9c0f09396a4a0e8b3d676fcb2dbb2c" } modify_external_catalogs: true
I20260202 02:52:16.284461 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_028fb132e4894101b8470628a6041c07"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:16.285079 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_028fb132e4894101b8470628a6041c07 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:16.294011 527 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c12ff20ca6b44ddba0c936a28cfb2eb9 has no consensus state
I20260202 02:52:16.295209 1351 tablet.cc:1722] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:16.300109 519 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 4f2a9404f7424dc78b5bd062ebd8cb53
I20260202 02:52:16.300706 519 catalog_manager.cc:6724] ReplaceTablet: tablet 4f2a9404f7424dc78b5bd062ebd8cb53 deleted and replaced by tablet 526f2c7b95184a0987e7aa94efc8475a
I20260202 02:52:16.302984 779 raft_consensus.cc:1275] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Refusing update from remote peer 1521a6d262b7451bbef8debe970dd6b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260202 02:52:16.303661 1337 consensus_queue.cc:1048] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:16.303813 901 tablet_service.cc:1552] Processing DeleteTablet for tablet 4f2a9404f7424dc78b5bd062ebd8cb53 with delete_type TABLET_DATA_DELETED (replaced by tablet 526f2c7b95184a0987e7aa94efc8475a) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:16.303875 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 4f2a9404f7424dc78b5bd062ebd8cb53 with delete_type TABLET_DATA_DELETED (replaced by tablet 526f2c7b95184a0987e7aa94efc8475a) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:16.304069 1338 raft_consensus.cc:493] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:16.304121 1182 tablet_replica.cc:333] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
W20260202 02:52:16.304155 859 consensus_peers.cc:597] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5 -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:16.304215 1338 raft_consensus.cc:515] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.304301 1182 raft_consensus.cc:2243] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:16.304497 1182 raft_consensus.cc:2272] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:16.305402 1338 leader_election.cc:290] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:16.305685 1182 ts_tablet_manager.cc:1916] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:16.306277 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "add934bf70ae4de8a40066f06fd94795" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:16.306905 859 leader_election.cc:343] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.307938 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "add934bf70ae4de8a40066f06fd94795" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:16.308185 778 raft_consensus.cc:2468] T add934bf70ae4de8a40066f06fd94795 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
I20260202 02:52:16.308707 857 leader_election.cc:304] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:16.309484 1338 raft_consensus.cc:2804] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:16.309607 1338 raft_consensus.cc:493] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:16.309695 1338 raft_consensus.cc:3060] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:16.312770 1338 raft_consensus.cc:515] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.312858 1246 server_base.cc:1060] running on GCE node
I20260202 02:52:16.314137 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "add934bf70ae4de8a40066f06fd94795" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:16.314670 1338 leader_election.cc:290] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
W20260202 02:52:16.314857 859 leader_election.cc:343] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.315698 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "add934bf70ae4de8a40066f06fd94795" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:16.315915 1173 tablet_bootstrap.cc:492] T fae7e3941c124cc2871390a0b6471f27 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:16.315917 778 raft_consensus.cc:3060] T add934bf70ae4de8a40066f06fd94795 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:16.316061 1173 ts_tablet_manager.cc:1403] T fae7e3941c124cc2871390a0b6471f27 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.171s user 0.003s sys 0.001s
I20260202 02:52:16.316943 1173 raft_consensus.cc:359] T fae7e3941c124cc2871390a0b6471f27 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:16.317140 1173 raft_consensus.cc:385] T fae7e3941c124cc2871390a0b6471f27 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.317215 1173 raft_consensus.cc:740] T fae7e3941c124cc2871390a0b6471f27 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.317430 1173 consensus_queue.cc:260] T fae7e3941c124cc2871390a0b6471f27 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
W20260202 02:52:16.318681 1345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260202 02:52:16.319160 778 raft_consensus.cc:2468] T add934bf70ae4de8a40066f06fd94795 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
I20260202 02:52:16.319885 1246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260202 02:52:16.320143 857 leader_election.cc:304] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:16.320541 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 4f2a9404f7424dc78b5bd062ebd8cb53 with delete_type TABLET_DATA_DELETED (replaced by tablet 526f2c7b95184a0987e7aa94efc8475a) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:16.321185 1338 raft_consensus.cc:2804] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:16.321377 1338 raft_consensus.cc:697] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:16.321692 1338 consensus_queue.cc:237] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.320405 1182 ts_tablet_manager.cc:1929] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:16.322213 1182 log.cc:1199] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/4f2a9404f7424dc78b5bd062ebd8cb53
I20260202 02:52:16.324043 1183 ts_tablet_manager.cc:1929] T 7e81d5780d9647b9baac0066282376ea P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:16.324234 1183 log.cc:1199] T 7e81d5780d9647b9baac0066282376ea P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/7e81d5780d9647b9baac0066282376ea
I20260202 02:52:16.324739 1183 ts_tablet_manager.cc:1950] T 7e81d5780d9647b9baac0066282376ea P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:16.327574 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_8cae34d2c6f1443ba07e3f2f8abd1989"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:16.328253 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8cae34d2c6f1443ba07e3f2f8abd1989 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:16.314158 1352 tablet.cc:1722] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:16.328497 1182 ts_tablet_manager.cc:1950] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:16.329514 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_8f23e3356aa14793a1aad274d2b845eb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:16.330118 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8f23e3356aa14793a1aad274d2b845eb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:16.330633 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 4f2a9404f7424dc78b5bd062ebd8cb53 (table default.Table_0842fd6a82e145e7a98b3fed2c0c6993 [id=4e076b2023be4c888ac1c27babde1607]) successfully deleted
I20260202 02:52:16.331600 523 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_563264f6196a4d6195a257c4ae9c9397" } new_table_name: "default.Table_a6b938c21e164294bd8b5fb205673f75" modify_external_catalogs: true
I20260202 02:52:16.332633 1269 raft_consensus.cc:493] T 23c38ef4e3c840808ec36f859cf6f6de P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:16.332828 1269 raft_consensus.cc:515] T 23c38ef4e3c840808ec36f859cf6f6de P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:16.334070 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "23c38ef4e3c840808ec36f859cf6f6de" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:16.334664 857 leader_election.cc:343] T 23c38ef4e3c840808ec36f859cf6f6de P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.337352 1333 mvcc.cc:204] Tried to move back new op lower bound from 7249923015882723328 to 7249923013562724352. Current Snapshot: MvccSnapshot[applied={T|T < 7249923015882723328}]
I20260202 02:52:16.337476 1173 ts_tablet_manager.cc:1434] T fae7e3941c124cc2871390a0b6471f27 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.021s user 0.002s sys 0.000s
I20260202 02:52:16.342423 1173 tablet_bootstrap.cc:492] T c2cc2250701d46a7abe7be03b11d45c8 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:16.340006 1362 tablet.cc:1722] T 80eb3013f5354022a2dd78bb294a8497 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260202 02:52:16.348508 524 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b76d1837813c46e8a0925b49939d8884 has no consensus state
I20260202 02:52:16.349483 524 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet fae7e3941c124cc2871390a0b6471f27 (table default.Table_3d26f7a34d0c48379e3cebda66e1ef13 [id=45e6d83589fa493195b06f633e295b00]). Expected version 2 got 0
I20260202 02:52:16.350190 524 master_service.cc:954] ReplaceTablet: received request to replace tablet 2ff6af17a6854451a1d7b904aa4f1f55 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:16.352239 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 7e81d5780d9647b9baac0066282376ea (table default.Table_629c3da327a740f684072799e959c2ca [id=30cd87ed1178421bb31ee622ce3254c1]) successfully deleted
I20260202 02:52:16.353468 1173 tablet_bootstrap.cc:654] T c2cc2250701d46a7abe7be03b11d45c8 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.353443 1362 tablet.cc:1722] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:16.362275 1184 raft_consensus.cc:493] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:16.362530 1184 raft_consensus.cc:515] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.363286 1180 ts_tablet_manager.cc:1929] T 72475ea720c6429aa68de9478b374280 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:16.363996 1184 leader_election.cc:290] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:16.364079 1180 log.cc:1199] T 72475ea720c6429aa68de9478b374280 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/72475ea720c6429aa68de9478b374280
I20260202 02:52:16.364523 1180 ts_tablet_manager.cc:1950] T 72475ea720c6429aa68de9478b374280 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:16.364598 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "098c5cdfcd0f4c52b6ae4f986f07705a" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:16.364964 1361 tablet.cc:1722] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260202 02:52:16.365180 857 leader_election.cc:343] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.365478 520 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 724d47241f0342768fb1321ef37c3a7b
I20260202 02:52:16.365677 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 72475ea720c6429aa68de9478b374280 (table default.Table_09d8f825c4c546839c0a820f1ddf0047 [id=ce957713e7fb49928d3fe57c3682f26f]) successfully deleted
I20260202 02:52:16.366191 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 724d47241f0342768fb1321ef37c3a7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:16 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:16.366264 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_4e6c3886f58b49698a00d45b957f4c9a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:16.366614 528 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet af9a1836e5214730a467e86f61165d8a
W20260202 02:52:16.366883 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4e6c3886f58b49698a00d45b957f4c9a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:16.366906 528 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_22489ecf56264d93993407ba7d0e6b55" } modify_external_catalogs: true
I20260202 02:52:16.367048 528 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_22489ecf56264d93993407ba7d0e6b55" } modify_external_catalogs: true
I20260202 02:52:16.370683 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "098c5cdfcd0f4c52b6ae4f986f07705a" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:16.370971 1180 tablet_replica.cc:333] T fc93352b24e345f99d15085e4886c748 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:16.371191 1180 raft_consensus.cc:2243] T fc93352b24e345f99d15085e4886c748 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
W20260202 02:52:16.371291 859 leader_election.cc:343] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.371387 1180 raft_consensus.cc:2272] T fc93352b24e345f99d15085e4886c748 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:16.371460 859 leader_election.cc:304] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:16.371991 1180 ts_tablet_manager.cc:1916] T fc93352b24e345f99d15085e4886c748 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:16.372042 1184 raft_consensus.cc:2749] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:16.372637 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 13309b20a96240b5aa9fbb691cbed981 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 13309b20a96240b5aa9fbb691cbed981 already in progress: creating tablet
I20260202 02:52:16.373170 901 tablet_service.cc:1552] Processing DeleteTablet for tablet 724d47241f0342768fb1321ef37c3a7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:16 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:16.373423 1182 tablet_replica.cc:333] T 724d47241f0342768fb1321ef37c3a7b P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:16.373612 1182 raft_consensus.cc:2243] T 724d47241f0342768fb1321ef37c3a7b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:16.373617 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 724d47241f0342768fb1321ef37c3a7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:16 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:16.373766 1182 raft_consensus.cc:2272] T 724d47241f0342768fb1321ef37c3a7b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:16.374404 1277 raft_consensus.cc:493] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:16.374557 1277 raft_consensus.cc:515] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.375260 1277 leader_election.cc:290] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:16.375962 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2ff6af17a6854451a1d7b904aa4f1f55" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:16.376437 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2ff6af17a6854451a1d7b904aa4f1f55" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
W20260202 02:52:16.376523 715 leader_election.cc:343] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.376646 923 raft_consensus.cc:2468] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
W20260202 02:52:16.377580 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet ea040c03420c4b1b98897f39160b4130 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ea040c03420c4b1b98897f39160b4130 already in progress: creating tablet
W20260202 02:52:16.378921 513 consensus_peers.cc:597] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b -> Peer a9c3e3612bb94fb7b94c308041acfaeb (127.0.104.188:39077): Couldn't send request to peer a9c3e3612bb94fb7b94c308041acfaeb. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.188:39077: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260202 02:52:16.379410 714 leader_election.cc:304] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:16.384608 1183 tablet_replica.cc:333] T e6b76967e5d3476997fa019c4a8afb91 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:16.384933 1183 raft_consensus.cc:2243] T e6b76967e5d3476997fa019c4a8afb91 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:16.384794 1180 ts_tablet_manager.cc:1929] T fc93352b24e345f99d15085e4886c748 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:16.385089 1180 log.cc:1199] T fc93352b24e345f99d15085e4886c748 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/fc93352b24e345f99d15085e4886c748
I20260202 02:52:16.385490 1180 ts_tablet_manager.cc:1950] T fc93352b24e345f99d15085e4886c748 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:16.386612 1180 tablet_replica.cc:333] T 4de41396cffd4d32b86c701b538d1e01 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:16.386832 1180 raft_consensus.cc:2243] T 4de41396cffd4d32b86c701b538d1e01 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:16.386991 1180 raft_consensus.cc:2272] T 4de41396cffd4d32b86c701b538d1e01 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:16.387835 1180 ts_tablet_manager.cc:1916] T 4de41396cffd4d32b86c701b538d1e01 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:16.388036 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet fc93352b24e345f99d15085e4886c748 (table default.Table_8aee8d27e02942fb91879229b190457f [id=c59468c8c5b7480d88cb4b7d447a5975]) successfully deleted
I20260202 02:52:16.389448 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "23c38ef4e3c840808ec36f859cf6f6de" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:16.389724 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet bf5ca99708044482970cb854d2eced18 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bf5ca99708044482970cb854d2eced18 already in progress: creating tablet
I20260202 02:52:16.389889 1269 leader_election.cc:290] T 23c38ef4e3c840808ec36f859cf6f6de P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
W20260202 02:52:16.390027 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet aa526a69cf914dd08142074636c956b7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aa526a69cf914dd08142074636c956b7 already in progress: creating tablet
I20260202 02:52:16.391048 1277 raft_consensus.cc:2804] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:16.391198 1277 raft_consensus.cc:493] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:16.391577 1182 ts_tablet_manager.cc:1916] T 724d47241f0342768fb1321ef37c3a7b P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:16.391707 859 leader_election.cc:343] T 23c38ef4e3c840808ec36f859cf6f6de P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.391865 859 leader_election.cc:304] T 23c38ef4e3c840808ec36f859cf6f6de P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:16.396522 1338 raft_consensus.cc:2749] T 23c38ef4e3c840808ec36f859cf6f6de P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:16.397281 526 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet fae7e3941c124cc2871390a0b6471f27 (table default.Table_3d26f7a34d0c48379e3cebda66e1ef13 [id=45e6d83589fa493195b06f633e295b00]). Expected version 2 got 0
I20260202 02:52:16.397630 1277 raft_consensus.cc:3060] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:16.398101 526 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_601ac359670d4a44a24253267b53a875" } modify_external_catalogs: true
I20260202 02:52:16.398295 526 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_601ac359670d4a44a24253267b53a875" } modify_external_catalogs: true
I20260202 02:52:16.399243 1183 raft_consensus.cc:2272] T e6b76967e5d3476997fa019c4a8afb91 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:16.399881 1183 ts_tablet_manager.cc:1916] T e6b76967e5d3476997fa019c4a8afb91 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:16.404978 1180 ts_tablet_manager.cc:1929] T 4de41396cffd4d32b86c701b538d1e01 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:16.405165 1180 log.cc:1199] T 4de41396cffd4d32b86c701b538d1e01 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/4de41396cffd4d32b86c701b538d1e01
I20260202 02:52:16.405525 1180 ts_tablet_manager.cc:1950] T 4de41396cffd4d32b86c701b538d1e01 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:16.407332 1246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260202 02:52:16.411222 519 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ef11a1ba065d483c9f7862345ab1aedb" } modify_external_catalogs: true
I20260202 02:52:16.411715 519 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ef11a1ba065d483c9f7862345ab1aedb" } modify_external_catalogs: true
W20260202 02:52:16.412130 523 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fe7b6004e5094c0dbe5847144a2489b9 has no consensus state
I20260202 02:52:16.412575 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_5c1272b44a5b4a84918e3c6c84dca87e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:16.413187 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5c1272b44a5b4a84918e3c6c84dca87e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:16.415062 1246 hybrid_clock.cc:648] HybridClock initialized: now 1770000736414868 us; error 135 us; skew 500 ppm
I20260202 02:52:16.415690 1246 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260202 02:52:16.417042 1174 tablet_bootstrap.cc:492] T c050a2882f764dda8df2a7df6547f149 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:16.417259 1174 ts_tablet_manager.cc:1403] T c050a2882f764dda8df2a7df6547f149 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.188s user 0.060s sys 0.042s
I20260202 02:52:16.418090 1174 raft_consensus.cc:359] T c050a2882f764dda8df2a7df6547f149 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.418299 1174 raft_consensus.cc:385] T c050a2882f764dda8df2a7df6547f149 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.418370 1174 raft_consensus.cc:740] T c050a2882f764dda8df2a7df6547f149 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.418605 1174 consensus_queue.cc:260] T c050a2882f764dda8df2a7df6547f149 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.419353 1174 ts_tablet_manager.cc:1434] T c050a2882f764dda8df2a7df6547f149 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:16.419734 1174 tablet_bootstrap.cc:492] T c1fc01252d95407e954cfcbf7e715912 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:16.422605 1174 tablet_bootstrap.cc:654] T c1fc01252d95407e954cfcbf7e715912 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.426209 1173 tablet_bootstrap.cc:492] T c2cc2250701d46a7abe7be03b11d45c8 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:16.428042 1173 ts_tablet_manager.cc:1403] T c2cc2250701d46a7abe7be03b11d45c8 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.086s user 0.004s sys 0.000s
I20260202 02:52:16.428628 1277 raft_consensus.cc:515] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.429132 1173 raft_consensus.cc:359] T c2cc2250701d46a7abe7be03b11d45c8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:16.429375 1173 raft_consensus.cc:385] T c2cc2250701d46a7abe7be03b11d45c8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.429473 1173 raft_consensus.cc:740] T c2cc2250701d46a7abe7be03b11d45c8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.429802 1173 consensus_queue.cc:260] T c2cc2250701d46a7abe7be03b11d45c8 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:16.430053 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2ff6af17a6854451a1d7b904aa4f1f55" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:16.431530 1174 tablet_bootstrap.cc:492] T c1fc01252d95407e954cfcbf7e715912 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:16.431692 1174 ts_tablet_manager.cc:1403] T c1fc01252d95407e954cfcbf7e715912 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.006s
I20260202 02:52:16.431715 1173 ts_tablet_manager.cc:1434] T c2cc2250701d46a7abe7be03b11d45c8 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260202 02:52:16.432632 1173 tablet_bootstrap.cc:492] T 58f09c607cce4ccc82afd75c002ed1a8 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
W20260202 02:52:16.432539 715 leader_election.cc:343] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.432528 1174 raft_consensus.cc:359] T c1fc01252d95407e954cfcbf7e715912 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.434301 1174 raft_consensus.cc:385] T c1fc01252d95407e954cfcbf7e715912 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.434413 1174 raft_consensus.cc:740] T c1fc01252d95407e954cfcbf7e715912 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.434826 1174 consensus_queue.cc:260] T c1fc01252d95407e954cfcbf7e715912 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.435760 1174 ts_tablet_manager.cc:1434] T c1fc01252d95407e954cfcbf7e715912 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260202 02:52:16.436075 1174 tablet_bootstrap.cc:492] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:16.436906 1338 raft_consensus.cc:493] T fc93352b24e345f99d15085e4886c748 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:16.437539 1338 raft_consensus.cc:515] T fc93352b24e345f99d15085e4886c748 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.438011 1173 tablet_bootstrap.cc:654] T 58f09c607cce4ccc82afd75c002ed1a8 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.438942 1174 tablet_bootstrap.cc:654] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.439304 1338 leader_election.cc:290] T fc93352b24e345f99d15085e4886c748 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:16.440073 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fc93352b24e345f99d15085e4886c748" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:16.440302 857 leader_election.cc:343] T fc93352b24e345f99d15085e4886c748 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Not found: Tablet not found: fc93352b24e345f99d15085e4886c748
W20260202 02:52:16.440935 859 leader_election.cc:343] T fc93352b24e345f99d15085e4886c748 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.441087 859 leader_election.cc:304] T fc93352b24e345f99d15085e4886c748 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:16.441817 1338 raft_consensus.cc:2749] T fc93352b24e345f99d15085e4886c748 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:16.442704 1359 tablet.cc:1722] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:16.447168 1324 raft_consensus.cc:493] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:16.447371 1324 raft_consensus.cc:515] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.448035 1324 leader_election.cc:290] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:16.451325 1361 tablet.cc:1722] T fae7e3941c124cc2871390a0b6471f27 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:16.456697 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_222f75e4260d41b2813393c25a2a6e14"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:16.457422 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_222f75e4260d41b2813393c25a2a6e14 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:16.459473 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c0937b6910534ddc8a2217f0cbd539c5" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:16.461146 715 leader_election.cc:343] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.466702 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c0937b6910534ddc8a2217f0cbd539c5" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:16.469683 923 raft_consensus.cc:2468] T c0937b6910534ddc8a2217f0cbd539c5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:16.474143 714 leader_election.cc:304] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:16.474541 1324 raft_consensus.cc:2804] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:16.474642 1324 raft_consensus.cc:493] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:16.474759 1324 raft_consensus.cc:3060] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:16.475945 527 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet 80eb3013f5354022a2dd78bb294a8497 (table default.Table_25c6d490f243450ab2d9b43ec9cda7bf [id=316b8a276d9b415fafe8511aa7c32fc5]). Expected version 2 got 0
I20260202 02:52:16.476195 527 catalog_manager.cc:5654] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f442ef85029461091de847118c34c0d (127.0.104.129). New cstate: current_term: 1 leader_uuid: "1f442ef85029461091de847118c34c0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } }
I20260202 02:52:16.476907 527 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet 251455168e33430aafea8e0de42ca1b8 (table default.Table_86f34199535c451cb54a2e20e70f1149 [id=4f19275f416a425c9b9d100b2a83870a]). Expected version 1 got 0
I20260202 02:52:16.477437 527 catalog_manager.cc:5654] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f442ef85029461091de847118c34c0d (127.0.104.129). New cstate: current_term: 1 leader_uuid: "1f442ef85029461091de847118c34c0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } }
I20260202 02:52:16.477867 527 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet ffd8aa2bc6e344b78dce518411997fb5 (table default.Table_ab8d74f30cd14c61b85950a82d0f3787 [id=3ccb047c30974c0f9cbe778971e61644]). Expected version 3 got 0
W20260202 02:52:16.478221 521 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c0937b6910534ddc8a2217f0cbd539c5 consensus state has no leader
I20260202 02:52:16.478186 1324 raft_consensus.cc:515] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.478837 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 4de41396cffd4d32b86c701b538d1e01 (table default.Table_d142ff2cb10e491997bf0e302d6a9fa5 [id=a0b8442c02974d6596db002c9292be0e]) successfully deleted
I20260202 02:52:16.478952 1324 leader_election.cc:290] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:16.479658 1180 tablet_replica.cc:333] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:16.479878 1180 raft_consensus.cc:2243] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:16.479911 521 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_9232ab3d803b45a8b491947852fc8a98" } new_table_name: "default.Table_dda472e63fac4873ab3ccefcac3635cd" modify_external_catalogs: true
I20260202 02:52:16.480155 1180 pending_rounds.cc:70] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d: Trying to abort 1 pending ops.
I20260202 02:52:16.480221 1180 pending_rounds.cc:77] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923015104425984 op_type: NO_OP noop_request { }
I20260202 02:52:16.480417 1180 raft_consensus.cc:2889] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:16.480527 1180 raft_consensus.cc:2272] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:16.482741 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c0937b6910534ddc8a2217f0cbd539c5" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:16.483071 1173 tablet_bootstrap.cc:492] T 58f09c607cce4ccc82afd75c002ed1a8 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:16.483227 1173 ts_tablet_manager.cc:1403] T 58f09c607cce4ccc82afd75c002ed1a8 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.051s user 0.005s sys 0.000s
W20260202 02:52:16.483578 715 leader_election.cc:343] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.484131 1173 raft_consensus.cc:359] T 58f09c607cce4ccc82afd75c002ed1a8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:16.484200 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c0937b6910534ddc8a2217f0cbd539c5" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:16.484365 1173 raft_consensus.cc:385] T 58f09c607cce4ccc82afd75c002ed1a8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.484464 1173 raft_consensus.cc:740] T 58f09c607cce4ccc82afd75c002ed1a8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.484472 923 raft_consensus.cc:3060] T c0937b6910534ddc8a2217f0cbd539c5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:16.484709 1173 consensus_queue.cc:260] T 58f09c607cce4ccc82afd75c002ed1a8 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:16.485332 1183 ts_tablet_manager.cc:1929] T e6b76967e5d3476997fa019c4a8afb91 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:16.485987 1183 log.cc:1199] T e6b76967e5d3476997fa019c4a8afb91 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/e6b76967e5d3476997fa019c4a8afb91
I20260202 02:52:16.486420 1183 ts_tablet_manager.cc:1950] T e6b76967e5d3476997fa019c4a8afb91 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:16.488134 1180 ts_tablet_manager.cc:1916] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:16.492664 923 raft_consensus.cc:2468] T c0937b6910534ddc8a2217f0cbd539c5 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:16.492803 1183 tablet_replica.cc:333] T fde2c1fba78248329f3662dd1036c7a2 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:16.493006 1183 raft_consensus.cc:2243] T fde2c1fba78248329f3662dd1036c7a2 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:16.493333 714 leader_election.cc:304] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:16.493917 1324 raft_consensus.cc:2804] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:16.494424 1324 raft_consensus.cc:697] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:16.494721 1324 consensus_queue.cc:237] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.495398 751 tablet_service.cc:1505] Processing CreateTablet for tablet 63a9029bc39c4fbd8ac4bf4bc37d6b4e (DEFAULT_TABLE table=default.Table_51cfbe7668e344b1bcca61f0ab935341 [id=fa95e04db2d443b5972de98c7c0e6457]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.495857 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63a9029bc39c4fbd8ac4bf4bc37d6b4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.498049 901 tablet_service.cc:1505] Processing CreateTablet for tablet c63a670d5c5e489897bc6dbb142b844d (DEFAULT_TABLE table=default.Table_fe76707af2ff46d1b44c46cef9e88be4 [id=7219522ffc3849faab94d754a42a8511]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.498579 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c63a670d5c5e489897bc6dbb142b844d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.498594 1183 raft_consensus.cc:2272] T fde2c1fba78248329f3662dd1036c7a2 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:16.499578 1183 ts_tablet_manager.cc:1916] T fde2c1fba78248329f3662dd1036c7a2 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:16.498178 1173 ts_tablet_manager.cc:1434] T 58f09c607cce4ccc82afd75c002ed1a8 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.015s user 0.001s sys 0.002s
I20260202 02:52:16.512748 1173 tablet_bootstrap.cc:492] T 2ff6af17a6854451a1d7b904aa4f1f55 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:16.516932 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet e6b76967e5d3476997fa019c4a8afb91 (table default.Table_e6d3c2dc7ad1416c949573796c5dc398 [id=a9429c7c44f144baa985c0731279243e]) successfully deleted
I20260202 02:52:16.517127 1044 tablet_service.cc:1505] Processing CreateTablet for tablet c63a670d5c5e489897bc6dbb142b844d (DEFAULT_TABLE table=default.Table_fe76707af2ff46d1b44c46cef9e88be4 [id=7219522ffc3849faab94d754a42a8511]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.517594 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c63a670d5c5e489897bc6dbb142b844d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:16.519138 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 952d760159844ad08269654264e6381b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 952d760159844ad08269654264e6381b already in progress: creating tablet
W20260202 02:52:16.519481 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 97bd7223d41d40a6bcdc60b1f63abc67 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 97bd7223d41d40a6bcdc60b1f63abc67 already in progress: creating tablet
I20260202 02:52:16.520399 749 tablet_service.cc:1505] Processing CreateTablet for tablet c63a670d5c5e489897bc6dbb142b844d (DEFAULT_TABLE table=default.Table_fe76707af2ff46d1b44c46cef9e88be4 [id=7219522ffc3849faab94d754a42a8511]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.520450 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 63a9029bc39c4fbd8ac4bf4bc37d6b4e (DEFAULT_TABLE table=default.Table_51cfbe7668e344b1bcca61f0ab935341 [id=fa95e04db2d443b5972de98c7c0e6457]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.520874 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63a9029bc39c4fbd8ac4bf4bc37d6b4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.521107 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c63a670d5c5e489897bc6dbb142b844d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.526069 1173 tablet_bootstrap.cc:654] T 2ff6af17a6854451a1d7b904aa4f1f55 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.527637 1180 ts_tablet_manager.cc:1929] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:16.527803 1180 log.cc:1199] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/672ea3f731ba42ff8b1ff47db039ed22
I20260202 02:52:16.528923 1180 ts_tablet_manager.cc:1950] T 672ea3f731ba42ff8b1ff47db039ed22 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:16.529479 1183 ts_tablet_manager.cc:1929] T fde2c1fba78248329f3662dd1036c7a2 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:16.529702 1183 log.cc:1199] T fde2c1fba78248329f3662dd1036c7a2 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/fde2c1fba78248329f3662dd1036c7a2
I20260202 02:52:16.530169 1183 ts_tablet_manager.cc:1950] T fde2c1fba78248329f3662dd1036c7a2 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:16.531009 898 tablet_service.cc:1505] Processing CreateTablet for tablet 63a9029bc39c4fbd8ac4bf4bc37d6b4e (DEFAULT_TABLE table=default.Table_51cfbe7668e344b1bcca61f0ab935341 [id=fa95e04db2d443b5972de98c7c0e6457]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.531525 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63a9029bc39c4fbd8ac4bf4bc37d6b4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.531507 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 672ea3f731ba42ff8b1ff47db039ed22 (table default.Table_85da33c295954de58e2c6290005ac865 [id=ed3c3f07e9ef4a568d36ebd62e0bddc5]) successfully deleted
W20260202 02:52:16.532253 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 4f2a9404f7424dc78b5bd062ebd8cb53 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4f2a9404f7424dc78b5bd062ebd8cb53 already in progress: creating tablet
W20260202 02:52:16.532941 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 724d47241f0342768fb1321ef37c3a7b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 724d47241f0342768fb1321ef37c3a7b already in progress: creating tablet
I20260202 02:52:16.535368 1173 tablet_bootstrap.cc:492] T 2ff6af17a6854451a1d7b904aa4f1f55 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:16.535529 1173 ts_tablet_manager.cc:1403] T 2ff6af17a6854451a1d7b904aa4f1f55 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.001s
I20260202 02:52:16.536355 1173 raft_consensus.cc:359] T 2ff6af17a6854451a1d7b904aa4f1f55 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.536556 1173 raft_consensus.cc:385] T 2ff6af17a6854451a1d7b904aa4f1f55 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.536620 1173 raft_consensus.cc:740] T 2ff6af17a6854451a1d7b904aa4f1f55 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.536872 1173 consensus_queue.cc:260] T 2ff6af17a6854451a1d7b904aa4f1f55 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.538592 1173 ts_tablet_manager.cc:1434] T 2ff6af17a6854451a1d7b904aa4f1f55 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260202 02:52:16.539105 1173 tablet_bootstrap.cc:492] T 72475ea720c6429aa68de9478b374280 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:16.541764 1173 tablet_bootstrap.cc:654] T 72475ea720c6429aa68de9478b374280 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.549325 1277 leader_election.cc:290] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:16.549430 922 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2ff6af17a6854451a1d7b904aa4f1f55" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:16.549710 922 raft_consensus.cc:3060] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:16.551167 751 tablet_service.cc:1505] Processing CreateTablet for tablet 583e547d0a1943f09f4efe82070fc777 (DEFAULT_TABLE table=default.Table_2669ca9662b942c08bcbfd7641cfe2e3 [id=fb813e4c5b0849f8a488c7a22d1defae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.551682 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 583e547d0a1943f09f4efe82070fc777. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.552223 756 tablet_service.cc:1505] Processing CreateTablet for tablet b76d1837813c46e8a0925b49939d8884 (DEFAULT_TABLE table=default.Table_3919d090261e4b90b0672f2b863588b2 [id=fe7b88b74c864c2f98365c0956117bf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.552676 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b76d1837813c46e8a0925b49939d8884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.552642 922 raft_consensus.cc:2468] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:16.556831 714 leader_election.cc:304] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:16.565451 1277 raft_consensus.cc:2804] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:16.565924 1277 raft_consensus.cc:697] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:16.566318 1277 consensus_queue.cc:237] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.573227 898 tablet_service.cc:1505] Processing CreateTablet for tablet 583e547d0a1943f09f4efe82070fc777 (DEFAULT_TABLE table=default.Table_2669ca9662b942c08bcbfd7641cfe2e3 [id=fb813e4c5b0849f8a488c7a22d1defae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.573860 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 583e547d0a1943f09f4efe82070fc777. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.582916 1182 ts_tablet_manager.cc:1929] T 724d47241f0342768fb1321ef37c3a7b P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:16.583191 1182 log.cc:1199] T 724d47241f0342768fb1321ef37c3a7b P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/724d47241f0342768fb1321ef37c3a7b
I20260202 02:52:16.583631 1182 ts_tablet_manager.cc:1950] T 724d47241f0342768fb1321ef37c3a7b P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:16.584966 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 724d47241f0342768fb1321ef37c3a7b (table default.Table_5c9c0f09396a4a0e8b3d676fcb2dbb2c [id=6a47f828bd9c4d7aa7d385e4a1dda9ea]) successfully deleted
I20260202 02:52:16.587877 751 tablet_service.cc:1505] Processing CreateTablet for tablet d2fa4c3d6ec44047abf5986e502c1590 (DEFAULT_TABLE table=default.Table_cf37f87385ee44cea1ff80cc14050ff2 [id=697b6fe62ee845d895af350881a03556]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.588433 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2fa4c3d6ec44047abf5986e502c1590. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.588510 903 tablet_service.cc:1505] Processing CreateTablet for tablet d2fa4c3d6ec44047abf5986e502c1590 (DEFAULT_TABLE table=default.Table_cf37f87385ee44cea1ff80cc14050ff2 [id=697b6fe62ee845d895af350881a03556]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.588930 903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2fa4c3d6ec44047abf5986e502c1590. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.596494 1338 raft_consensus.cc:493] T 5986676f5484498da927b4c6da3f47ff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:16.596707 1338 raft_consensus.cc:515] T 5986676f5484498da927b4c6da3f47ff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.597540 1338 leader_election.cc:290] T 5986676f5484498da927b4c6da3f47ff P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:16.598475 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet fde2c1fba78248329f3662dd1036c7a2 (table default.Table_9fd715491a6a4bbc8685138d0194362d [id=98ddcbfb5ad14ae480a54845ec7c45a4]) successfully deleted
I20260202 02:52:16.598544 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 583e547d0a1943f09f4efe82070fc777 (DEFAULT_TABLE table=default.Table_2669ca9662b942c08bcbfd7641cfe2e3 [id=fb813e4c5b0849f8a488c7a22d1defae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.598659 1038 tablet_service.cc:1505] Processing CreateTablet for tablet d2fa4c3d6ec44047abf5986e502c1590 (DEFAULT_TABLE table=default.Table_cf37f87385ee44cea1ff80cc14050ff2 [id=697b6fe62ee845d895af350881a03556]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:16.598726 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 4f2a9404f7424dc78b5bd062ebd8cb53 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4f2a9404f7424dc78b5bd062ebd8cb53 already in progress: creating tablet
W20260202 02:52:16.598928 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 724d47241f0342768fb1321ef37c3a7b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 724d47241f0342768fb1321ef37c3a7b already in progress: creating tablet
I20260202 02:52:16.599087 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 583e547d0a1943f09f4efe82070fc777. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.599854 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5986676f5484498da927b4c6da3f47ff" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:16.600466 857 leader_election.cc:343] T 5986676f5484498da927b4c6da3f47ff P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.601382 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_2ebdd65f856c4fd1881a573bcd3c46d4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:16.601994 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2ebdd65f856c4fd1881a573bcd3c46d4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:16.603600 525 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_25c6d490f243450ab2d9b43ec9cda7bf [id=316b8a276d9b415fafe8511aa7c32fc5] alter complete (version 2)
I20260202 02:52:16.604157 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_22dd5a0be7944811bf358ec584461157"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:16.604544 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5986676f5484498da927b4c6da3f47ff" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:16.604735 1044 tablet_service.cc:1505] Processing CreateTablet for tablet b76d1837813c46e8a0925b49939d8884 (DEFAULT_TABLE table=default.Table_3919d090261e4b90b0672f2b863588b2 [id=fe7b88b74c864c2f98365c0956117bf1]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:16.605214 859 leader_election.cc:343] T 5986676f5484498da927b4c6da3f47ff P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.605392 859 leader_election.cc:304] T 5986676f5484498da927b4c6da3f47ff P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:16.605512 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b76d1837813c46e8a0925b49939d8884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.605831 1338 raft_consensus.cc:2749] T 5986676f5484498da927b4c6da3f47ff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:16.608757 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2fa4c3d6ec44047abf5986e502c1590. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:16.609022 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_22dd5a0be7944811bf358ec584461157 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:16.612155 901 tablet_service.cc:1505] Processing CreateTablet for tablet b76d1837813c46e8a0925b49939d8884 (DEFAULT_TABLE table=default.Table_3919d090261e4b90b0672f2b863588b2 [id=fe7b88b74c864c2f98365c0956117bf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.612673 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b76d1837813c46e8a0925b49939d8884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.621127 751 tablet_service.cc:1505] Processing CreateTablet for tablet 723921dfe54c4e66af5cab76a25fee0a (DEFAULT_TABLE table=default.Table_38d2944c79ed4c609e92afac1acb0e5f [id=c30593053ed7490588ff567e30984807]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.621608 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 723921dfe54c4e66af5cab76a25fee0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.623447 1175 tablet_bootstrap.cc:492] T bf5ca99708044482970cb854d2eced18 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:16.623636 1175 ts_tablet_manager.cc:1403] T bf5ca99708044482970cb854d2eced18 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.352s user 0.016s sys 0.016s
I20260202 02:52:16.624779 1175 raft_consensus.cc:359] T bf5ca99708044482970cb854d2eced18 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.625047 1175 raft_consensus.cc:385] T bf5ca99708044482970cb854d2eced18 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.625142 1175 raft_consensus.cc:740] T bf5ca99708044482970cb854d2eced18 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.625409 1175 consensus_queue.cc:260] T bf5ca99708044482970cb854d2eced18 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.627666 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 723921dfe54c4e66af5cab76a25fee0a (DEFAULT_TABLE table=default.Table_38d2944c79ed4c609e92afac1acb0e5f [id=c30593053ed7490588ff567e30984807]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.628466 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 723921dfe54c4e66af5cab76a25fee0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.632368 756 tablet_service.cc:1505] Processing CreateTablet for tablet d06b77ff1a6f4f78b73082b00145fcf6 (DEFAULT_TABLE table=default.Table_4336d55393794a208c76f0a580420596 [id=915b211258ad498286448a5b2598a289]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.633157 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d06b77ff1a6f4f78b73082b00145fcf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.634716 903 tablet_service.cc:1505] Processing CreateTablet for tablet d06b77ff1a6f4f78b73082b00145fcf6 (DEFAULT_TABLE table=default.Table_4336d55393794a208c76f0a580420596 [id=915b211258ad498286448a5b2598a289]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.635191 903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d06b77ff1a6f4f78b73082b00145fcf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.651072 1246 webserver.cc:492] Webserver started at http://127.0.104.188:35523/ using document root <none> and password file <none>
I20260202 02:52:16.652237 1246 fs_manager.cc:362] Metadata directory not provided
I20260202 02:52:16.652474 1246 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260202 02:52:16.653869 897 tablet_service.cc:1505] Processing CreateTablet for tablet c184d44d7f434b39a997a7387e5d3de0 (DEFAULT_TABLE table=default.Table_9451b9f5148f4d9aab65983dea967e32 [id=9ef504467a544f299b48499e49adf27f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.654350 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c184d44d7f434b39a997a7387e5d3de0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.654732 751 tablet_service.cc:1505] Processing CreateTablet for tablet c12ff20ca6b44ddba0c936a28cfb2eb9 (DEFAULT_TABLE table=default.Table_c9d046b019d94f46bee8e3fd28d253ca [id=bdf2f19a439d4be3aa2362cfde38ab7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.655148 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c12ff20ca6b44ddba0c936a28cfb2eb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.655977 750 tablet_service.cc:1505] Processing CreateTablet for tablet fe7b6004e5094c0dbe5847144a2489b9 (DEFAULT_TABLE table=default.Table_a6b938c21e164294bd8b5fb205673f75 [id=1abad082d47d4f31be7c697ed72e241c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.656368 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe7b6004e5094c0dbe5847144a2489b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.659824 899 tablet_service.cc:1505] Processing CreateTablet for tablet c12ff20ca6b44ddba0c936a28cfb2eb9 (DEFAULT_TABLE table=default.Table_c9d046b019d94f46bee8e3fd28d253ca [id=bdf2f19a439d4be3aa2362cfde38ab7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.659904 900 tablet_service.cc:1505] Processing CreateTablet for tablet fe7b6004e5094c0dbe5847144a2489b9 (DEFAULT_TABLE table=default.Table_a6b938c21e164294bd8b5fb205673f75 [id=1abad082d47d4f31be7c697ed72e241c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.660277 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c12ff20ca6b44ddba0c936a28cfb2eb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.660478 1037 tablet_service.cc:1505] Processing CreateTablet for tablet d06b77ff1a6f4f78b73082b00145fcf6 (DEFAULT_TABLE table=default.Table_4336d55393794a208c76f0a580420596 [id=915b211258ad498286448a5b2598a289]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.660887 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d06b77ff1a6f4f78b73082b00145fcf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.664702 1044 tablet_service.cc:1505] Processing CreateTablet for tablet c184d44d7f434b39a997a7387e5d3de0 (DEFAULT_TABLE table=default.Table_9451b9f5148f4d9aab65983dea967e32 [id=9ef504467a544f299b48499e49adf27f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.664949 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe7b6004e5094c0dbe5847144a2489b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.665230 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c184d44d7f434b39a997a7387e5d3de0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.666432 1175 ts_tablet_manager.cc:1434] T bf5ca99708044482970cb854d2eced18 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.043s user 0.003s sys 0.000s
I20260202 02:52:16.666785 1175 tablet_bootstrap.cc:492] T 560465c063d94ffaab33f9f79a58a357 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:16.668673 1038 tablet_service.cc:1505] Processing CreateTablet for tablet c12ff20ca6b44ddba0c936a28cfb2eb9 (DEFAULT_TABLE table=default.Table_c9d046b019d94f46bee8e3fd28d253ca [id=bdf2f19a439d4be3aa2362cfde38ab7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.669071 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c12ff20ca6b44ddba0c936a28cfb2eb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.670719 1041 tablet_service.cc:1505] Processing CreateTablet for tablet fe7b6004e5094c0dbe5847144a2489b9 (DEFAULT_TABLE table=default.Table_a6b938c21e164294bd8b5fb205673f75 [id=1abad082d47d4f31be7c697ed72e241c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.671175 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe7b6004e5094c0dbe5847144a2489b9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:16.681404 512 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 2ff6af17a6854451a1d7b904aa4f1f55 (table default.Table_5eefa56618e84f8d92f8d87132ccd7d4 [id=b4ab2ba86e1a49338a98d855ded99f5d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 2ff6af17a6854451a1d7b904aa4f1f55 consensus state has no leader
I20260202 02:52:16.688113 1175 tablet_bootstrap.cc:654] T 560465c063d94ffaab33f9f79a58a357 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.689721 749 tablet_service.cc:1505] Processing CreateTablet for tablet c184d44d7f434b39a997a7387e5d3de0 (DEFAULT_TABLE table=default.Table_9451b9f5148f4d9aab65983dea967e32 [id=9ef504467a544f299b48499e49adf27f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.690234 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c184d44d7f434b39a997a7387e5d3de0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.693612 1173 tablet_bootstrap.cc:492] T 72475ea720c6429aa68de9478b374280 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:16.693785 1173 ts_tablet_manager.cc:1403] T 72475ea720c6429aa68de9478b374280 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.155s user 0.028s sys 0.036s
I20260202 02:52:16.694437 1173 raft_consensus.cc:359] T 72475ea720c6429aa68de9478b374280 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.694581 1173 raft_consensus.cc:385] T 72475ea720c6429aa68de9478b374280 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.694666 1173 raft_consensus.cc:740] T 72475ea720c6429aa68de9478b374280 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.694908 1173 consensus_queue.cc:260] T 72475ea720c6429aa68de9478b374280 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.695889 1037 tablet_service.cc:1505] Processing CreateTablet for tablet bbf31e5619ed4f60b4d7c8b19ec8a549 (DEFAULT_TABLE table=default.Table_ef11a1ba065d483c9f7862345ab1aedb [id=0e54d0abe11c46fa8dbf0befb46219cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.696291 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbf31e5619ed4f60b4d7c8b19ec8a549. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.695016 901 tablet_service.cc:1505] Processing CreateTablet for tablet 723921dfe54c4e66af5cab76a25fee0a (DEFAULT_TABLE table=default.Table_38d2944c79ed4c609e92afac1acb0e5f [id=c30593053ed7490588ff567e30984807]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.696272 751 tablet_service.cc:1505] Processing CreateTablet for tablet bbf31e5619ed4f60b4d7c8b19ec8a549 (DEFAULT_TABLE table=default.Table_ef11a1ba065d483c9f7862345ab1aedb [id=0e54d0abe11c46fa8dbf0befb46219cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.696772 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 723921dfe54c4e66af5cab76a25fee0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.697000 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbf31e5619ed4f60b4d7c8b19ec8a549. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.699242 898 tablet_service.cc:1505] Processing CreateTablet for tablet bbf31e5619ed4f60b4d7c8b19ec8a549 (DEFAULT_TABLE table=default.Table_ef11a1ba065d483c9f7862345ab1aedb [id=0e54d0abe11c46fa8dbf0befb46219cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.699484 1173 ts_tablet_manager.cc:1434] T 72475ea720c6429aa68de9478b374280 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260202 02:52:16.699630 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbf31e5619ed4f60b4d7c8b19ec8a549. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.700498 1173 tablet_bootstrap.cc:492] T fc93352b24e345f99d15085e4886c748 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:16.701903 1174 tablet_bootstrap.cc:492] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:16.702093 1174 ts_tablet_manager.cc:1403] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.266s user 0.049s sys 0.096s
I20260202 02:52:16.704123 1174 raft_consensus.cc:359] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.704465 1174 raft_consensus.cc:385] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.704559 1174 raft_consensus.cc:740] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.704828 1174 consensus_queue.cc:260] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.705379 1174 ts_tablet_manager.cc:1434] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260202 02:52:16.705940 1174 tablet_bootstrap.cc:492] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:16.709509 1174 tablet_bootstrap.cc:654] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.709614 1246 fs_manager.cc:714] Time spent opening directory manager: real 0.054s user 0.009s sys 0.004s
I20260202 02:52:16.710868 1173 tablet_bootstrap.cc:654] T fc93352b24e345f99d15085e4886c748 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.711005 526 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 098c5cdfcd0f4c52b6ae4f986f07705a
I20260202 02:52:16.712553 519 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet bbf31e5619ed4f60b4d7c8b19ec8a549
I20260202 02:52:16.714001 519 master_service.cc:954] ReplaceTablet: received request to replace tablet 2ff6af17a6854451a1d7b904aa4f1f55 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:16.714468 528 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 415d14533fd54f20adb00455a7bcad5f
I20260202 02:52:16.715144 528 master_service.cc:954] ReplaceTablet: received request to replace tablet cb0dcacdf0f1419da0b25f20cac1d62d from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:16.717270 750 tablet_service.cc:1552] Processing DeleteTablet for tablet 415d14533fd54f20adb00455a7bcad5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:16 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:16.717584 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 098c5cdfcd0f4c52b6ae4f986f07705a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:16 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:16.718098 1180 tablet_replica.cc:333] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:16.718299 1180 raft_consensus.cc:2243] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:16.718487 1180 raft_consensus.cc:2272] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:16.719329 1180 ts_tablet_manager.cc:1916] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:16.719957 522 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.104.130:36691 (request call id 31) took 915 ms (client timeout 1000 ms). Trace:
W20260202 02:52:16.721233 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 415d14533fd54f20adb00455a7bcad5f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 415d14533fd54f20adb00455a7bcad5f already in progress: creating tablet
I20260202 02:52:16.722339 750 tablet_service.cc:1505] Processing CreateTablet for tablet f58a7b25815c4b5bac568aa11c19610c (DEFAULT_TABLE table=default.Table_983bf3e0c5424c2588a0e754af1a5687 [id=c97e407e519d4a538860f66931f6aefd]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:16.720103 522 rpcz_store.cc:269] 0202 02:52:15.804093 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:15.860810 (+ 56717us) service_pool.cc:225] Handling call
0202 02:52:16.719882 (+859072us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:16.227027 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:16.227208 (+ 181us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:16.227212 (+ 4us) write_op.cc:435] Acquired schema lock
0202 02:52:16.227215 (+ 3us) tablet.cc:660] Decoding operations
0202 02:52:16.227371 (+ 156us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:16.227386 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:16.227392 (+ 6us) tablet.cc:683] Acquiring locks for 1 operations
0202 02:52:16.227511 (+ 119us) tablet.cc:699] Row locks acquired
0202 02:52:16.227514 (+ 3us) write_op.cc:260] PREPARE: finished
0202 02:52:16.227554 (+ 40us) write_op.cc:270] Start()
0202 02:52:16.227587 (+ 33us) write_op.cc:276] Timestamp: P: 1770000736227501 usec, L: 0
0202 02:52:16.227591 (+ 4us) op_driver.cc:348] REPLICATION: starting
0202 02:52:16.227803 (+ 212us) log.cc:844] Serialized 533 byte log entry
0202 02:52:16.231172 (+ 3369us) op_driver.cc:464] REPLICATION: finished
0202 02:52:16.231312 (+ 140us) write_op.cc:301] APPLY: starting
0202 02:52:16.231330 (+ 18us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:16.231383 (+ 53us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:16.231409 (+ 26us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:16.231464 (+ 55us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:16.231497 (+ 33us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0202 02:52:16.231504 (+ 7us) write_op.cc:312] APPLY: finished
0202 02:52:16.231587 (+ 83us) log.cc:844] Serialized 23 byte log entry
0202 02:52:16.231657 (+ 70us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:16.231676 (+ 19us) write_op.cc:454] Released schema lock
0202 02:52:16.231684 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0202 02:52:16.288169 (+ 56485us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:16.288178 (+ 9us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":17,"child_traces":[["op",{"apply.queue_time_us":92,"apply.run_cpu_time_us":430,"apply.run_wall_time_us":430,"num_ops":1,"prepare.queue_time_us":105930,"prepare.run_cpu_time_us":815,"prepare.run_wall_time_us":61136,"raft.queue_time_us":61,"raft.run_cpu_time_us":291,"raft.run_wall_time_us":290,"replication_time_us":3551}]]}
I20260202 02:52:16.722868 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f58a7b25815c4b5bac568aa11c19610c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.723075 522 master_service.cc:954] ReplaceTablet: received request to replace tablet 724d47241f0342768fb1321ef37c3a7b from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:16.723166 749 tablet_service.cc:1552] Processing DeleteTablet for tablet bbf31e5619ed4f60b4d7c8b19ec8a549 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:16 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:16.725153 749 tablet_service.cc:1505] Processing CreateTablet for tablet bc1539e5c8aa4d6e98f687728b891530 (DEFAULT_TABLE table=default.Table_f1e417bf19b94bc491c3937276c669fd [id=9a8a252de32848288c9b3752c7d3feb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.726637 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc1539e5c8aa4d6e98f687728b891530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.727669 758 tablet_service.cc:1505] Processing CreateTablet for tablet c064ea0dade840b4840a5715c2d83f69 (DEFAULT_TABLE table=default.Table_5fa327d665484811934aa3b11188a76d [id=f7c5b24c065443d89a2b1f9c34409c9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.728082 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c064ea0dade840b4840a5715c2d83f69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.729159 754 tablet_service.cc:1505] Processing CreateTablet for tablet 0d3633f5f9f54115bc78b9c1b78aba67 (DEFAULT_TABLE table=default.Table_63bb1b19c34a430e94ad0d7bd9bbe795 [id=af3058593c2747bf99db3b8da1aaedc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.729605 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d3633f5f9f54115bc78b9c1b78aba67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.730459 897 tablet_service.cc:1552] Processing DeleteTablet for tablet bbf31e5619ed4f60b4d7c8b19ec8a549 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:16 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:16.731063 899 tablet_service.cc:1552] Processing DeleteTablet for tablet 415d14533fd54f20adb00455a7bcad5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:16 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:16.731257 900 tablet_service.cc:1552] Processing DeleteTablet for tablet 098c5cdfcd0f4c52b6ae4f986f07705a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:16 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:16.731761 903 tablet_service.cc:1505] Processing CreateTablet for tablet 267bd4752c5f4b99a3fe778ba8f38a2f (DEFAULT_TABLE table=default.Table_9232ab3d803b45a8b491947852fc8a98 [id=40105de1d8074d3e983ff62557d14f48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.731045 1173 tablet_bootstrap.cc:492] T fc93352b24e345f99d15085e4886c748 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:16.732098 1173 ts_tablet_manager.cc:1403] T fc93352b24e345f99d15085e4886c748 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.032s user 0.008s sys 0.001s
I20260202 02:52:16.732936 1173 raft_consensus.cc:359] T fc93352b24e345f99d15085e4886c748 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.733687 1173 raft_consensus.cc:385] T fc93352b24e345f99d15085e4886c748 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.733806 1173 raft_consensus.cc:740] T fc93352b24e345f99d15085e4886c748 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.734009 896 tablet_service.cc:1552] Processing DeleteTablet for tablet 952d760159844ad08269654264e6381b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:16.734057 1173 consensus_queue.cc:260] T fc93352b24e345f99d15085e4886c748 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.734259 899 tablet_service.cc:1552] Processing DeleteTablet for tablet c1fc01252d95407e954cfcbf7e715912 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:16.734408 900 tablet_service.cc:1552] Processing DeleteTablet for tablet 575016c0e9f84e99b962c5f14ffa96cc with delete_type TABLET_DATA_DELETED (replaced by tablet ff6f5710de4b4c48948aacdf5be29e8b) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:16.735071 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 267bd4752c5f4b99a3fe778ba8f38a2f (DEFAULT_TABLE table=default.Table_9232ab3d803b45a8b491947852fc8a98 [id=40105de1d8074d3e983ff62557d14f48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.735464 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 267bd4752c5f4b99a3fe778ba8f38a2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.737090 756 tablet_service.cc:1505] Processing CreateTablet for tablet 267bd4752c5f4b99a3fe778ba8f38a2f (DEFAULT_TABLE table=default.Table_9232ab3d803b45a8b491947852fc8a98 [id=40105de1d8074d3e983ff62557d14f48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.738283 1379 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260202 02:52:16.739382 1173 ts_tablet_manager.cc:1434] T fc93352b24e345f99d15085e4886c748 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260202 02:52:16.739912 1173 tablet_bootstrap.cc:492] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:16.740484 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet bbf31e5619ed4f60b4d7c8b19ec8a549 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:16 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:16.740751 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet 415d14533fd54f20adb00455a7bcad5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:16 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:16.740967 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet 098c5cdfcd0f4c52b6ae4f986f07705a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:16 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:16.741358 1039 tablet_service.cc:1505] Processing CreateTablet for tablet 0d3633f5f9f54115bc78b9c1b78aba67 (DEFAULT_TABLE table=default.Table_63bb1b19c34a430e94ad0d7bd9bbe795 [id=af3058593c2747bf99db3b8da1aaedc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.741779 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d3633f5f9f54115bc78b9c1b78aba67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.742961 1173 tablet_bootstrap.cc:654] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.745576 1246 fs_manager.cc:730] Time spent opening block manager: real 0.034s user 0.008s sys 0.005s
I20260202 02:52:16.745647 897 tablet_service.cc:1552] Processing DeleteTablet for tablet 97bd7223d41d40a6bcdc60b1f63abc67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:16.745913 1182 tablet_replica.cc:333] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:16.746105 1182 raft_consensus.cc:2243] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:16.746694 1182 raft_consensus.cc:2272] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:16.747313 1182 ts_tablet_manager.cc:1916] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:16.746251 1246 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-2/data,/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-2/wal
uuid: "a9c3e3612bb94fb7b94c308041acfaeb"
format_stamp: "Formatted at 2026-02-02 02:52:11 on dist-test-slave-llj2"
I20260202 02:52:16.747418 1175 tablet_bootstrap.cc:492] T 560465c063d94ffaab33f9f79a58a357 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:16.747447 1246 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260202 02:52:16.747835 524 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 2ff6af17a6854451a1d7b904aa4f1f55
I20260202 02:52:16.747854 1175 ts_tablet_manager.cc:1403] T 560465c063d94ffaab33f9f79a58a357 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.081s user 0.003s sys 0.001s
I20260202 02:52:16.747992 903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 267bd4752c5f4b99a3fe778ba8f38a2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.748425 524 catalog_manager.cc:6724] ReplaceTablet: tablet 2ff6af17a6854451a1d7b904aa4f1f55 deleted and replaced by tablet 78f122342dd144b48de89902faa0f94f
I20260202 02:52:16.749253 900 tablet_service.cc:1552] Processing DeleteTablet for tablet fc93352b24e345f99d15085e4886c748 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:16.750240 899 tablet_service.cc:1552] Processing DeleteTablet for tablet 13309b20a96240b5aa9fbb691cbed981 with delete_type TABLET_DATA_DELETED (replaced by tablet c064ea0dade840b4840a5715c2d83f69) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:16.750635 900 tablet_service.cc:1552] Processing DeleteTablet for tablet add934bf70ae4de8a40066f06fd94795 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:16.751051 1175 raft_consensus.cc:359] T 560465c063d94ffaab33f9f79a58a357 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.751377 1175 raft_consensus.cc:385] T 560465c063d94ffaab33f9f79a58a357 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.751477 1175 raft_consensus.cc:740] T 560465c063d94ffaab33f9f79a58a357 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.751777 1175 consensus_queue.cc:260] T 560465c063d94ffaab33f9f79a58a357 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.752259 1175 ts_tablet_manager.cc:1434] T 560465c063d94ffaab33f9f79a58a357 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260202 02:52:16.752820 900 tablet_service.cc:1552] Processing DeleteTablet for tablet f24c0b99574d437a99b273204f105460 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:16.753559 519 master_service.cc:954] ReplaceTablet: received request to replace tablet 0d297b6243c348ee82c146a8339cf6c4 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:16.753860 522 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_38d2944c79ed4c609e92afac1acb0e5f" } new_table_name: "default.Table_bdfbf5f796b949e196b1c1609fbb3655" modify_external_catalogs: true
I20260202 02:52:16.754413 896 tablet_service.cc:1505] Processing CreateTablet for tablet f58a7b25815c4b5bac568aa11c19610c (DEFAULT_TABLE table=default.Table_983bf3e0c5424c2588a0e754af1a5687 [id=c97e407e519d4a538860f66931f6aefd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.754588 526 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_8dc3d64cfa464a00864bf7850006f1f0" } new_table_name: "default.Table_fcd378fcc866495ea6b021dc22247351" modify_external_catalogs: true
I20260202 02:52:16.755059 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f58a7b25815c4b5bac568aa11c19610c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.755015 899 tablet_service.cc:1505] Processing CreateTablet for tablet 0d3633f5f9f54115bc78b9c1b78aba67 (DEFAULT_TABLE table=default.Table_63bb1b19c34a430e94ad0d7bd9bbe795 [id=af3058593c2747bf99db3b8da1aaedc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.755451 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d3633f5f9f54115bc78b9c1b78aba67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.755692 895 tablet_service.cc:1552] Processing DeleteTablet for tablet 2ff6af17a6854451a1d7b904aa4f1f55 with delete_type TABLET_DATA_DELETED (replaced by tablet 78f122342dd144b48de89902faa0f94f) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:16.756214 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet bbf31e5619ed4f60b4d7c8b19ec8a549 because the tablet was not found. No further retry: Not found: Tablet not found: bbf31e5619ed4f60b4d7c8b19ec8a549
I20260202 02:52:16.756469 900 tablet_service.cc:1505] Processing CreateTablet for tablet bc1539e5c8aa4d6e98f687728b891530 (DEFAULT_TABLE table=default.Table_f1e417bf19b94bc491c3937276c669fd [id=9a8a252de32848288c9b3752c7d3feb7]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:16.756526 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 415d14533fd54f20adb00455a7bcad5f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 415d14533fd54f20adb00455a7bcad5f already in progress: creating tablet
I20260202 02:52:16.756824 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc1539e5c8aa4d6e98f687728b891530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.757004 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_38c5bca682e343c1be1ebdc253c9e848"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:16.757531 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_38c5bca682e343c1be1ebdc253c9e848 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:16.756147 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_b0e344dc1a824cd9b41aa8991b7911dd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:16.759088 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b0e344dc1a824cd9b41aa8991b7911dd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:16.759725 897 tablet_service.cc:1552] Processing DeleteTablet for tablet 672ea3f731ba42ff8b1ff47db039ed22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:16.760864 1175 tablet_bootstrap.cc:492] T bf74f96bea2e4ab7adec42c3c5207e79 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:16.762712 1175 tablet_bootstrap.cc:654] T bf74f96bea2e4ab7adec42c3c5207e79 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.764779 1038 tablet_service.cc:1505] Processing CreateTablet for tablet f58a7b25815c4b5bac568aa11c19610c (DEFAULT_TABLE table=default.Table_983bf3e0c5424c2588a0e754af1a5687 [id=c97e407e519d4a538860f66931f6aefd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.765719 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f58a7b25815c4b5bac568aa11c19610c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:16.766974 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet bbf31e5619ed4f60b4d7c8b19ec8a549 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bbf31e5619ed4f60b4d7c8b19ec8a549 already in progress: creating tablet
W20260202 02:52:16.767310 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 415d14533fd54f20adb00455a7bcad5f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 415d14533fd54f20adb00455a7bcad5f already in progress: creating tablet
W20260202 02:52:16.767519 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 098c5cdfcd0f4c52b6ae4f986f07705a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 098c5cdfcd0f4c52b6ae4f986f07705a already in progress: creating tablet
I20260202 02:52:16.768054 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 267bd4752c5f4b99a3fe778ba8f38a2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.769217 1043 tablet_service.cc:1552] Processing DeleteTablet for tablet c2cc2250701d46a7abe7be03b11d45c8 with delete_type TABLET_DATA_DELETED (replaced by tablet 0d297b6243c348ee82c146a8339cf6c4) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:16.770004 1042 tablet_service.cc:1552] Processing DeleteTablet for tablet 58f09c607cce4ccc82afd75c002ed1a8 with delete_type TABLET_DATA_DELETED (replaced by tablet 560465c063d94ffaab33f9f79a58a357) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:16.770219 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 2ff6af17a6854451a1d7b904aa4f1f55 with delete_type TABLET_DATA_DELETED (replaced by tablet 78f122342dd144b48de89902faa0f94f) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:16.770313 1041 tablet_service.cc:1505] Processing CreateTablet for tablet bc1539e5c8aa4d6e98f687728b891530 (DEFAULT_TABLE table=default.Table_f1e417bf19b94bc491c3937276c669fd [id=9a8a252de32848288c9b3752c7d3feb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.770569 1183 tablet_replica.cc:333] T c2cc2250701d46a7abe7be03b11d45c8 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:16.771173 1183 raft_consensus.cc:2243] T c2cc2250701d46a7abe7be03b11d45c8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:16.771375 1183 raft_consensus.cc:2272] T c2cc2250701d46a7abe7be03b11d45c8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:16.772892 1183 ts_tablet_manager.cc:1916] T c2cc2250701d46a7abe7be03b11d45c8 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:16.775512 752 tablet_service.cc:1552] Processing DeleteTablet for tablet add934bf70ae4de8a40066f06fd94795 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:16.775703 1180 ts_tablet_manager.cc:1929] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:16.775512 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 97bd7223d41d40a6bcdc60b1f63abc67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:16.775846 1180 log.cc:1199] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/098c5cdfcd0f4c52b6ae4f986f07705a
I20260202 02:52:16.775780 753 tablet_service.cc:1552] Processing DeleteTablet for tablet 672ea3f731ba42ff8b1ff47db039ed22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:16.776432 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc1539e5c8aa4d6e98f687728b891530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.776692 750 tablet_service.cc:1552] Processing DeleteTablet for tablet 2ff6af17a6854451a1d7b904aa4f1f55 with delete_type TABLET_DATA_DELETED (replaced by tablet 78f122342dd144b48de89902faa0f94f) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:16.777160 755 tablet_service.cc:1552] Processing DeleteTablet for tablet f24c0b99574d437a99b273204f105460 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:16.784426 1180 ts_tablet_manager.cc:1950] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:16.785684 1180 tablet_replica.cc:333] T add934bf70ae4de8a40066f06fd94795 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:16.786350 1180 raft_consensus.cc:2243] T add934bf70ae4de8a40066f06fd94795 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:16.786541 1180 raft_consensus.cc:2272] T add934bf70ae4de8a40066f06fd94795 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:16.787159 1180 ts_tablet_manager.cc:1916] T add934bf70ae4de8a40066f06fd94795 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:16.790431 521 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 267bd4752c5f4b99a3fe778ba8f38a2f consensus state has no leader
I20260202 02:52:16.798179 521 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_d142ff2cb10e491997bf0e302d6a9fa5" } modify_external_catalogs: true
I20260202 02:52:16.798390 521 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_d142ff2cb10e491997bf0e302d6a9fa5" } modify_external_catalogs: true
I20260202 02:52:16.801261 1180 ts_tablet_manager.cc:1929] T add934bf70ae4de8a40066f06fd94795 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:16.801424 1180 log.cc:1199] T add934bf70ae4de8a40066f06fd94795 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/add934bf70ae4de8a40066f06fd94795
I20260202 02:52:16.801780 1180 ts_tablet_manager.cc:1950] T add934bf70ae4de8a40066f06fd94795 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:16.802908 1180 tablet_replica.cc:333] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:16.803119 1180 raft_consensus.cc:2243] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:16.803325 1180 raft_consensus.cc:2272] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:16.803876 1180 ts_tablet_manager.cc:1916] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:16.808530 1174 tablet_bootstrap.cc:492] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:16.808713 1174 ts_tablet_manager.cc:1403] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.103s user 0.002s sys 0.014s
I20260202 02:52:16.809511 525 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_dcf32fa66b524c84877c4668489c9962" } modify_external_catalogs: true
I20260202 02:52:16.809556 1174 raft_consensus.cc:359] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.809724 1174 raft_consensus.cc:385] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.809721 525 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_dcf32fa66b524c84877c4668489c9962" } modify_external_catalogs: true
I20260202 02:52:16.809801 1174 raft_consensus.cc:740] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.810530 1174 consensus_queue.cc:260] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.811204 1174 ts_tablet_manager.cc:1434] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:16.813344 1182 ts_tablet_manager.cc:1929] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:16.813477 1182 log.cc:1199] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/098c5cdfcd0f4c52b6ae4f986f07705a
I20260202 02:52:16.811699 1174 tablet_bootstrap.cc:492] T aa526a69cf914dd08142074636c956b7 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:16.815523 1182 ts_tablet_manager.cc:1950] T 098c5cdfcd0f4c52b6ae4f986f07705a P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:16.818965 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 098c5cdfcd0f4c52b6ae4f986f07705a (table default.Table_601ac359670d4a44a24253267b53a875 [id=cae0773031e040a2972ccb42c18f2267]) successfully deleted
W20260202 02:52:16.819231 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet bbf31e5619ed4f60b4d7c8b19ec8a549 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bbf31e5619ed4f60b4d7c8b19ec8a549 already in progress: creating tablet
I20260202 02:52:16.819502 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet add934bf70ae4de8a40066f06fd94795 (table default.Table_9fb69280bd6048d898c8e92f2ef9f61e [id=7cf1e30412cb48ab962f526efee72121]) successfully deleted
W20260202 02:52:16.819671 513 catalog_manager.cc:4952] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 672ea3f731ba42ff8b1ff47db039ed22 because the tablet was not found. No further retry: Not found: Tablet not found: 672ea3f731ba42ff8b1ff47db039ed22
I20260202 02:52:16.819990 1182 tablet_replica.cc:333] T c1fc01252d95407e954cfcbf7e715912 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:16.820225 1182 raft_consensus.cc:2243] T c1fc01252d95407e954cfcbf7e715912 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:16.820425 1182 raft_consensus.cc:2272] T c1fc01252d95407e954cfcbf7e715912 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:16.820897 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 098c5cdfcd0f4c52b6ae4f986f07705a (table default.Table_601ac359670d4a44a24253267b53a875 [id=cae0773031e040a2972ccb42c18f2267]) successfully deleted
I20260202 02:52:16.821151 1182 ts_tablet_manager.cc:1916] T c1fc01252d95407e954cfcbf7e715912 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:16.821205 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 952d760159844ad08269654264e6381b because the tablet was not found. No further retry: Not found: Tablet not found: 952d760159844ad08269654264e6381b
I20260202 02:52:16.823657 1042 tablet_service.cc:1552] Processing DeleteTablet for tablet fc93352b24e345f99d15085e4886c748 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:16.823879 1174 tablet_bootstrap.cc:654] T aa526a69cf914dd08142074636c956b7 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.824121 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 72475ea720c6429aa68de9478b374280 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:16.829929 1338 raft_consensus.cc:493] T 0d297b6243c348ee82c146a8339cf6c4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:16.830117 1338 raft_consensus.cc:515] T 0d297b6243c348ee82c146a8339cf6c4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:16.830650 1175 tablet_bootstrap.cc:492] T bf74f96bea2e4ab7adec42c3c5207e79 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:16.830817 1338 leader_election.cc:290] T 0d297b6243c348ee82c146a8339cf6c4 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:16.830837 1175 ts_tablet_manager.cc:1403] T bf74f96bea2e4ab7adec42c3c5207e79 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.078s user 0.005s sys 0.001s
I20260202 02:52:16.831459 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0d297b6243c348ee82c146a8339cf6c4" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:16.831539 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0d297b6243c348ee82c146a8339cf6c4" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:16.831734 1175 raft_consensus.cc:359] T bf74f96bea2e4ab7adec42c3c5207e79 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:16.831934 1175 raft_consensus.cc:385] T bf74f96bea2e4ab7adec42c3c5207e79 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.832041 1175 raft_consensus.cc:740] T bf74f96bea2e4ab7adec42c3c5207e79 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
W20260202 02:52:16.832710 859 leader_election.cc:343] T 0d297b6243c348ee82c146a8339cf6c4 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.832646 1175 consensus_queue.cc:260] T bf74f96bea2e4ab7adec42c3c5207e79 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
W20260202 02:52:16.832877 857 leader_election.cc:343] T 0d297b6243c348ee82c146a8339cf6c4 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.833045 857 leader_election.cc:304] T 0d297b6243c348ee82c146a8339cf6c4 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:16.833297 1338 raft_consensus.cc:2749] T 0d297b6243c348ee82c146a8339cf6c4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:16.833618 1175 ts_tablet_manager.cc:1434] T bf74f96bea2e4ab7adec42c3c5207e79 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260202 02:52:16.834237 1180 ts_tablet_manager.cc:1929] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:16.834373 1180 log.cc:1199] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/97bd7223d41d40a6bcdc60b1f63abc67
I20260202 02:52:16.837250 1180 ts_tablet_manager.cc:1950] T 97bd7223d41d40a6bcdc60b1f63abc67 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:16.838580 1183 ts_tablet_manager.cc:1929] T c2cc2250701d46a7abe7be03b11d45c8 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:16.838728 1180 tablet_replica.cc:333] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:16.838954 1182 ts_tablet_manager.cc:1929] T c1fc01252d95407e954cfcbf7e715912 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:16.838945 1180 raft_consensus.cc:2243] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:16.839371 1180 pending_rounds.cc:70] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d: Trying to abort 1 pending ops.
I20260202 02:52:16.839491 1180 pending_rounds.cc:77] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923016978866176 op_type: NO_OP noop_request { }
W20260202 02:52:16.839373 715 consensus_peers.cc:597] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:16.843752 1174 tablet_bootstrap.cc:492] T aa526a69cf914dd08142074636c956b7 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:16.843958 1174 ts_tablet_manager.cc:1403] T aa526a69cf914dd08142074636c956b7 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.032s user 0.006s sys 0.001s
I20260202 02:52:16.844791 1174 raft_consensus.cc:359] T aa526a69cf914dd08142074636c956b7 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.839625 1180 raft_consensus.cc:2889] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:16.845037 1174 raft_consensus.cc:385] T aa526a69cf914dd08142074636c956b7 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.839422 1175 tablet_bootstrap.cc:492] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:16.845249 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 97bd7223d41d40a6bcdc60b1f63abc67 (table default.Table_53eaadb859904d6b95ef0e536019ad79 [id=0ca5aac09bb54feba75b2d3784a69140]) successfully deleted
I20260202 02:52:16.845289 1174 raft_consensus.cc:740] T aa526a69cf914dd08142074636c956b7 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.845343 1180 raft_consensus.cc:2272] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:16.845556 1174 consensus_queue.cc:260] T aa526a69cf914dd08142074636c956b7 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.845995 923 raft_consensus.cc:1275] T 251455168e33430aafea8e0de42ca1b8 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Refusing update from remote peer 1f442ef85029461091de847118c34c0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260202 02:52:16.846331 1174 ts_tablet_manager.cc:1434] T aa526a69cf914dd08142074636c956b7 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:16.847440 1324 consensus_queue.cc:1048] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:16.847892 1175 tablet_bootstrap.cc:654] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.847920 1180 ts_tablet_manager.cc:1916] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:16.849709 1174 tablet_bootstrap.cc:492] T 952d760159844ad08269654264e6381b P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:16.852775 1183 log.cc:1199] T c2cc2250701d46a7abe7be03b11d45c8 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/c2cc2250701d46a7abe7be03b11d45c8
I20260202 02:52:16.854159 896 tablet_service.cc:1505] Processing CreateTablet for tablet c064ea0dade840b4840a5715c2d83f69 (DEFAULT_TABLE table=default.Table_5fa327d665484811934aa3b11188a76d [id=f7c5b24c065443d89a2b1f9c34409c9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.854594 1183 ts_tablet_manager.cc:1950] T c2cc2250701d46a7abe7be03b11d45c8 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:16.854898 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c064ea0dade840b4840a5715c2d83f69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.855278 1174 tablet_bootstrap.cc:654] T 952d760159844ad08269654264e6381b P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.855695 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet c2cc2250701d46a7abe7be03b11d45c8 (table default.Table_d210d1530706429197d6294b29f18c87 [id=7c76848b98614fbf8b2950e5d55cc609]) successfully deleted
I20260202 02:52:16.848222 1041 tablet_service.cc:1505] Processing CreateTablet for tablet c064ea0dade840b4840a5715c2d83f69 (DEFAULT_TABLE table=default.Table_5fa327d665484811934aa3b11188a76d [id=f7c5b24c065443d89a2b1f9c34409c9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:16.856207 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c064ea0dade840b4840a5715c2d83f69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:16.862573 1338 raft_consensus.cc:493] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:16.862718 1338 raft_consensus.cc:515] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:16.863384 1338 leader_election.cc:290] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:16.864068 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bd7b0a0fcb7645fdb9edfeccb3d18268" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:16.864303 1174 tablet_bootstrap.cc:492] T 952d760159844ad08269654264e6381b P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:16.864488 1174 ts_tablet_manager.cc:1403] T 952d760159844ad08269654264e6381b P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.001s
I20260202 02:52:16.864609 528 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet cb0dcacdf0f1419da0b25f20cac1d62d
W20260202 02:52:16.864707 859 leader_election.cc:343] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.865090 528 catalog_manager.cc:6724] ReplaceTablet: tablet cb0dcacdf0f1419da0b25f20cac1d62d deleted and replaced by tablet d67e91a022f1401ea724dff898c3b81b
I20260202 02:52:16.865326 1174 raft_consensus.cc:359] T 952d760159844ad08269654264e6381b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:16.862432 1182 log.cc:1199] T c1fc01252d95407e954cfcbf7e715912 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/c1fc01252d95407e954cfcbf7e715912
I20260202 02:52:16.865491 1174 raft_consensus.cc:385] T 952d760159844ad08269654264e6381b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.865568 1174 raft_consensus.cc:740] T 952d760159844ad08269654264e6381b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.865769 1174 consensus_queue.cc:260] T 952d760159844ad08269654264e6381b P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:16.865950 1182 ts_tablet_manager.cc:1950] T c1fc01252d95407e954cfcbf7e715912 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:16.866803 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bd7b0a0fcb7645fdb9edfeccb3d18268" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:16.867924 857 leader_election.cc:343] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.868186 857 leader_election.cc:304] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:16.868234 896 tablet_service.cc:1552] Processing DeleteTablet for tablet cb0dcacdf0f1419da0b25f20cac1d62d with delete_type TABLET_DATA_DELETED (replaced by tablet d67e91a022f1401ea724dff898c3b81b) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:16.868551 1338 raft_consensus.cc:2749] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:16.869248 1174 ts_tablet_manager.cc:1434] T 952d760159844ad08269654264e6381b P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260202 02:52:16.869613 1174 tablet_bootstrap.cc:492] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:16.870622 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet c1fc01252d95407e954cfcbf7e715912 (table default.Table_f23c311557a14f4a8c66eaf4af7a8a92 [id=10880639cfc743fcafecc538fa2ecd85]) successfully deleted
I20260202 02:52:16.870863 1182 tablet_replica.cc:333] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:16.871400 1182 raft_consensus.cc:2243] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:16.871434 1174 tablet_bootstrap.cc:654] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.871690 1182 pending_rounds.cc:70] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5: Trying to abort 1 pending ops.
I20260202 02:52:16.871757 1182 pending_rounds.cc:77] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923015049121792 op_type: NO_OP noop_request { }
I20260202 02:52:16.871893 1182 raft_consensus.cc:2889] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:16.872004 1182 raft_consensus.cc:2272] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:16.872500 756 tablet_service.cc:1552] Processing DeleteTablet for tablet cb0dcacdf0f1419da0b25f20cac1d62d with delete_type TABLET_DATA_DELETED (replaced by tablet d67e91a022f1401ea724dff898c3b81b) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:16.873276 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet cb0dcacdf0f1419da0b25f20cac1d62d with delete_type TABLET_DATA_DELETED (replaced by tablet d67e91a022f1401ea724dff898c3b81b) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:16.874066 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_024075b20add4ddcbc4b27601a94eed4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:16.874714 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_024075b20add4ddcbc4b27601a94eed4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:16.875695 1182 ts_tablet_manager.cc:1916] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:16.882790 1180 ts_tablet_manager.cc:1929] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:16.882997 1359 tablet.cc:1722] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:16.883132 1180 log.cc:1199] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/2ff6af17a6854451a1d7b904aa4f1f55
I20260202 02:52:16.884526 1180 ts_tablet_manager.cc:1950] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:16.884882 1175 tablet_bootstrap.cc:492] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:16.885033 1175 ts_tablet_manager.cc:1403] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.046s user 0.003s sys 0.006s
I20260202 02:52:16.885742 1175 raft_consensus.cc:359] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.885941 1175 raft_consensus.cc:385] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.886000 1175 raft_consensus.cc:740] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.886219 1175 consensus_queue.cc:260] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.886839 1175 ts_tablet_manager.cc:1434] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260202 02:52:16.887169 1175 tablet_bootstrap.cc:492] T cb0dcacdf0f1419da0b25f20cac1d62d P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:16.889169 1183 tablet_replica.cc:333] T 58f09c607cce4ccc82afd75c002ed1a8 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:16.889671 1183 raft_consensus.cc:2243] T 58f09c607cce4ccc82afd75c002ed1a8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:16.889914 1361 tablet.cc:1722] T 251455168e33430aafea8e0de42ca1b8 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260202 02:52:16.892678 522 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 723921dfe54c4e66af5cab76a25fee0a consensus state has no leader
I20260202 02:52:16.893170 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_1fd940660d9b4ef292d7aaa5e0e336cb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:16.893808 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1fd940660d9b4ef292d7aaa5e0e336cb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:16.892694 519 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 0d297b6243c348ee82c146a8339cf6c4
I20260202 02:52:16.895072 1183 raft_consensus.cc:2272] T 58f09c607cce4ccc82afd75c002ed1a8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:16.889468 1173 tablet_bootstrap.cc:492] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:16.895486 519 catalog_manager.cc:6724] ReplaceTablet: tablet 0d297b6243c348ee82c146a8339cf6c4 deleted and replaced by tablet d64f42d187bc4c9f9381c931449ec0cc
I20260202 02:52:16.895813 519 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_86f34199535c451cb54a2e20e70f1149" } new_table_name: "default.Table_ce2d44550fb5403a99372bc5d3d79265" modify_external_catalogs: true
I20260202 02:52:16.896319 1173 ts_tablet_manager.cc:1403] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.157s user 0.004s sys 0.000s
I20260202 02:52:16.897167 1173 raft_consensus.cc:359] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.897341 1173 raft_consensus.cc:385] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.897423 1173 raft_consensus.cc:740] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.897940 896 tablet_service.cc:1552] Processing DeleteTablet for tablet 0d297b6243c348ee82c146a8339cf6c4 with delete_type TABLET_DATA_DELETED (replaced by tablet d64f42d187bc4c9f9381c931449ec0cc) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:16.897980 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet 0d297b6243c348ee82c146a8339cf6c4 with delete_type TABLET_DATA_DELETED (replaced by tablet d64f42d187bc4c9f9381c931449ec0cc) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:16.898257 1173 consensus_queue.cc:260] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.898857 1173 ts_tablet_manager.cc:1434] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:16.899483 1173 tablet_bootstrap.cc:492] T 672ea3f731ba42ff8b1ff47db039ed22 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:16.900066 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 0d297b6243c348ee82c146a8339cf6c4 with delete_type TABLET_DATA_DELETED (replaced by tablet d64f42d187bc4c9f9381c931449ec0cc) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:16.900367 1180 tablet_replica.cc:333] T f24c0b99574d437a99b273204f105460 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:16.900600 1180 raft_consensus.cc:2243] T f24c0b99574d437a99b273204f105460 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:16.900791 1180 raft_consensus.cc:2272] T f24c0b99574d437a99b273204f105460 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:16.900703 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 2ff6af17a6854451a1d7b904aa4f1f55 (table default.Table_5eefa56618e84f8d92f8d87132ccd7d4 [id=b4ab2ba86e1a49338a98d855ded99f5d]) successfully deleted
I20260202 02:52:16.902797 1174 tablet_bootstrap.cc:492] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:16.902987 1174 ts_tablet_manager.cc:1403] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.033s user 0.000s sys 0.003s
I20260202 02:52:16.903338 1183 ts_tablet_manager.cc:1916] T 58f09c607cce4ccc82afd75c002ed1a8 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:16.903388 1180 ts_tablet_manager.cc:1916] T f24c0b99574d437a99b273204f105460 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:16.904277 1175 tablet_bootstrap.cc:654] T cb0dcacdf0f1419da0b25f20cac1d62d P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.903829 1174 raft_consensus.cc:359] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:16.907027 1174 raft_consensus.cc:385] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.907146 1174 raft_consensus.cc:740] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.907457 1174 consensus_queue.cc:260] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:16.909838 1182 ts_tablet_manager.cc:1929] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:16.911000 1174 ts_tablet_manager.cc:1434] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.008s user 0.000s sys 0.002s
I20260202 02:52:16.911257 1182 log.cc:1199] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/575016c0e9f84e99b962c5f14ffa96cc
W20260202 02:52:16.911566 526 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet eaa47f3388144e8b852804c167f232ce has no consensus state
I20260202 02:52:16.912590 526 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 938813ee0a8a40b6bf0e537e3ec1a0d4 (table default.Table_e70b044a267b465d94fede069f2f0df6 [id=1229c6d37b994a669d3363fdf6f8c6c8]). Expected version 1 got 0
I20260202 02:52:16.913702 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_ba9c16aaa48849db9fece59168c16886"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:16.914266 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ba9c16aaa48849db9fece59168c16886 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:16.914448 1174 tablet_bootstrap.cc:492] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:16.916427 1182 ts_tablet_manager.cc:1950] T 575016c0e9f84e99b962c5f14ffa96cc P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:16.916846 1174 tablet_bootstrap.cc:654] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.917793 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 575016c0e9f84e99b962c5f14ffa96cc (table default.Table_4720e764353243a495245b92e5b913fb [id=f2276e1c2ca04a70804053da61dd3447]) successfully deleted
I20260202 02:52:16.918713 1175 tablet_bootstrap.cc:492] T cb0dcacdf0f1419da0b25f20cac1d62d P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:16.918879 1175 ts_tablet_manager.cc:1403] T cb0dcacdf0f1419da0b25f20cac1d62d P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.032s user 0.003s sys 0.001s
I20260202 02:52:16.919646 1175 raft_consensus.cc:359] T cb0dcacdf0f1419da0b25f20cac1d62d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:16.919744 715 consensus_peers.cc:597] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:16.919847 1175 raft_consensus.cc:385] T cb0dcacdf0f1419da0b25f20cac1d62d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.919940 1175 raft_consensus.cc:740] T cb0dcacdf0f1419da0b25f20cac1d62d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.920068 1180 ts_tablet_manager.cc:1929] T f24c0b99574d437a99b273204f105460 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:16.920287 1180 log.cc:1199] T f24c0b99574d437a99b273204f105460 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/f24c0b99574d437a99b273204f105460
I20260202 02:52:16.920796 1175 consensus_queue.cc:260] T cb0dcacdf0f1419da0b25f20cac1d62d P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.921059 1180 ts_tablet_manager.cc:1950] T f24c0b99574d437a99b273204f105460 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:16.922345 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet f24c0b99574d437a99b273204f105460 (table default.Table_f1e417bf19b94bc491c3937276c669fd [id=9a8a252de32848288c9b3752c7d3feb7]) successfully deleted
I20260202 02:52:16.922963 1175 ts_tablet_manager.cc:1434] T cb0dcacdf0f1419da0b25f20cac1d62d P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260202 02:52:16.923292 1175 tablet_bootstrap.cc:492] T 415d14533fd54f20adb00455a7bcad5f P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:16.923815 1338 raft_consensus.cc:493] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:16.923967 1338 raft_consensus.cc:515] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.924669 1338 leader_election.cc:290] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:16.925361 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d2d27ceea1bb4122b18f0fcf4d2d1bf4" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:16.925912 859 leader_election.cc:343] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.926684 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d2d27ceea1bb4122b18f0fcf4d2d1bf4" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:16.926967 777 raft_consensus.cc:2468] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
I20260202 02:52:16.928457 857 leader_election.cc:304] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:16.929019 1338 raft_consensus.cc:2804] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:16.929157 1338 raft_consensus.cc:493] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:16.929255 1338 raft_consensus.cc:3060] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:16.932094 1338 raft_consensus.cc:515] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:16.934091 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet cb0dcacdf0f1419da0b25f20cac1d62d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cb0dcacdf0f1419da0b25f20cac1d62d already in progress: creating tablet
W20260202 02:52:16.934422 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 0d297b6243c348ee82c146a8339cf6c4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0d297b6243c348ee82c146a8339cf6c4 already in progress: creating tablet
I20260202 02:52:16.934737 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d2d27ceea1bb4122b18f0fcf4d2d1bf4" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:16.935091 1246 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
W20260202 02:52:16.936113 859 leader_election.cc:343] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.936224 1246 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260202 02:52:16.936586 1246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260202 02:52:16.936969 1338 leader_election.cc:290] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:16.937187 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d2d27ceea1bb4122b18f0fcf4d2d1bf4" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:16.937538 777 raft_consensus.cc:3060] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:16.937983 1182 tablet_replica.cc:333] T fc93352b24e345f99d15085e4886c748 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:16.938207 1182 raft_consensus.cc:2243] T fc93352b24e345f99d15085e4886c748 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:16.938414 1182 raft_consensus.cc:2272] T fc93352b24e345f99d15085e4886c748 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:16.939038 1182 ts_tablet_manager.cc:1916] T fc93352b24e345f99d15085e4886c748 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:16.940598 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 97bd7223d41d40a6bcdc60b1f63abc67 because the tablet was not found. No further retry: Not found: Tablet not found: 97bd7223d41d40a6bcdc60b1f63abc67
I20260202 02:52:16.942325 777 raft_consensus.cc:2468] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
I20260202 02:52:16.943109 857 leader_election.cc:304] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:16.943682 1338 raft_consensus.cc:2804] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:16.944988 1338 raft_consensus.cc:697] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:16.904572 1173 tablet_bootstrap.cc:654] T 672ea3f731ba42ff8b1ff47db039ed22 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.945353 1338 consensus_queue.cc:237] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.951418 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_95c7a47b60374ca2959b0feb63df471f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:16.952171 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_95c7a47b60374ca2959b0feb63df471f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:16.952729 1175 tablet_bootstrap.cc:654] T 415d14533fd54f20adb00455a7bcad5f P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.959929 1338 raft_consensus.cc:493] T ea040c03420c4b1b98897f39160b4130 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:16.960170 1338 raft_consensus.cc:515] T ea040c03420c4b1b98897f39160b4130 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:16.961376 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ea040c03420c4b1b98897f39160b4130" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:16.962157 859 leader_election.cc:343] T ea040c03420c4b1b98897f39160b4130 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.967264 1338 leader_election.cc:290] T ea040c03420c4b1b98897f39160b4130 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:16.967396 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ea040c03420c4b1b98897f39160b4130" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:16.968024 857 leader_election.cc:343] T ea040c03420c4b1b98897f39160b4130 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:16.968168 857 leader_election.cc:304] T ea040c03420c4b1b98897f39160b4130 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:16.968842 1338 raft_consensus.cc:2749] T ea040c03420c4b1b98897f39160b4130 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:16.970597 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_4336d55393794a208c76f0a580420596" } new_table_name: "default.Table_5308cabde3e94be1ab7a0b4e15ad67a9" modify_external_catalogs: true
I20260202 02:52:16.971966 1182 ts_tablet_manager.cc:1929] T fc93352b24e345f99d15085e4886c748 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:16.976307 1182 log.cc:1199] T fc93352b24e345f99d15085e4886c748 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/fc93352b24e345f99d15085e4886c748
I20260202 02:52:16.977169 1182 ts_tablet_manager.cc:1950] T fc93352b24e345f99d15085e4886c748 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:16.980360 1175 tablet_bootstrap.cc:492] T 415d14533fd54f20adb00455a7bcad5f P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:16.980628 1175 ts_tablet_manager.cc:1403] T 415d14533fd54f20adb00455a7bcad5f P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.057s user 0.006s sys 0.001s
I20260202 02:52:16.980859 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet fc93352b24e345f99d15085e4886c748 (table default.Table_8aee8d27e02942fb91879229b190457f [id=c59468c8c5b7480d88cb4b7d447a5975]) successfully deleted
I20260202 02:52:16.981551 1175 raft_consensus.cc:359] T 415d14533fd54f20adb00455a7bcad5f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.981817 1175 raft_consensus.cc:385] T 415d14533fd54f20adb00455a7bcad5f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:16.981889 1175 raft_consensus.cc:740] T 415d14533fd54f20adb00455a7bcad5f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:16.982142 1175 consensus_queue.cc:260] T 415d14533fd54f20adb00455a7bcad5f P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:16.984727 1175 ts_tablet_manager.cc:1434] T 415d14533fd54f20adb00455a7bcad5f P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260202 02:52:16.985704 1175 tablet_bootstrap.cc:492] T ff6f5710de4b4c48948aacdf5be29e8b P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:16.987673 1175 tablet_bootstrap.cc:654] T ff6f5710de4b4c48948aacdf5be29e8b P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:16.988605 521 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet bf74f96bea2e4ab7adec42c3c5207e79
I20260202 02:52:16.989569 1182 tablet_replica.cc:333] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:16.989499 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_6cd4b2d180cf4fdebad15ee5ea5ea7de"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:16.989847 1182 raft_consensus.cc:2243] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:16.990128 1182 pending_rounds.cc:70] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5: Trying to abort 1 pending ops.
W20260202 02:52:16.990203 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6cd4b2d180cf4fdebad15ee5ea5ea7de in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:16.990202 1182 pending_rounds.cc:77] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923015976751104 op_type: NO_OP noop_request { }
I20260202 02:52:16.990311 1182 raft_consensus.cc:2889] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:16.990422 1182 raft_consensus.cc:2272] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:16.991591 525 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet 90dc892c5fb647b9bf1b96da2aa1761e
I20260202 02:52:16.992030 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_69b88340567c4887947ed8458ffd1cba"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:16.992671 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_69b88340567c4887947ed8458ffd1cba in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:16.993920 1182 ts_tablet_manager.cc:1916] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:16.993923 756 tablet_service.cc:1552] Processing DeleteTablet for tablet bf74f96bea2e4ab7adec42c3c5207e79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:16 UTC) from {username='slave'} at 127.0.0.1:42950
W20260202 02:52:16.994975 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 13309b20a96240b5aa9fbb691cbed981 because the tablet was not found. No further retry: Not found: Tablet not found: 13309b20a96240b5aa9fbb691cbed981
I20260202 02:52:16.995182 896 tablet_service.cc:1552] Processing DeleteTablet for tablet bf74f96bea2e4ab7adec42c3c5207e79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:16 UTC) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:17.004870 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet bf74f96bea2e4ab7adec42c3c5207e79 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bf74f96bea2e4ab7adec42c3c5207e79 already in progress: creating tablet
I20260202 02:52:17.006603 1182 ts_tablet_manager.cc:1929] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:17.006553 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet bf74f96bea2e4ab7adec42c3c5207e79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:16 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:17.007879 1175 tablet_bootstrap.cc:492] T ff6f5710de4b4c48948aacdf5be29e8b P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:17.008512 1175 ts_tablet_manager.cc:1403] T ff6f5710de4b4c48948aacdf5be29e8b P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.000s
I20260202 02:52:17.009384 1175 raft_consensus.cc:359] T ff6f5710de4b4c48948aacdf5be29e8b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:17.009588 1175 raft_consensus.cc:385] T ff6f5710de4b4c48948aacdf5be29e8b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.009670 1175 raft_consensus.cc:740] T ff6f5710de4b4c48948aacdf5be29e8b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.009881 1178 raft_consensus.cc:493] T fc93352b24e345f99d15085e4886c748 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.009922 1175 consensus_queue.cc:260] T ff6f5710de4b4c48948aacdf5be29e8b P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:17.010056 1178 raft_consensus.cc:515] T fc93352b24e345f99d15085e4886c748 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.010710 1178 leader_election.cc:290] T fc93352b24e345f99d15085e4886c748 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:17.012184 998 leader_election.cc:343] T fc93352b24e345f99d15085e4886c748 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Not found: Tablet not found: fc93352b24e345f99d15085e4886c748
I20260202 02:52:17.012501 1175 ts_tablet_manager.cc:1434] T ff6f5710de4b4c48948aacdf5be29e8b P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260202 02:52:17.012802 1175 tablet_bootstrap.cc:492] T 761f5a51335a4de5977f5f7fef0d4260 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
W20260202 02:52:17.014143 999 leader_election.cc:343] T fc93352b24e345f99d15085e4886c748 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: fc93352b24e345f99d15085e4886c748
I20260202 02:52:17.014299 999 leader_election.cc:304] T fc93352b24e345f99d15085e4886c748 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:17.014827 1178 raft_consensus.cc:2749] T fc93352b24e345f99d15085e4886c748 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:17.018961 1173 tablet_bootstrap.cc:492] T 672ea3f731ba42ff8b1ff47db039ed22 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:17.019025 1175 tablet_bootstrap.cc:654] T 761f5a51335a4de5977f5f7fef0d4260 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.019310 1173 ts_tablet_manager.cc:1403] T 672ea3f731ba42ff8b1ff47db039ed22 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.120s user 0.004s sys 0.000s
I20260202 02:52:17.020289 1173 raft_consensus.cc:359] T 672ea3f731ba42ff8b1ff47db039ed22 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.020752 1182 log.cc:1199] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/add934bf70ae4de8a40066f06fd94795
I20260202 02:52:17.021378 1173 raft_consensus.cc:385] T 672ea3f731ba42ff8b1ff47db039ed22 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.021564 1173 raft_consensus.cc:740] T 672ea3f731ba42ff8b1ff47db039ed22 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.021588 1182 ts_tablet_manager.cc:1950] T add934bf70ae4de8a40066f06fd94795 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:17.021984 1173 consensus_queue.cc:260] T 672ea3f731ba42ff8b1ff47db039ed22 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.023974 1175 tablet_bootstrap.cc:492] T 761f5a51335a4de5977f5f7fef0d4260 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:17.024170 1175 ts_tablet_manager.cc:1403] T 761f5a51335a4de5977f5f7fef0d4260 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.000s
I20260202 02:52:17.025084 1175 raft_consensus.cc:359] T 761f5a51335a4de5977f5f7fef0d4260 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.025301 1175 raft_consensus.cc:385] T 761f5a51335a4de5977f5f7fef0d4260 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.025390 1175 raft_consensus.cc:740] T 761f5a51335a4de5977f5f7fef0d4260 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.025583 1182 tablet_replica.cc:333] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:17.025643 1175 consensus_queue.cc:260] T 761f5a51335a4de5977f5f7fef0d4260 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.026299 1175 ts_tablet_manager.cc:1434] T 761f5a51335a4de5977f5f7fef0d4260 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260202 02:52:17.026721 1175 tablet_bootstrap.cc:492] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:17.028025 528 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_22dd5a0be7944811bf358ec584461157" } modify_external_catalogs: true
I20260202 02:52:17.028254 528 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_22dd5a0be7944811bf358ec584461157" } modify_external_catalogs: true
I20260202 02:52:17.028546 1175 tablet_bootstrap.cc:654] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.024498 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet add934bf70ae4de8a40066f06fd94795 (table default.Table_9fb69280bd6048d898c8e92f2ef9f61e [id=7cf1e30412cb48ab962f526efee72121]) successfully deleted
W20260202 02:52:17.031457 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet f24c0b99574d437a99b273204f105460 because the tablet was not found. No further retry: Not found: Tablet not found: f24c0b99574d437a99b273204f105460
I20260202 02:52:17.032569 1173 ts_tablet_manager.cc:1434] T 672ea3f731ba42ff8b1ff47db039ed22 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.013s user 0.000s sys 0.003s
I20260202 02:52:17.032713 1175 tablet_bootstrap.cc:492] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:17.032881 1175 ts_tablet_manager.cc:1403] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.001s
I20260202 02:52:17.033840 1175 raft_consensus.cc:359] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.034379 1175 raft_consensus.cc:385] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.034389 1182 raft_consensus.cc:2243] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:17.034513 1175 raft_consensus.cc:740] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.035357 1182 raft_consensus.cc:2272] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:17.036116 1182 ts_tablet_manager.cc:1916] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:17.037020 1173 tablet_bootstrap.cc:492] T 4de41396cffd4d32b86c701b538d1e01 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:17.037541 1175 consensus_queue.cc:260] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.039098 1173 tablet_bootstrap.cc:654] T 4de41396cffd4d32b86c701b538d1e01 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.043007 1178 raft_consensus.cc:493] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.043269 1178 raft_consensus.cc:515] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.043987 1178 leader_election.cc:290] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:17.045271 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "938813ee0a8a40b6bf0e537e3ec1a0d4" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:17.052381 1182 ts_tablet_manager.cc:1929] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:17.052783 1182 log.cc:1199] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/2ff6af17a6854451a1d7b904aa4f1f55
I20260202 02:52:17.052827 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "938813ee0a8a40b6bf0e537e3ec1a0d4" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:17.052625 1175 ts_tablet_manager.cc:1434] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260202 02:52:17.053483 999 leader_election.cc:304] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:17.054095 1178 raft_consensus.cc:2749] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:17.056005 1175 tablet_bootstrap.cc:492] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:17.057787 1182 ts_tablet_manager.cc:1950] T 2ff6af17a6854451a1d7b904aa4f1f55 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:17.035820 1183 ts_tablet_manager.cc:1929] T 58f09c607cce4ccc82afd75c002ed1a8 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:17.059897 1183 log.cc:1199] T 58f09c607cce4ccc82afd75c002ed1a8 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/58f09c607cce4ccc82afd75c002ed1a8
I20260202 02:52:17.060173 522 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_0842fd6a82e145e7a98b3fed2c0c6993" } modify_external_catalogs: true
I20260202 02:52:17.060397 1183 ts_tablet_manager.cc:1950] T 58f09c607cce4ccc82afd75c002ed1a8 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:17.060453 522 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_0842fd6a82e145e7a98b3fed2c0c6993" } modify_external_catalogs: true
I20260202 02:52:17.060536 1175 tablet_bootstrap.cc:654] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.061439 1183 tablet_replica.cc:333] T 2ff6af17a6854451a1d7b904aa4f1f55 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:17.061661 1183 raft_consensus.cc:2243] T 2ff6af17a6854451a1d7b904aa4f1f55 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:17.061837 1183 raft_consensus.cc:2272] T 2ff6af17a6854451a1d7b904aa4f1f55 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:17.062431 1183 ts_tablet_manager.cc:1916] T 2ff6af17a6854451a1d7b904aa4f1f55 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:17.063290 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 2ff6af17a6854451a1d7b904aa4f1f55 (table default.Table_5eefa56618e84f8d92f8d87132ccd7d4 [id=b4ab2ba86e1a49338a98d855ded99f5d]) successfully deleted
I20260202 02:52:17.063413 1182 tablet_replica.cc:333] T 672ea3f731ba42ff8b1ff47db039ed22 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:17.063643 1182 raft_consensus.cc:2243] T 672ea3f731ba42ff8b1ff47db039ed22 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:17.063649 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 58f09c607cce4ccc82afd75c002ed1a8 (table default.Table_3b335a0806154f98a907f551596ba353 [id=0da4db61b3fe4cf4846edd5530b9327f]) successfully deleted
I20260202 02:52:17.063984 1182 raft_consensus.cc:2272] T 672ea3f731ba42ff8b1ff47db039ed22 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:17.063954 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet 672ea3f731ba42ff8b1ff47db039ed22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:17.064721 1182 ts_tablet_manager.cc:1916] T 672ea3f731ba42ff8b1ff47db039ed22 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:17.065132 1174 tablet_bootstrap.cc:492] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:17.065303 1174 ts_tablet_manager.cc:1403] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.151s user 0.046s sys 0.058s
I20260202 02:52:17.066216 1174 raft_consensus.cc:359] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:17.066430 1174 raft_consensus.cc:385] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.066519 1174 raft_consensus.cc:740] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.069667 1175 tablet_bootstrap.cc:492] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:17.069950 1174 consensus_queue.cc:260] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:17.070467 1175 ts_tablet_manager.cc:1403] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.007s
I20260202 02:52:17.070631 1174 ts_tablet_manager.cc:1434] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260202 02:52:17.070959 1174 tablet_bootstrap.cc:492] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:17.071300 1175 raft_consensus.cc:359] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.072461 1175 raft_consensus.cc:385] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.072575 1175 raft_consensus.cc:740] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.074375 1173 tablet_bootstrap.cc:492] T 4de41396cffd4d32b86c701b538d1e01 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:17.074647 1175 consensus_queue.cc:260] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.075268 1175 ts_tablet_manager.cc:1434] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.005s user 0.000s sys 0.001s
I20260202 02:52:17.074548 1173 ts_tablet_manager.cc:1403] T 4de41396cffd4d32b86c701b538d1e01 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.038s user 0.002s sys 0.002s
I20260202 02:52:17.078126 1173 raft_consensus.cc:359] T 4de41396cffd4d32b86c701b538d1e01 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.081408 1173 raft_consensus.cc:385] T 4de41396cffd4d32b86c701b538d1e01 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.083252 1173 raft_consensus.cc:740] T 4de41396cffd4d32b86c701b538d1e01 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.083802 519 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_8f23e3356aa14793a1aad274d2b845eb" } modify_external_catalogs: true
I20260202 02:52:17.084040 519 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_8f23e3356aa14793a1aad274d2b845eb" } modify_external_catalogs: true
I20260202 02:52:17.084300 1173 consensus_queue.cc:260] T 4de41396cffd4d32b86c701b538d1e01 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.089062 1175 tablet_bootstrap.cc:492] T c63a670d5c5e489897bc6dbb142b844d P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:17.090338 1174 tablet_bootstrap.cc:654] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.092031 526 master_service.cc:954] ReplaceTablet: received request to replace tablet c064ea0dade840b4840a5715c2d83f69 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:17.093060 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet 4de41396cffd4d32b86c701b538d1e01 with delete_type TABLET_DATA_DELETED (replaced by tablet bf74f96bea2e4ab7adec42c3c5207e79) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:17.096661 1183 ts_tablet_manager.cc:1929] T 2ff6af17a6854451a1d7b904aa4f1f55 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:17.097446 1359 tablet.cc:1722] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:17.097956 1183 log.cc:1199] T 2ff6af17a6854451a1d7b904aa4f1f55 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/2ff6af17a6854451a1d7b904aa4f1f55
I20260202 02:52:17.098349 1183 ts_tablet_manager.cc:1950] T 2ff6af17a6854451a1d7b904aa4f1f55 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:17.099120 1173 ts_tablet_manager.cc:1434] T 4de41396cffd4d32b86c701b538d1e01 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.023s user 0.000s sys 0.003s
I20260202 02:52:17.099573 1183 tablet_replica.cc:333] T fc93352b24e345f99d15085e4886c748 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:17.100086 1183 raft_consensus.cc:2243] T fc93352b24e345f99d15085e4886c748 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:17.100353 1183 raft_consensus.cc:2272] T fc93352b24e345f99d15085e4886c748 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:17.101895 1173 tablet_bootstrap.cc:492] T 13309b20a96240b5aa9fbb691cbed981 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:17.103747 524 master_service.cc:954] ReplaceTablet: received request to replace tablet 0d297b6243c348ee82c146a8339cf6c4 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:17.104310 1173 tablet_bootstrap.cc:654] T 13309b20a96240b5aa9fbb691cbed981 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.105070 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 2ff6af17a6854451a1d7b904aa4f1f55 (table default.Table_5eefa56618e84f8d92f8d87132ccd7d4 [id=b4ab2ba86e1a49338a98d855ded99f5d]) successfully deleted
I20260202 02:52:17.106736 1174 tablet_bootstrap.cc:492] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:17.106910 1174 ts_tablet_manager.cc:1403] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.036s user 0.003s sys 0.000s
I20260202 02:52:17.107774 1174 raft_consensus.cc:359] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.107973 1174 raft_consensus.cc:385] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.108054 1174 raft_consensus.cc:740] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.108301 1174 consensus_queue.cc:260] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.109151 1174 ts_tablet_manager.cc:1434] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260202 02:52:17.109678 1174 tablet_bootstrap.cc:492] T 0d297b6243c348ee82c146a8339cf6c4 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:17.111732 1174 tablet_bootstrap.cc:654] T 0d297b6243c348ee82c146a8339cf6c4 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:17.112341 527 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d06b77ff1a6f4f78b73082b00145fcf6 consensus state has no leader
I20260202 02:52:17.112668 527 master_service.cc:954] ReplaceTablet: received request to replace tablet c184d44d7f434b39a997a7387e5d3de0 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:17.112828 521 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_6a67d7cee0254eba9a53f4d52a30ee39" } new_table_name: "default.Table_b3001bd4a86843ada9ee50b8e4915a9a" modify_external_catalogs: true
I20260202 02:52:17.114318 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_6de529f984b247d79857c52705d4974a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:17.114940 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6de529f984b247d79857c52705d4974a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:17.117202 520 catalog_manager.cc:5654] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130). New cstate: current_term: 1 leader_uuid: "1521a6d262b7451bbef8debe970dd6b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: HEALTHY } } }
I20260202 02:52:17.117784 520 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet d2d27ceea1bb4122b18f0fcf4d2d1bf4 (table default.Table_10aa0dafec0547069eb92ecf12b3c780 [id=1f9c48c8e2484856a66a0719268f22ae]). Expected version 1 got 0
I20260202 02:52:17.118198 520 catalog_manager.cc:5654] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130). New cstate: current_term: 1 leader_uuid: "1521a6d262b7451bbef8debe970dd6b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: HEALTHY } } }
I20260202 02:52:17.118760 520 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 560465c063d94ffaab33f9f79a58a357 (table default.Table_3b335a0806154f98a907f551596ba353 [id=0da4db61b3fe4cf4846edd5530b9327f]). Expected version 1 got 0
I20260202 02:52:17.118983 520 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 5986676f5484498da927b4c6da3f47ff (table default.Table_8cb3cbab1a654f88a07d6d480fc72e52 [id=d8485f23f63b493781bea557dbe8a0c1]). Expected version 1 got 0
I20260202 02:52:17.119211 901 tablet_service.cc:1505] Processing CreateTablet for tablet b428356fba0c4fb5a6033c5d39740cc5 (DEFAULT_TABLE table=default.Table_5c1272b44a5b4a84918e3c6c84dca87e [id=e87761e6b64a4476a05fcbcf73f0a2f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.118340 1361 tablet.cc:1722] T 251455168e33430aafea8e0de42ca1b8 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:17.121831 896 tablet_service.cc:1505] Processing CreateTablet for tablet f93e8d3c4faf48eaa98c3cbf88d585e2 (DEFAULT_TABLE table=default.Table_2ebdd65f856c4fd1881a573bcd3c46d4 [id=84a13cdc74684fc1ade2eb2d0a7f3c86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.122254 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b428356fba0c4fb5a6033c5d39740cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.123040 1041 tablet_service.cc:1505] Processing CreateTablet for tablet f93e8d3c4faf48eaa98c3cbf88d585e2 (DEFAULT_TABLE table=default.Table_2ebdd65f856c4fd1881a573bcd3c46d4 [id=84a13cdc74684fc1ade2eb2d0a7f3c86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.123446 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f93e8d3c4faf48eaa98c3cbf88d585e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.127414 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f93e8d3c4faf48eaa98c3cbf88d585e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.128701 1038 tablet_service.cc:1505] Processing CreateTablet for tablet b428356fba0c4fb5a6033c5d39740cc5 (DEFAULT_TABLE table=default.Table_5c1272b44a5b4a84918e3c6c84dca87e [id=e87761e6b64a4476a05fcbcf73f0a2f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.129153 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b428356fba0c4fb5a6033c5d39740cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.130600 1178 raft_consensus.cc:493] T fae7e3941c124cc2871390a0b6471f27 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.130772 1178 raft_consensus.cc:515] T fae7e3941c124cc2871390a0b6471f27 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:17.131453 1178 leader_election.cc:290] T fae7e3941c124cc2871390a0b6471f27 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:17.132349 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fae7e3941c124cc2871390a0b6471f27" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:17.133618 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fae7e3941c124cc2871390a0b6471f27" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:17.134109 900 tablet_service.cc:1505] Processing CreateTablet for tablet 8095e0c6ff0041fc8b8c8b0778a20b00 (DEFAULT_TABLE table=default.Table_4e6c3886f58b49698a00d45b957f4c9a [id=b5cc03b77f414304b29c5fb4bb7dc3ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.134608 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8095e0c6ff0041fc8b8c8b0778a20b00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.134804 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 8095e0c6ff0041fc8b8c8b0778a20b00 (DEFAULT_TABLE table=default.Table_4e6c3886f58b49698a00d45b957f4c9a [id=b5cc03b77f414304b29c5fb4bb7dc3ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.135198 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8095e0c6ff0041fc8b8c8b0778a20b00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.135538 756 tablet_service.cc:1505] Processing CreateTablet for tablet f93e8d3c4faf48eaa98c3cbf88d585e2 (DEFAULT_TABLE table=default.Table_2ebdd65f856c4fd1881a573bcd3c46d4 [id=84a13cdc74684fc1ade2eb2d0a7f3c86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.135936 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f93e8d3c4faf48eaa98c3cbf88d585e2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:17.136467 859 consensus_peers.cc:597] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5 -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:17.136638 749 tablet_service.cc:1505] Processing CreateTablet for tablet b428356fba0c4fb5a6033c5d39740cc5 (DEFAULT_TABLE table=default.Table_5c1272b44a5b4a84918e3c6c84dca87e [id=e87761e6b64a4476a05fcbcf73f0a2f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.137140 758 tablet_service.cc:1505] Processing CreateTablet for tablet 8095e0c6ff0041fc8b8c8b0778a20b00 (DEFAULT_TABLE table=default.Table_4e6c3886f58b49698a00d45b957f4c9a [id=b5cc03b77f414304b29c5fb4bb7dc3ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.137583 754 tablet_service.cc:1505] Processing CreateTablet for tablet 70306ed9dec54286885d965f2c012da4 (DEFAULT_TABLE table=default.Table_028fb132e4894101b8470628a6041c07 [id=31abb0facd164caaa15591259eb951c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.139194 898 tablet_service.cc:1505] Processing CreateTablet for tablet 70306ed9dec54286885d965f2c012da4 (DEFAULT_TABLE table=default.Table_028fb132e4894101b8470628a6041c07 [id=31abb0facd164caaa15591259eb951c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.139663 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70306ed9dec54286885d965f2c012da4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.140414 999 leader_election.cc:304] T fae7e3941c124cc2871390a0b6471f27 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:17.141007 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70306ed9dec54286885d965f2c012da4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.141538 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8095e0c6ff0041fc8b8c8b0778a20b00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.142055 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b428356fba0c4fb5a6033c5d39740cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.142047 1178 raft_consensus.cc:2749] T fae7e3941c124cc2871390a0b6471f27 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:17.146044 755 tablet_service.cc:1505] Processing CreateTablet for tablet 9884c90cf1d245b2a7eaaf00d71b1c3e (DEFAULT_TABLE table=default.Table_22dd5a0be7944811bf358ec584461157 [id=7c8c316a6dd74622a257e70c3299ea1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.146553 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9884c90cf1d245b2a7eaaf00d71b1c3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.150240 1178 raft_consensus.cc:493] T 4de41396cffd4d32b86c701b538d1e01 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.150372 750 tablet_service.cc:1505] Processing CreateTablet for tablet 75f7824681344d8d88df9402694fff1b (DEFAULT_TABLE table=default.Table_ae49654353a64141b6d7f0e6afe13d9d [id=431440e879c04f0f9ad63c64d8741b56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.150502 1178 raft_consensus.cc:515] T 4de41396cffd4d32b86c701b538d1e01 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.150797 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75f7824681344d8d88df9402694fff1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.151298 1178 leader_election.cc:290] T 4de41396cffd4d32b86c701b538d1e01 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:17.152226 998 leader_election.cc:343] T 4de41396cffd4d32b86c701b538d1e01 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Not found: Tablet not found: 4de41396cffd4d32b86c701b538d1e01
I20260202 02:52:17.152508 1277 raft_consensus.cc:493] T c050a2882f764dda8df2a7df6547f149 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.152655 1277 raft_consensus.cc:515] T c050a2882f764dda8df2a7df6547f149 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.153275 1277 leader_election.cc:290] T c050a2882f764dda8df2a7df6547f149 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:17.153921 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c050a2882f764dda8df2a7df6547f149" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:17.155527 715 leader_election.cc:343] T c050a2882f764dda8df2a7df6547f149 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:17.156021 1173 tablet_bootstrap.cc:492] T 13309b20a96240b5aa9fbb691cbed981 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:17.156241 1173 ts_tablet_manager.cc:1403] T 13309b20a96240b5aa9fbb691cbed981 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.054s user 0.000s sys 0.006s
W20260202 02:52:17.157166 999 leader_election.cc:343] T 4de41396cffd4d32b86c701b538d1e01 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 4de41396cffd4d32b86c701b538d1e01
I20260202 02:52:17.157292 999 leader_election.cc:304] T 4de41396cffd4d32b86c701b538d1e01 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:17.157438 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c050a2882f764dda8df2a7df6547f149" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:17.157653 923 raft_consensus.cc:2468] T c050a2882f764dda8df2a7df6547f149 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:17.157816 1178 raft_consensus.cc:2749] T 4de41396cffd4d32b86c701b538d1e01 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:17.158521 1173 raft_consensus.cc:359] T 13309b20a96240b5aa9fbb691cbed981 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.159343 1173 raft_consensus.cc:385] T 13309b20a96240b5aa9fbb691cbed981 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.159487 1173 raft_consensus.cc:740] T 13309b20a96240b5aa9fbb691cbed981 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.159813 1178 raft_consensus.cc:493] T 672ea3f731ba42ff8b1ff47db039ed22 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.159970 1178 raft_consensus.cc:515] T 672ea3f731ba42ff8b1ff47db039ed22 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.160593 1178 leader_election.cc:290] T 672ea3f731ba42ff8b1ff47db039ed22 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:17.161221 1338 raft_consensus.cc:493] T bf74f96bea2e4ab7adec42c3c5207e79 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.161442 1338 raft_consensus.cc:515] T bf74f96bea2e4ab7adec42c3c5207e79 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:17.161504 1173 consensus_queue.cc:260] T 13309b20a96240b5aa9fbb691cbed981 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.162128 1338 leader_election.cc:290] T bf74f96bea2e4ab7adec42c3c5207e79 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:17.162854 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "672ea3f731ba42ff8b1ff47db039ed22" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:17.163167 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf74f96bea2e4ab7adec42c3c5207e79" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:17.163774 1173 ts_tablet_manager.cc:1434] T 13309b20a96240b5aa9fbb691cbed981 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
W20260202 02:52:17.163777 859 leader_election.cc:343] T bf74f96bea2e4ab7adec42c3c5207e79 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:17.165036 1182 ts_tablet_manager.cc:1929] T 672ea3f731ba42ff8b1ff47db039ed22 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260202 02:52:17.165072 999 leader_election.cc:343] T 672ea3f731ba42ff8b1ff47db039ed22 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Illegal state: Tablet not RUNNING: STOPPED
I20260202 02:52:17.165302 1182 log.cc:1199] T 672ea3f731ba42ff8b1ff47db039ed22 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/672ea3f731ba42ff8b1ff47db039ed22
I20260202 02:52:17.165817 1182 ts_tablet_manager.cc:1950] T 672ea3f731ba42ff8b1ff47db039ed22 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
W20260202 02:52:17.167129 998 leader_election.cc:343] T 672ea3f731ba42ff8b1ff47db039ed22 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Not found: Tablet not found: 672ea3f731ba42ff8b1ff47db039ed22
I20260202 02:52:17.167227 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 672ea3f731ba42ff8b1ff47db039ed22 (table default.Table_85da33c295954de58e2c6290005ac865 [id=ed3c3f07e9ef4a568d36ebd62e0bddc5]) successfully deleted
I20260202 02:52:17.167296 1182 tablet_replica.cc:333] T cb0dcacdf0f1419da0b25f20cac1d62d P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:17.167302 998 leader_election.cc:304] T 672ea3f731ba42ff8b1ff47db039ed22 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:17.167550 1182 raft_consensus.cc:2243] T cb0dcacdf0f1419da0b25f20cac1d62d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:17.167724 1182 raft_consensus.cc:2272] T cb0dcacdf0f1419da0b25f20cac1d62d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:17.167752 1178 raft_consensus.cc:2749] T 672ea3f731ba42ff8b1ff47db039ed22 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:17.167933 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf74f96bea2e4ab7adec42c3c5207e79" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:17.168751 1182 ts_tablet_manager.cc:1916] T cb0dcacdf0f1419da0b25f20cac1d62d P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:17.169795 857 leader_election.cc:343] T bf74f96bea2e4ab7adec42c3c5207e79 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:17.170606 857 leader_election.cc:304] T bf74f96bea2e4ab7adec42c3c5207e79 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:17.170971 1338 raft_consensus.cc:2749] T bf74f96bea2e4ab7adec42c3c5207e79 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:17.171144 714 leader_election.cc:304] T c050a2882f764dda8df2a7df6547f149 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:17.174525 1277 raft_consensus.cc:2804] T c050a2882f764dda8df2a7df6547f149 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:17.170361 1175 tablet_bootstrap.cc:654] T c63a670d5c5e489897bc6dbb142b844d P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.178467 901 tablet_service.cc:1505] Processing CreateTablet for tablet 9884c90cf1d245b2a7eaaf00d71b1c3e (DEFAULT_TABLE table=default.Table_22dd5a0be7944811bf358ec584461157 [id=7c8c316a6dd74622a257e70c3299ea1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.179034 1178 raft_consensus.cc:493] T 13309b20a96240b5aa9fbb691cbed981 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.179284 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9884c90cf1d245b2a7eaaf00d71b1c3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.179220 1178 raft_consensus.cc:515] T 13309b20a96240b5aa9fbb691cbed981 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.179967 1178 leader_election.cc:290] T 13309b20a96240b5aa9fbb691cbed981 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:17.180421 1277 raft_consensus.cc:493] T c050a2882f764dda8df2a7df6547f149 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:17.180559 1277 raft_consensus.cc:3060] T c050a2882f764dda8df2a7df6547f149 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:17.181329 999 leader_election.cc:343] T 13309b20a96240b5aa9fbb691cbed981 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 13309b20a96240b5aa9fbb691cbed981
W20260202 02:52:17.181799 998 leader_election.cc:343] T 13309b20a96240b5aa9fbb691cbed981 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Not found: Tablet not found: 13309b20a96240b5aa9fbb691cbed981
I20260202 02:52:17.181937 998 leader_election.cc:304] T 13309b20a96240b5aa9fbb691cbed981 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:17.182451 1178 raft_consensus.cc:2749] T 13309b20a96240b5aa9fbb691cbed981 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:17.188920 1277 raft_consensus.cc:515] T c050a2882f764dda8df2a7df6547f149 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.189963 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c050a2882f764dda8df2a7df6547f149" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
W20260202 02:52:17.190523 715 leader_election.cc:343] T c050a2882f764dda8df2a7df6547f149 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:17.190830 1277 leader_election.cc:290] T c050a2882f764dda8df2a7df6547f149 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:17.193670 1175 tablet_bootstrap.cc:492] T c63a670d5c5e489897bc6dbb142b844d P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:17.193876 1175 ts_tablet_manager.cc:1403] T c63a670d5c5e489897bc6dbb142b844d P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.118s user 0.001s sys 0.004s
I20260202 02:52:17.194814 1175 raft_consensus.cc:359] T c63a670d5c5e489897bc6dbb142b844d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.195111 1175 raft_consensus.cc:385] T c63a670d5c5e489897bc6dbb142b844d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.195220 1175 raft_consensus.cc:740] T c63a670d5c5e489897bc6dbb142b844d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.195488 1175 consensus_queue.cc:260] T c63a670d5c5e489897bc6dbb142b844d P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.195803 922 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c050a2882f764dda8df2a7df6547f149" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:17.196002 922 raft_consensus.cc:3060] T c050a2882f764dda8df2a7df6547f149 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:17.199276 922 raft_consensus.cc:2468] T c050a2882f764dda8df2a7df6547f149 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:17.200251 1175 ts_tablet_manager.cc:1434] T c63a670d5c5e489897bc6dbb142b844d P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.006s user 0.001s sys 0.002s
I20260202 02:52:17.203449 903 tablet_service.cc:1505] Processing CreateTablet for tablet 0186572e7bb9499a932a636ab280f544 (DEFAULT_TABLE table=default.Table_8cae34d2c6f1443ba07e3f2f8abd1989 [id=dc77705cb5704fd58af1eed3edd45675]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.204388 900 tablet_service.cc:1505] Processing CreateTablet for tablet 526f2c7b95184a0987e7aa94efc8475a (DEFAULT_TABLE table=default.Table_0842fd6a82e145e7a98b3fed2c0c6993 [id=4e076b2023be4c888ac1c27babde1607]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.204901 896 tablet_service.cc:1505] Processing CreateTablet for tablet 80d443bdb66d4cc68ae283e08c410fe6 (DEFAULT_TABLE table=default.Table_78e75899f83c46f09c66dc84192f5438 [id=0bd47d48c02944a19ce210fc83794879]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.204996 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 70306ed9dec54286885d965f2c012da4 (DEFAULT_TABLE table=default.Table_028fb132e4894101b8470628a6041c07 [id=31abb0facd164caaa15591259eb951c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.205449 1039 tablet_service.cc:1505] Processing CreateTablet for tablet 9884c90cf1d245b2a7eaaf00d71b1c3e (DEFAULT_TABLE table=default.Table_22dd5a0be7944811bf358ec584461157 [id=7c8c316a6dd74622a257e70c3299ea1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.205864 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 75f7824681344d8d88df9402694fff1b (DEFAULT_TABLE table=default.Table_ae49654353a64141b6d7f0e6afe13d9d [id=431440e879c04f0f9ad63c64d8741b56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.205369 898 tablet_service.cc:1505] Processing CreateTablet for tablet 75f7824681344d8d88df9402694fff1b (DEFAULT_TABLE table=default.Table_ae49654353a64141b6d7f0e6afe13d9d [id=431440e879c04f0f9ad63c64d8741b56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.206223 1042 tablet_service.cc:1505] Processing CreateTablet for tablet 526f2c7b95184a0987e7aa94efc8475a (DEFAULT_TABLE table=default.Table_0842fd6a82e145e7a98b3fed2c0c6993 [id=4e076b2023be4c888ac1c27babde1607]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.206436 903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0186572e7bb9499a932a636ab280f544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.206625 1043 tablet_service.cc:1505] Processing CreateTablet for tablet 80d443bdb66d4cc68ae283e08c410fe6 (DEFAULT_TABLE table=default.Table_78e75899f83c46f09c66dc84192f5438 [id=0bd47d48c02944a19ce210fc83794879]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.207024 1040 tablet_service.cc:1505] Processing CreateTablet for tablet 0186572e7bb9499a932a636ab280f544 (DEFAULT_TABLE table=default.Table_8cae34d2c6f1443ba07e3f2f8abd1989 [id=dc77705cb5704fd58af1eed3edd45675]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.207801 897 tablet_service.cc:1505] Processing CreateTablet for tablet 78f122342dd144b48de89902faa0f94f (DEFAULT_TABLE table=default.Table_5eefa56618e84f8d92f8d87132ccd7d4 [id=b4ab2ba86e1a49338a98d855ded99f5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.208213 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78f122342dd144b48de89902faa0f94f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.210534 759 tablet_service.cc:1505] Processing CreateTablet for tablet 78f122342dd144b48de89902faa0f94f (DEFAULT_TABLE table=default.Table_5eefa56618e84f8d92f8d87132ccd7d4 [id=b4ab2ba86e1a49338a98d855ded99f5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.211474 753 tablet_service.cc:1505] Processing CreateTablet for tablet 0186572e7bb9499a932a636ab280f544 (DEFAULT_TABLE table=default.Table_8cae34d2c6f1443ba07e3f2f8abd1989 [id=dc77705cb5704fd58af1eed3edd45675]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.212175 752 tablet_service.cc:1505] Processing CreateTablet for tablet eaa47f3388144e8b852804c167f232ce (DEFAULT_TABLE table=default.Table_fcd378fcc866495ea6b021dc22247351 [id=52687dbd0e554cfb81d88e8fcdd4dd5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.205030 1036 tablet_service.cc:1505] Processing CreateTablet for tablet eaa47f3388144e8b852804c167f232ce (DEFAULT_TABLE table=default.Table_fcd378fcc866495ea6b021dc22247351 [id=52687dbd0e554cfb81d88e8fcdd4dd5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.212713 899 tablet_service.cc:1505] Processing CreateTablet for tablet eaa47f3388144e8b852804c167f232ce (DEFAULT_TABLE table=default.Table_fcd378fcc866495ea6b021dc22247351 [id=52687dbd0e554cfb81d88e8fcdd4dd5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.213198 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eaa47f3388144e8b852804c167f232ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.213333 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78f122342dd144b48de89902faa0f94f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.214397 714 leader_election.cc:304] T c050a2882f764dda8df2a7df6547f149 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:17.215605 1322 raft_consensus.cc:2804] T c050a2882f764dda8df2a7df6547f149 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:17.215766 1322 raft_consensus.cc:697] T c050a2882f764dda8df2a7df6547f149 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:17.217363 1277 consensus_queue.cc:1048] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260202 02:52:17.218147 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 526f2c7b95184a0987e7aa94efc8475a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.220739 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80d443bdb66d4cc68ae283e08c410fe6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.222178 754 tablet_service.cc:1505] Processing CreateTablet for tablet 526f2c7b95184a0987e7aa94efc8475a (DEFAULT_TABLE table=default.Table_0842fd6a82e145e7a98b3fed2c0c6993 [id=4e076b2023be4c888ac1c27babde1607]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.225991 751 tablet_service.cc:1505] Processing CreateTablet for tablet 80d443bdb66d4cc68ae283e08c410fe6 (DEFAULT_TABLE table=default.Table_78e75899f83c46f09c66dc84192f5438 [id=0bd47d48c02944a19ce210fc83794879]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.230976 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80d443bdb66d4cc68ae283e08c410fe6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.233978 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75f7824681344d8d88df9402694fff1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.241099 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0186572e7bb9499a932a636ab280f544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.244123 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eaa47f3388144e8b852804c167f232ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.248004 1175 tablet_bootstrap.cc:492] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:17.250648 1321 raft_consensus.cc:493] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.251613 1063 raft_consensus.cc:3060] T fae7e3941c124cc2871390a0b6471f27 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:17.257563 1175 tablet_bootstrap.cc:654] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.260164 1182 ts_tablet_manager.cc:1929] T cb0dcacdf0f1419da0b25f20cac1d62d P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:17.260789 1324 raft_consensus.cc:493] T 952d760159844ad08269654264e6381b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.261479 1324 raft_consensus.cc:515] T 952d760159844ad08269654264e6381b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:17.262130 1324 leader_election.cc:290] T 952d760159844ad08269654264e6381b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:17.262233 1182 log.cc:1199] T cb0dcacdf0f1419da0b25f20cac1d62d P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/cb0dcacdf0f1419da0b25f20cac1d62d
I20260202 02:52:17.262717 1182 ts_tablet_manager.cc:1950] T cb0dcacdf0f1419da0b25f20cac1d62d P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:17.263003 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "952d760159844ad08269654264e6381b" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:17.263077 1322 consensus_queue.cc:237] T c050a2882f764dda8df2a7df6547f149 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.263756 1182 tablet_replica.cc:333] T 0d297b6243c348ee82c146a8339cf6c4 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:17.260540 1321 raft_consensus.cc:515] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.265204 1321 leader_election.cc:290] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:17.265386 1182 raft_consensus.cc:2243] T 0d297b6243c348ee82c146a8339cf6c4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:17.265672 1182 raft_consensus.cc:2272] T 0d297b6243c348ee82c146a8339cf6c4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:17.266364 1182 ts_tablet_manager.cc:1916] T 0d297b6243c348ee82c146a8339cf6c4 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:17.267431 1175 tablet_bootstrap.cc:492] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:17.268723 1338 raft_consensus.cc:493] T 761f5a51335a4de5977f5f7fef0d4260 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.268888 1338 raft_consensus.cc:515] T 761f5a51335a4de5977f5f7fef0d4260 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.269583 1338 leader_election.cc:290] T 761f5a51335a4de5977f5f7fef0d4260 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:17.268510 1175 ts_tablet_manager.cc:1403] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.002s
I20260202 02:52:17.270535 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "761f5a51335a4de5977f5f7fef0d4260" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:17.271330 715 leader_election.cc:343] T 952d760159844ad08269654264e6381b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:17.271471 859 leader_election.cc:343] T 761f5a51335a4de5977f5f7fef0d4260 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:17.272174 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5986676f5484498da927b4c6da3f47ff" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:17.272439 1175 raft_consensus.cc:359] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.272655 1175 raft_consensus.cc:385] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.272732 1175 raft_consensus.cc:740] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.273015 1175 consensus_queue.cc:260] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.273684 1175 ts_tablet_manager.cc:1434] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
W20260202 02:52:17.273905 715 leader_election.cc:343] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:17.277798 1175 tablet_bootstrap.cc:492] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:17.281675 1175 tablet_bootstrap.cc:654] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.284042 1174 tablet_bootstrap.cc:492] T 0d297b6243c348ee82c146a8339cf6c4 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:17.284360 1174 ts_tablet_manager.cc:1403] T 0d297b6243c348ee82c146a8339cf6c4 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.175s user 0.002s sys 0.002s
I20260202 02:52:17.285291 1174 raft_consensus.cc:359] T 0d297b6243c348ee82c146a8339cf6c4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:17.288792 1174 raft_consensus.cc:385] T 0d297b6243c348ee82c146a8339cf6c4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.289325 1174 raft_consensus.cc:740] T 0d297b6243c348ee82c146a8339cf6c4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.286082 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 526f2c7b95184a0987e7aa94efc8475a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.290741 1175 tablet_bootstrap.cc:492] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:17.291875 1175 ts_tablet_manager.cc:1403] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.001s
I20260202 02:52:17.292187 1174 consensus_queue.cc:260] T 0d297b6243c348ee82c146a8339cf6c4 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:17.292857 1175 raft_consensus.cc:359] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.293077 1175 raft_consensus.cc:385] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.293174 1175 raft_consensus.cc:740] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.293465 1175 consensus_queue.cc:260] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.293567 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet cb0dcacdf0f1419da0b25f20cac1d62d (table default.Table_c245b83c1987454fabbe8646d14252d6 [id=08498a654b6a4a9ca3f968d70daad9b6]) successfully deleted
I20260202 02:52:17.294462 1175 ts_tablet_manager.cc:1434] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:17.294780 1175 tablet_bootstrap.cc:492] T fe7b6004e5094c0dbe5847144a2489b9 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:17.295495 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "761f5a51335a4de5977f5f7fef0d4260" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:17.296736 1174 ts_tablet_manager.cc:1434] T 0d297b6243c348ee82c146a8339cf6c4 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.012s user 0.002s sys 0.000s
I20260202 02:52:17.297129 1174 tablet_bootstrap.cc:492] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:17.297808 1183 ts_tablet_manager.cc:1916] T fc93352b24e345f99d15085e4886c748 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:17.302959 527 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet c184d44d7f434b39a997a7387e5d3de0
I20260202 02:52:17.303426 527 catalog_manager.cc:6724] ReplaceTablet: tablet c184d44d7f434b39a997a7387e5d3de0 deleted and replaced by tablet 3711910e050c4515bed4294d1271d5a9
I20260202 02:52:17.304116 1182 ts_tablet_manager.cc:1929] T 0d297b6243c348ee82c146a8339cf6c4 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:17.304242 1182 log.cc:1199] T 0d297b6243c348ee82c146a8339cf6c4 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/0d297b6243c348ee82c146a8339cf6c4
I20260202 02:52:17.305235 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_addc8f2e088548c99dbfaf1ae14135d0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:17.305891 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_addc8f2e088548c99dbfaf1ae14135d0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:17.305934 1182 ts_tablet_manager.cc:1950] T 0d297b6243c348ee82c146a8339cf6c4 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:17.306135 1174 tablet_bootstrap.cc:654] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:17.306397 521 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 61a9203035f2495f8f99924303ebf120 consensus state has no leader
I20260202 02:52:17.306602 1175 tablet_bootstrap.cc:654] T fe7b6004e5094c0dbe5847144a2489b9 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.306658 521 master_service.cc:954] ReplaceTablet: received request to replace tablet 560465c063d94ffaab33f9f79a58a357 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:17.307247 525 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_51cfbe7668e344b1bcca61f0ab935341" } new_table_name: "default.Table_6c46c4d0140e47cb8edc42ccd2748741" modify_external_catalogs: true
I20260202 02:52:17.308414 1182 tablet_replica.cc:333] T bf74f96bea2e4ab7adec42c3c5207e79 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:17.308662 1182 raft_consensus.cc:2243] T bf74f96bea2e4ab7adec42c3c5207e79 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:17.308836 1182 raft_consensus.cc:2272] T bf74f96bea2e4ab7adec42c3c5207e79 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:17.309964 1182 ts_tablet_manager.cc:1916] T bf74f96bea2e4ab7adec42c3c5207e79 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:17.310873 528 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 9884c90cf1d245b2a7eaaf00d71b1c3e
I20260202 02:52:17.311618 528 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_fcd378fcc866495ea6b021dc22247351" } new_table_name: "default.Table_87277c5d80184738b96b773f026b32a5" modify_external_catalogs: true
I20260202 02:52:17.312654 526 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet c064ea0dade840b4840a5715c2d83f69
I20260202 02:52:17.313090 526 catalog_manager.cc:6724] ReplaceTablet: tablet c064ea0dade840b4840a5715c2d83f69 deleted and replaced by tablet 622a113af04a41c6852c29fc8a8acd78
I20260202 02:52:17.313277 526 master_service.cc:954] ReplaceTablet: received request to replace tablet fae7e3941c124cc2871390a0b6471f27 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:17.313792 522 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 526f2c7b95184a0987e7aa94efc8475a
I20260202 02:52:17.314427 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_614c0ff5900b4d64b9c0af8c88a33ecb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:17.315009 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_614c0ff5900b4d64b9c0af8c88a33ecb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:17.318743 1175 tablet_bootstrap.cc:492] T fe7b6004e5094c0dbe5847144a2489b9 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:17.318953 519 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet eb0ac55fbb534b29a86d6b3e8aa72ebc
I20260202 02:52:17.319015 1175 ts_tablet_manager.cc:1403] T fe7b6004e5094c0dbe5847144a2489b9 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.001s
I20260202 02:52:17.319723 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_662111850b514df5ad7c69aa21783397"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:17.319937 1175 raft_consensus.cc:359] T fe7b6004e5094c0dbe5847144a2489b9 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.320207 1175 raft_consensus.cc:385] T fe7b6004e5094c0dbe5847144a2489b9 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.320276 1175 raft_consensus.cc:740] T fe7b6004e5094c0dbe5847144a2489b9 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
W20260202 02:52:17.320272 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_662111850b514df5ad7c69aa21783397 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:17.320564 1175 consensus_queue.cc:260] T fe7b6004e5094c0dbe5847144a2489b9 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.321252 1175 ts_tablet_manager.cc:1434] T fe7b6004e5094c0dbe5847144a2489b9 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260202 02:52:17.321713 1175 tablet_bootstrap.cc:492] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:17.323609 1175 tablet_bootstrap.cc:654] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.324872 523 catalog_manager.cc:5654] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f442ef85029461091de847118c34c0d (127.0.104.129). New cstate: current_term: 1 leader_uuid: "1f442ef85029461091de847118c34c0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } }
I20260202 02:52:17.325296 523 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet c0937b6910534ddc8a2217f0cbd539c5 (table default.Table_e0149b31be0c49d19600de4849d6577e [id=46b5d47377474ff9bac2acdb26b08c65]). Expected version 1 got 0
I20260202 02:52:17.325652 1182 ts_tablet_manager.cc:1929] T bf74f96bea2e4ab7adec42c3c5207e79 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:17.325807 1182 log.cc:1199] T bf74f96bea2e4ab7adec42c3c5207e79 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/bf74f96bea2e4ab7adec42c3c5207e79
I20260202 02:52:17.326164 1182 ts_tablet_manager.cc:1950] T bf74f96bea2e4ab7adec42c3c5207e79 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:17.326406 524 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_a6b938c21e164294bd8b5fb205673f75" } modify_external_catalogs: true
I20260202 02:52:17.326597 524 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_a6b938c21e164294bd8b5fb205673f75" } modify_external_catalogs: true
I20260202 02:52:17.328218 754 tablet_service.cc:1552] Processing DeleteTablet for tablet c184d44d7f434b39a997a7387e5d3de0 with delete_type TABLET_DATA_DELETED (replaced by tablet 3711910e050c4515bed4294d1271d5a9) from {username='slave'} at 127.0.0.1:42950
W20260202 02:52:17.329694 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet c184d44d7f434b39a997a7387e5d3de0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c184d44d7f434b39a997a7387e5d3de0 already in progress: creating tablet
I20260202 02:52:17.330829 1175 tablet_bootstrap.cc:492] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:17.331012 1175 ts_tablet_manager.cc:1403] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.003s
I20260202 02:52:17.331414 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 526f2c7b95184a0987e7aa94efc8475a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:17.331897 1175 raft_consensus.cc:359] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.332119 1175 raft_consensus.cc:385] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.332216 1175 raft_consensus.cc:740] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.332470 1175 consensus_queue.cc:260] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:17.332974 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 526f2c7b95184a0987e7aa94efc8475a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 526f2c7b95184a0987e7aa94efc8475a already in progress: creating tablet
I20260202 02:52:17.333142 1175 ts_tablet_manager.cc:1434] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260202 02:52:17.333498 1175 tablet_bootstrap.cc:492] T c184d44d7f434b39a997a7387e5d3de0 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:17.333635 758 tablet_service.cc:1552] Processing DeleteTablet for tablet eb0ac55fbb534b29a86d6b3e8aa72ebc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:42950
W20260202 02:52:17.334520 513 catalog_manager.cc:4952] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet eb0ac55fbb534b29a86d6b3e8aa72ebc because the tablet was not found. No further retry: Not found: Tablet not found: eb0ac55fbb534b29a86d6b3e8aa72ebc
I20260202 02:52:17.335577 1175 tablet_bootstrap.cc:654] T c184d44d7f434b39a997a7387e5d3de0 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.337491 752 tablet_service.cc:1552] Processing DeleteTablet for tablet 9884c90cf1d245b2a7eaaf00d71b1c3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:17.337760 753 tablet_service.cc:1552] Processing DeleteTablet for tablet c064ea0dade840b4840a5715c2d83f69 with delete_type TABLET_DATA_DELETED (replaced by tablet 622a113af04a41c6852c29fc8a8acd78) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:17.338498 1321 raft_consensus.cc:493] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.338660 1321 raft_consensus.cc:515] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:17.339350 1321 leader_election.cc:290] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
W20260202 02:52:17.340991 515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.190:33981, user_credentials={real_user=slave}} blocked reactor thread for 44820.8us
W20260202 02:52:17.342986 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 9884c90cf1d245b2a7eaaf00d71b1c3e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9884c90cf1d245b2a7eaaf00d71b1c3e already in progress: creating tablet
I20260202 02:52:17.343534 1321 raft_consensus.cc:493] T aa526a69cf914dd08142074636c956b7 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.343713 1321 raft_consensus.cc:515] T aa526a69cf914dd08142074636c956b7 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.344242 1175 tablet_bootstrap.cc:492] T c184d44d7f434b39a997a7387e5d3de0 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:17.344403 1321 leader_election.cc:290] T aa526a69cf914dd08142074636c956b7 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:17.344465 1175 ts_tablet_manager.cc:1403] T c184d44d7f434b39a997a7387e5d3de0 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.005s
I20260202 02:52:17.299227 1173 tablet_bootstrap.cc:492] T 251455168e33430aafea8e0de42ca1b8 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:17.336441 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70306ed9dec54286885d965f2c012da4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.341059 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f2a9404f7424dc78b5bd062ebd8cb53" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:17.345361 1175 raft_consensus.cc:359] T c184d44d7f434b39a997a7387e5d3de0 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:17.346553 1175 raft_consensus.cc:385] T c184d44d7f434b39a997a7387e5d3de0 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.346668 1175 raft_consensus.cc:740] T c184d44d7f434b39a997a7387e5d3de0 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.346930 1175 consensus_queue.cc:260] T c184d44d7f434b39a997a7387e5d3de0 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:17.347654 1175 ts_tablet_manager.cc:1434] T c184d44d7f434b39a997a7387e5d3de0 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
W20260202 02:52:17.347896 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet c064ea0dade840b4840a5715c2d83f69 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c064ea0dade840b4840a5715c2d83f69 already in progress: creating tablet
I20260202 02:52:17.348121 1175 tablet_bootstrap.cc:492] T d06b77ff1a6f4f78b73082b00145fcf6 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:17.348706 1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eaa47f3388144e8b852804c167f232ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.348990 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 0d297b6243c348ee82c146a8339cf6c4 (table default.Table_d210d1530706429197d6294b29f18c87 [id=7c76848b98614fbf8b2950e5d55cc609]) successfully deleted
W20260202 02:52:17.349231 714 leader_election.cc:343] T 952d760159844ad08269654264e6381b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 952d760159844ad08269654264e6381b
I20260202 02:52:17.349395 714 leader_election.cc:304] T 952d760159844ad08269654264e6381b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:17.349610 921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5986676f5484498da927b4c6da3f47ff" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:17.349941 921 raft_consensus.cc:2468] T 5986676f5484498da927b4c6da3f47ff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:17.350051 1175 tablet_bootstrap.cc:654] T d06b77ff1a6f4f78b73082b00145fcf6 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.350319 1324 raft_consensus.cc:2749] T 952d760159844ad08269654264e6381b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:17.351375 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet bf74f96bea2e4ab7adec42c3c5207e79 (table default.Table_d142ff2cb10e491997bf0e302d6a9fa5 [id=a0b8442c02974d6596db002c9292be0e]) successfully deleted
I20260202 02:52:17.352152 898 tablet_service.cc:1552] Processing DeleteTablet for tablet c184d44d7f434b39a997a7387e5d3de0 with delete_type TABLET_DATA_DELETED (replaced by tablet 3711910e050c4515bed4294d1271d5a9) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:17.352492 1182 tablet_replica.cc:333] T c184d44d7f434b39a997a7387e5d3de0 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:17.352584 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9884c90cf1d245b2a7eaaf00d71b1c3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.352727 1182 raft_consensus.cc:2243] T c184d44d7f434b39a997a7387e5d3de0 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:17.352923 1182 raft_consensus.cc:2272] T c184d44d7f434b39a997a7387e5d3de0 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:17.353227 896 tablet_service.cc:1552] Processing DeleteTablet for tablet 526f2c7b95184a0987e7aa94efc8475a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:17.353870 903 tablet_service.cc:1552] Processing DeleteTablet for tablet eb0ac55fbb534b29a86d6b3e8aa72ebc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:17.355223 901 tablet_service.cc:1552] Processing DeleteTablet for tablet 9884c90cf1d245b2a7eaaf00d71b1c3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:17.355530 898 tablet_service.cc:1552] Processing DeleteTablet for tablet c064ea0dade840b4840a5715c2d83f69 with delete_type TABLET_DATA_DELETED (replaced by tablet 622a113af04a41c6852c29fc8a8acd78) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:17.356967 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0186572e7bb9499a932a636ab280f544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.357815 1182 ts_tablet_manager.cc:1916] T c184d44d7f434b39a997a7387e5d3de0 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:17.359181 753 tablet_service.cc:1505] Processing CreateTablet for tablet 757492288d7c44fb91b25b7e6e21e01b (DEFAULT_TABLE table=default.Table_222f75e4260d41b2813393c25a2a6e14 [id=02e9e243f710428091de2f9497532c37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.359701 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 757492288d7c44fb91b25b7e6e21e01b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.364464 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75f7824681344d8d88df9402694fff1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.366801 1338 raft_consensus.cc:493] T ff6f5710de4b4c48948aacdf5be29e8b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.367024 1338 raft_consensus.cc:515] T ff6f5710de4b4c48948aacdf5be29e8b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:17.367807 1338 leader_election.cc:290] T ff6f5710de4b4c48948aacdf5be29e8b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:17.370514 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 526f2c7b95184a0987e7aa94efc8475a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.370458 1338 consensus_queue.cc:1048] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260202 02:52:17.371160 857 leader_election.cc:343] T 761f5a51335a4de5977f5f7fef0d4260 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:17.371290 857 leader_election.cc:304] T 761f5a51335a4de5977f5f7fef0d4260 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:17.371536 1338 raft_consensus.cc:2749] T 761f5a51335a4de5977f5f7fef0d4260 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:17.371949 1182 ts_tablet_manager.cc:1929] T c184d44d7f434b39a997a7387e5d3de0 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:17.372095 1182 log.cc:1199] T c184d44d7f434b39a997a7387e5d3de0 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/c184d44d7f434b39a997a7387e5d3de0
I20260202 02:52:17.373266 714 leader_election.cc:304] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:17.373733 1322 raft_consensus.cc:2804] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:17.373930 1322 raft_consensus.cc:493] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:17.374080 1322 raft_consensus.cc:3060] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:17.374464 715 leader_election.cc:343] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:17.375012 1182 ts_tablet_manager.cc:1950] T c184d44d7f434b39a997a7387e5d3de0 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:17.377146 1322 raft_consensus.cc:515] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.377261 1035 tablet_service.cc:1552] Processing DeleteTablet for tablet 9884c90cf1d245b2a7eaaf00d71b1c3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:17.377496 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80d443bdb66d4cc68ae283e08c410fe6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.378350 1322 leader_election.cc:290] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:17.378468 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "aa526a69cf914dd08142074636c956b7" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:17.380155 1034 tablet_service.cc:1552] Processing DeleteTablet for tablet c064ea0dade840b4840a5715c2d83f69 with delete_type TABLET_DATA_DELETED (replaced by tablet 622a113af04a41c6852c29fc8a8acd78) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:17.380524 1035 tablet_service.cc:1552] Processing DeleteTablet for tablet 526f2c7b95184a0987e7aa94efc8475a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:17.380919 752 tablet_service.cc:1505] Processing CreateTablet for tablet eb0ac55fbb534b29a86d6b3e8aa72ebc (DEFAULT_TABLE table=default.Table_8f23e3356aa14793a1aad274d2b845eb [id=21c284c2be45496ca760633f24f090d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.381383 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb0ac55fbb534b29a86d6b3e8aa72ebc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.386154 1173 tablet_bootstrap.cc:654] T 251455168e33430aafea8e0de42ca1b8 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.387363 1175 tablet_bootstrap.cc:492] T d06b77ff1a6f4f78b73082b00145fcf6 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:17.387557 1175 ts_tablet_manager.cc:1403] T d06b77ff1a6f4f78b73082b00145fcf6 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.040s user 0.005s sys 0.004s
I20260202 02:52:17.389093 1175 raft_consensus.cc:359] T d06b77ff1a6f4f78b73082b00145fcf6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.389321 1175 raft_consensus.cc:385] T d06b77ff1a6f4f78b73082b00145fcf6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.389405 1175 raft_consensus.cc:740] T d06b77ff1a6f4f78b73082b00145fcf6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.389636 1175 consensus_queue.cc:260] T d06b77ff1a6f4f78b73082b00145fcf6 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.390331 1175 ts_tablet_manager.cc:1434] T d06b77ff1a6f4f78b73082b00145fcf6 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260202 02:52:17.390791 1277 raft_consensus.cc:493] T c1fc01252d95407e954cfcbf7e715912 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.390980 1175 tablet_bootstrap.cc:492] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:17.391242 1277 raft_consensus.cc:515] T c1fc01252d95407e954cfcbf7e715912 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.393386 1175 tablet_bootstrap.cc:654] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.394150 1277 leader_election.cc:290] T c1fc01252d95407e954cfcbf7e715912 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:17.398855 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet c184d44d7f434b39a997a7387e5d3de0 with delete_type TABLET_DATA_DELETED (replaced by tablet 3711910e050c4515bed4294d1271d5a9) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:17.400522 715 leader_election.cc:343] T aa526a69cf914dd08142074636c956b7 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:17.401734 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ff6f5710de4b4c48948aacdf5be29e8b" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:17.401984 1175 tablet_bootstrap.cc:492] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:17.402191 1175 ts_tablet_manager.cc:1403] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260202 02:52:17.403560 1175 raft_consensus.cc:359] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.403805 1175 raft_consensus.cc:385] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.403894 1175 raft_consensus.cc:740] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.404150 1175 consensus_queue.cc:260] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.404942 1175 ts_tablet_manager.cc:1434] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260202 02:52:17.405323 1175 tablet_bootstrap.cc:492] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:17.406000 758 tablet_service.cc:1505] Processing CreateTablet for tablet 75b97d7e848241cf8d894b0ae92a8f8f (DEFAULT_TABLE table=default.Table_5e366ca4b9744597add5ca93f7fe7850 [id=f2e865926cf5453292c34daa4af8be2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.406550 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75b97d7e848241cf8d894b0ae92a8f8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.407075 1040 tablet_service.cc:1552] Processing DeleteTablet for tablet eb0ac55fbb534b29a86d6b3e8aa72ebc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:17.408007 1175 tablet_bootstrap.cc:654] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.408504 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5986676f5484498da927b4c6da3f47ff" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:17.415669 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ff6f5710de4b4c48948aacdf5be29e8b" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:17.416395 1175 tablet_bootstrap.cc:492] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:17.416589 1175 ts_tablet_manager.cc:1403] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.007s
I20260202 02:52:17.417485 1175 raft_consensus.cc:359] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.417698 1175 raft_consensus.cc:385] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.417781 1175 raft_consensus.cc:740] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.418040 1175 consensus_queue.cc:260] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.418756 1175 ts_tablet_manager.cc:1434] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:17.419132 1175 tablet_bootstrap.cc:492] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:17.421669 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c1fc01252d95407e954cfcbf7e715912" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:17.421928 1175 tablet_bootstrap.cc:654] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:17.423800 859 leader_election.cc:343] T ff6f5710de4b4c48948aacdf5be29e8b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:17.424391 715 leader_election.cc:343] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:17.424639 715 leader_election.cc:343] T c1fc01252d95407e954cfcbf7e715912 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:17.425997 859 consensus_peers.cc:597] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5 -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:17.422281 1034 tablet_service.cc:1505] Processing CreateTablet for tablet 75b97d7e848241cf8d894b0ae92a8f8f (DEFAULT_TABLE table=default.Table_5e366ca4b9744597add5ca93f7fe7850 [id=f2e865926cf5453292c34daa4af8be2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.427793 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75b97d7e848241cf8d894b0ae92a8f8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.429003 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet 13309b20a96240b5aa9fbb691cbed981 with delete_type TABLET_DATA_DELETED (replaced by tablet c064ea0dade840b4840a5715c2d83f69) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:17.430979 714 leader_election.cc:343] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 4f2a9404f7424dc78b5bd062ebd8cb53
I20260202 02:52:17.431123 714 leader_election.cc:304] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:17.431222 1175 tablet_bootstrap.cc:492] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:17.431428 1175 ts_tablet_manager.cc:1403] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.004s
I20260202 02:52:17.430811 1035 tablet_service.cc:1505] Processing CreateTablet for tablet eb0ac55fbb534b29a86d6b3e8aa72ebc (DEFAULT_TABLE table=default.Table_8f23e3356aa14793a1aad274d2b845eb [id=21c284c2be45496ca760633f24f090d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.431558 1322 raft_consensus.cc:2749] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:17.432436 1175 raft_consensus.cc:359] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.432713 1175 raft_consensus.cc:385] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.432793 1175 raft_consensus.cc:740] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.433065 1175 consensus_queue.cc:260] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.433629 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb0ac55fbb534b29a86d6b3e8aa72ebc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.433759 1175 ts_tablet_manager.cc:1434] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260202 02:52:17.434885 714 leader_election.cc:343] T aa526a69cf914dd08142074636c956b7 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: aa526a69cf914dd08142074636c956b7
I20260202 02:52:17.435022 714 leader_election.cc:304] T aa526a69cf914dd08142074636c956b7 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:17.435111 1175 tablet_bootstrap.cc:492] T bc1539e5c8aa4d6e98f687728b891530 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:17.435431 1322 raft_consensus.cc:2749] T aa526a69cf914dd08142074636c956b7 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:17.436471 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet c184d44d7f434b39a997a7387e5d3de0 (table default.Table_9451b9f5148f4d9aab65983dea967e32 [id=9ef504467a544f299b48499e49adf27f]) successfully deleted
W20260202 02:52:17.436960 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 526f2c7b95184a0987e7aa94efc8475a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 526f2c7b95184a0987e7aa94efc8475a already in progress: creating tablet
W20260202 02:52:17.437403 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet eb0ac55fbb534b29a86d6b3e8aa72ebc because the tablet was not found. No further retry: Not found: Tablet not found: eb0ac55fbb534b29a86d6b3e8aa72ebc
I20260202 02:52:17.437841 1175 tablet_bootstrap.cc:654] T bc1539e5c8aa4d6e98f687728b891530 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:17.437862 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 9884c90cf1d245b2a7eaaf00d71b1c3e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9884c90cf1d245b2a7eaaf00d71b1c3e already in progress: creating tablet
W20260202 02:52:17.438261 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet c064ea0dade840b4840a5715c2d83f69 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c064ea0dade840b4840a5715c2d83f69 already in progress: creating tablet
I20260202 02:52:17.447333 1338 raft_consensus.cc:493] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.447532 1338 raft_consensus.cc:515] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.448266 1338 leader_election.cc:290] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:17.448638 1175 tablet_bootstrap.cc:492] T bc1539e5c8aa4d6e98f687728b891530 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:17.448841 1175 ts_tablet_manager.cc:1403] T bc1539e5c8aa4d6e98f687728b891530 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.006s
I20260202 02:52:17.449713 1175 raft_consensus.cc:359] T bc1539e5c8aa4d6e98f687728b891530 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:17.449941 1175 raft_consensus.cc:385] T bc1539e5c8aa4d6e98f687728b891530 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.450047 1175 raft_consensus.cc:740] T bc1539e5c8aa4d6e98f687728b891530 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.450282 1175 consensus_queue.cc:260] T bc1539e5c8aa4d6e98f687728b891530 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:17.450618 1338 consensus_queue.cc:1048] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260202 02:52:17.451385 857 leader_election.cc:343] T ff6f5710de4b4c48948aacdf5be29e8b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:17.452057 758 tablet_service.cc:1505] Processing CreateTablet for tablet ef073880898049e2be93d2cb6dc0c0bb (DEFAULT_TABLE table=default.Table_f52d623ce2cb43bf9598994d5298752f [id=78a7fb3a2eed4565b55d5dacb8a3d7df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.452950 857 leader_election.cc:304] T ff6f5710de4b4c48948aacdf5be29e8b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
W20260202 02:52:17.455456 525 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 63a9029bc39c4fbd8ac4bf4bc37d6b4e consensus state has no leader
I20260202 02:52:17.455235 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "63a9029bc39c4fbd8ac4bf4bc37d6b4e" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:17.456219 525 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_78e75899f83c46f09c66dc84192f5438" } modify_external_catalogs: true
I20260202 02:52:17.456467 525 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_78e75899f83c46f09c66dc84192f5438" } modify_external_catalogs: true
I20260202 02:52:17.461730 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef073880898049e2be93d2cb6dc0c0bb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:17.456588 859 leader_election.cc:343] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:17.460777 1175 ts_tablet_manager.cc:1434] T bc1539e5c8aa4d6e98f687728b891530 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.012s user 0.007s sys 0.002s
I20260202 02:52:17.468500 1338 raft_consensus.cc:2749] T ff6f5710de4b4c48948aacdf5be29e8b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:17.469877 1175 tablet_bootstrap.cc:492] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:17.469969 900 tablet_service.cc:1505] Processing CreateTablet for tablet 75b97d7e848241cf8d894b0ae92a8f8f (DEFAULT_TABLE table=default.Table_5e366ca4b9744597add5ca93f7fe7850 [id=f2e865926cf5453292c34daa4af8be2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.470436 903 tablet_service.cc:1505] Processing CreateTablet for tablet ef073880898049e2be93d2cb6dc0c0bb (DEFAULT_TABLE table=default.Table_f52d623ce2cb43bf9598994d5298752f [id=78a7fb3a2eed4565b55d5dacb8a3d7df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.470736 1361 tablet.cc:1722] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:17.469091 898 tablet_service.cc:1505] Processing CreateTablet for tablet 757492288d7c44fb91b25b7e6e21e01b (DEFAULT_TABLE table=default.Table_222f75e4260d41b2813393c25a2a6e14 [id=02e9e243f710428091de2f9497532c37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.469273 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5986676f5484498da927b4c6da3f47ff" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:17.471228 923 raft_consensus.cc:3060] T 5986676f5484498da927b4c6da3f47ff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:17.471314 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75b97d7e848241cf8d894b0ae92a8f8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.471836 1175 tablet_bootstrap.cc:654] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.474815 923 raft_consensus.cc:2468] T 5986676f5484498da927b4c6da3f47ff P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:17.474836 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "63a9029bc39c4fbd8ac4bf4bc37d6b4e" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:17.469558 901 tablet_service.cc:1505] Processing CreateTablet for tablet eb0ac55fbb534b29a86d6b3e8aa72ebc (DEFAULT_TABLE table=default.Table_8f23e3356aa14793a1aad274d2b845eb [id=21c284c2be45496ca760633f24f090d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.475461 903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef073880898049e2be93d2cb6dc0c0bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.475695 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb0ac55fbb534b29a86d6b3e8aa72ebc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:17.475652 714 leader_election.cc:343] T c1fc01252d95407e954cfcbf7e715912 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: c1fc01252d95407e954cfcbf7e715912
I20260202 02:52:17.475786 714 leader_election.cc:304] T c1fc01252d95407e954cfcbf7e715912 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
W20260202 02:52:17.479663 857 leader_election.cc:343] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:17.479853 857 leader_election.cc:304] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:17.480230 1338 raft_consensus.cc:2749] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:17.480571 1338 raft_consensus.cc:493] T 415d14533fd54f20adb00455a7bcad5f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.480746 1338 raft_consensus.cc:515] T 415d14533fd54f20adb00455a7bcad5f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.481297 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 757492288d7c44fb91b25b7e6e21e01b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.481405 1338 leader_election.cc:290] T 415d14533fd54f20adb00455a7bcad5f P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:17.484135 714 leader_election.cc:304] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:17.484632 1321 raft_consensus.cc:2749] T c1fc01252d95407e954cfcbf7e715912 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:17.484869 1321 raft_consensus.cc:2804] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:17.484999 1321 raft_consensus.cc:697] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:17.485247 1321 consensus_queue.cc:237] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.486373 1359 tablet.cc:1722] T 61a9203035f2495f8f99924303ebf120 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:17.489159 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "415d14533fd54f20adb00455a7bcad5f" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:17.492401 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_e749897cf0214764baf40d6c8d5fc316"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:17.493012 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e749897cf0214764baf40d6c8d5fc316 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:17.494797 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_04337834a3ce43ae8f6012b818d70d9f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:17.495406 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_04337834a3ce43ae8f6012b818d70d9f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:17.496683 528 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet eaa47f3388144e8b852804c167f232ce consensus state has no leader
I20260202 02:52:17.497119 528 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_d210d1530706429197d6294b29f18c87" } new_table_name: "default.Table_0c83932cab13463f9cca28129f18b576" modify_external_catalogs: true
I20260202 02:52:17.498857 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_4b9f73d0a12f4218872a70ed1e421519"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:17.499475 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4b9f73d0a12f4218872a70ed1e421519 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:17.500845 521 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 560465c063d94ffaab33f9f79a58a357
I20260202 02:52:17.501689 524 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet fe7b6004e5094c0dbe5847144a2489b9
I20260202 02:52:17.502103 1178 raft_consensus.cc:493] T 72475ea720c6429aa68de9478b374280 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.502137 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "415d14533fd54f20adb00455a7bcad5f" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:17.502374 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_b18a63e4731b418abcb84c36082553c1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:17.502947 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b18a63e4731b418abcb84c36082553c1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:17.502897 859 leader_election.cc:343] T 415d14533fd54f20adb00455a7bcad5f P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:17.503489 1178 raft_consensus.cc:515] T 72475ea720c6429aa68de9478b374280 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.505806 1178 leader_election.cc:290] T 72475ea720c6429aa68de9478b374280 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:17.506181 1040 tablet_service.cc:1505] Processing CreateTablet for tablet 757492288d7c44fb91b25b7e6e21e01b (DEFAULT_TABLE table=default.Table_222f75e4260d41b2813393c25a2a6e14 [id=02e9e243f710428091de2f9497532c37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.506693 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 757492288d7c44fb91b25b7e6e21e01b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.508258 520 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_3d26f7a34d0c48379e3cebda66e1ef13 [id=45e6d83589fa493195b06f633e295b00] alter complete (version 2)
I20260202 02:52:17.511441 1038 tablet_service.cc:1505] Processing CreateTablet for tablet ef073880898049e2be93d2cb6dc0c0bb (DEFAULT_TABLE table=default.Table_f52d623ce2cb43bf9598994d5298752f [id=78a7fb3a2eed4565b55d5dacb8a3d7df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.513249 521 catalog_manager.cc:6724] ReplaceTablet: tablet 560465c063d94ffaab33f9f79a58a357 deleted and replaced by tablet 20ea39bc5d3d46f5a4ba427277d6c6a3
I20260202 02:52:17.513693 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_26642341d3d54c0fa9af19fef3a5b06d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:17.514367 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_26642341d3d54c0fa9af19fef3a5b06d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:17.517289 1174 tablet_bootstrap.cc:492] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:17.517387 758 tablet_service.cc:1552] Processing DeleteTablet for tablet 560465c063d94ffaab33f9f79a58a357 with delete_type TABLET_DATA_DELETED (replaced by tablet 20ea39bc5d3d46f5a4ba427277d6c6a3) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:17.517500 1174 ts_tablet_manager.cc:1403] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.220s user 0.011s sys 0.002s
I20260202 02:52:17.518357 1174 raft_consensus.cc:359] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:17.518573 1174 raft_consensus.cc:385] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.518640 1174 raft_consensus.cc:740] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.518884 1174 consensus_queue.cc:260] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:17.519407 1174 ts_tablet_manager.cc:1434] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260202 02:52:17.519752 1174 tablet_bootstrap.cc:492] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:17.523751 1338 raft_consensus.cc:493] T bf5ca99708044482970cb854d2eced18 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.523965 1338 raft_consensus.cc:515] T bf5ca99708044482970cb854d2eced18 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.524745 1338 leader_election.cc:290] T bf5ca99708044482970cb854d2eced18 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:17.525521 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 560465c063d94ffaab33f9f79a58a357 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 560465c063d94ffaab33f9f79a58a357 already in progress: creating tablet
I20260202 02:52:17.525693 1043 tablet_service.cc:1552] Processing DeleteTablet for tablet fe7b6004e5094c0dbe5847144a2489b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:17.527091 998 leader_election.cc:343] T 72475ea720c6429aa68de9478b374280 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Not found: Tablet not found: 72475ea720c6429aa68de9478b374280
I20260202 02:52:17.527557 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf5ca99708044482970cb854d2eced18" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:17.528502 859 leader_election.cc:343] T bf5ca99708044482970cb854d2eced18 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:17.529961 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef073880898049e2be93d2cb6dc0c0bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.533245 525 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 80d443bdb66d4cc68ae283e08c410fe6
I20260202 02:52:17.534170 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_2ca7a7f464b54d398f58e0ca6a5e269b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:17.534535 1174 tablet_bootstrap.cc:654] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:17.534830 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2ca7a7f464b54d398f58e0ca6a5e269b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:17.534624 1041 tablet_service.cc:1505] Processing CreateTablet for tablet 78f122342dd144b48de89902faa0f94f (DEFAULT_TABLE table=default.Table_5eefa56618e84f8d92f8d87132ccd7d4 [id=b4ab2ba86e1a49338a98d855ded99f5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.537081 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78f122342dd144b48de89902faa0f94f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.538019 1043 tablet_service.cc:1552] Processing DeleteTablet for tablet 560465c063d94ffaab33f9f79a58a357 with delete_type TABLET_DATA_DELETED (replaced by tablet 20ea39bc5d3d46f5a4ba427277d6c6a3) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:17.538244 758 tablet_service.cc:1552] Processing DeleteTablet for tablet 80d443bdb66d4cc68ae283e08c410fe6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:42950
W20260202 02:52:17.541532 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 80d443bdb66d4cc68ae283e08c410fe6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 80d443bdb66d4cc68ae283e08c410fe6 already in progress: creating tablet
I20260202 02:52:17.545696 1174 tablet_bootstrap.cc:492] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:17.545889 1174 ts_tablet_manager.cc:1403] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.002s
I20260202 02:52:17.546761 1174 raft_consensus.cc:359] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:17.546977 1174 raft_consensus.cc:385] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.547089 1174 raft_consensus.cc:740] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.547307 1444 tablet.cc:1722] T fae7e3941c124cc2871390a0b6471f27 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:17.548152 1043 tablet_service.cc:1552] Processing DeleteTablet for tablet 80d443bdb66d4cc68ae283e08c410fe6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:17.549088 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf5ca99708044482970cb854d2eced18" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:17.550518 900 tablet_service.cc:1552] Processing DeleteTablet for tablet fe7b6004e5094c0dbe5847144a2489b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:17.550805 903 tablet_service.cc:1552] Processing DeleteTablet for tablet 560465c063d94ffaab33f9f79a58a357 with delete_type TABLET_DATA_DELETED (replaced by tablet 20ea39bc5d3d46f5a4ba427277d6c6a3) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:17.550858 857 leader_election.cc:343] T 415d14533fd54f20adb00455a7bcad5f P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:17.551024 857 leader_election.cc:304] T 415d14533fd54f20adb00455a7bcad5f P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:17.551410 1338 raft_consensus.cc:2749] T 415d14533fd54f20adb00455a7bcad5f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:17.551561 1174 consensus_queue.cc:260] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
W20260202 02:52:17.551837 857 leader_election.cc:343] T bf5ca99708044482970cb854d2eced18 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:17.551956 857 leader_election.cc:304] T bf5ca99708044482970cb854d2eced18 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:17.552227 1338 raft_consensus.cc:2749] T bf5ca99708044482970cb854d2eced18 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:17.552672 1182 tablet_replica.cc:333] T fe7b6004e5094c0dbe5847144a2489b9 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:17.552888 1182 raft_consensus.cc:2243] T fe7b6004e5094c0dbe5847144a2489b9 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260202 02:52:17.552865 999 leader_election.cc:343] T 72475ea720c6429aa68de9478b374280 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 72475ea720c6429aa68de9478b374280
I20260202 02:52:17.552994 999 leader_election.cc:304] T 72475ea720c6429aa68de9478b374280 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:17.553047 1182 raft_consensus.cc:2272] T fe7b6004e5094c0dbe5847144a2489b9 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:17.553416 1178 raft_consensus.cc:2749] T 72475ea720c6429aa68de9478b374280 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:17.553376 1321 consensus_queue.cc:1048] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:17.554245 1063 raft_consensus.cc:3060] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:17.551036 898 tablet_service.cc:1552] Processing DeleteTablet for tablet 80d443bdb66d4cc68ae283e08c410fe6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:17.555318 1182 ts_tablet_manager.cc:1916] T fe7b6004e5094c0dbe5847144a2489b9 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:17.555337 1174 ts_tablet_manager.cc:1434] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260202 02:52:17.556370 1174 tablet_bootstrap.cc:492] T bf5ca99708044482970cb854d2eced18 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:17.559122 1359 tablet.cc:1722] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260202 02:52:17.560788 1361 tablet.cc:1722] T ffd8aa2bc6e344b78dce518411997fb5 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260202 02:52:17.565202 1174 tablet_bootstrap.cc:654] T bf5ca99708044482970cb854d2eced18 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.573670 1182 ts_tablet_manager.cc:1929] T fe7b6004e5094c0dbe5847144a2489b9 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:17.574146 1182 log.cc:1199] T fe7b6004e5094c0dbe5847144a2489b9 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/fe7b6004e5094c0dbe5847144a2489b9
I20260202 02:52:17.574642 1182 ts_tablet_manager.cc:1950] T fe7b6004e5094c0dbe5847144a2489b9 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:17.580844 1182 tablet_replica.cc:333] T 560465c063d94ffaab33f9f79a58a357 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:17.581264 1182 raft_consensus.cc:2243] T 560465c063d94ffaab33f9f79a58a357 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:17.581534 1182 raft_consensus.cc:2272] T 560465c063d94ffaab33f9f79a58a357 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:17.582201 1182 ts_tablet_manager.cc:1916] T 560465c063d94ffaab33f9f79a58a357 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:17.582293 1173 tablet_bootstrap.cc:492] T 251455168e33430aafea8e0de42ca1b8 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:17.582594 1173 ts_tablet_manager.cc:1403] T 251455168e33430aafea8e0de42ca1b8 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.283s user 0.008s sys 0.004s
I20260202 02:52:17.582722 753 tablet_service.cc:1552] Processing DeleteTablet for tablet fe7b6004e5094c0dbe5847144a2489b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:42950
W20260202 02:52:17.583516 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet fe7b6004e5094c0dbe5847144a2489b9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fe7b6004e5094c0dbe5847144a2489b9 already in progress: creating tablet
I20260202 02:52:17.584756 1173 raft_consensus.cc:359] T 251455168e33430aafea8e0de42ca1b8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.584980 1173 raft_consensus.cc:385] T 251455168e33430aafea8e0de42ca1b8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.585166 1173 raft_consensus.cc:740] T 251455168e33430aafea8e0de42ca1b8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.585446 1173 consensus_queue.cc:260] T 251455168e33430aafea8e0de42ca1b8 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.587845 1173 ts_tablet_manager.cc:1434] T 251455168e33430aafea8e0de42ca1b8 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260202 02:52:17.596112 1174 tablet_bootstrap.cc:492] T bf5ca99708044482970cb854d2eced18 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:17.598387 1174 ts_tablet_manager.cc:1403] T bf5ca99708044482970cb854d2eced18 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.042s user 0.007s sys 0.007s
I20260202 02:52:17.599401 1174 raft_consensus.cc:359] T bf5ca99708044482970cb854d2eced18 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.602062 1173 tablet_bootstrap.cc:492] T add934bf70ae4de8a40066f06fd94795 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:17.602298 1174 raft_consensus.cc:385] T bf5ca99708044482970cb854d2eced18 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.602715 1174 raft_consensus.cc:740] T bf5ca99708044482970cb854d2eced18 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.603973 1173 tablet_bootstrap.cc:654] T add934bf70ae4de8a40066f06fd94795 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.604544 1174 consensus_queue.cc:260] T bf5ca99708044482970cb854d2eced18 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.605261 1174 ts_tablet_manager.cc:1434] T bf5ca99708044482970cb854d2eced18 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20260202 02:52:17.605698 1174 tablet_bootstrap.cc:492] T 724d47241f0342768fb1321ef37c3a7b P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:17.608872 1174 tablet_bootstrap.cc:654] T 724d47241f0342768fb1321ef37c3a7b P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.609906 1183 ts_tablet_manager.cc:1929] T fc93352b24e345f99d15085e4886c748 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:17.610131 1183 log.cc:1199] T fc93352b24e345f99d15085e4886c748 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/fc93352b24e345f99d15085e4886c748
I20260202 02:52:17.610584 1183 ts_tablet_manager.cc:1950] T fc93352b24e345f99d15085e4886c748 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:17.610899 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet fe7b6004e5094c0dbe5847144a2489b9 (table default.Table_a6b938c21e164294bd8b5fb205673f75 [id=1abad082d47d4f31be7c697ed72e241c]) successfully deleted
I20260202 02:52:17.613436 1182 ts_tablet_manager.cc:1929] T 560465c063d94ffaab33f9f79a58a357 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:17.613616 1182 log.cc:1199] T 560465c063d94ffaab33f9f79a58a357 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/560465c063d94ffaab33f9f79a58a357
I20260202 02:52:17.614051 1182 ts_tablet_manager.cc:1950] T 560465c063d94ffaab33f9f79a58a357 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:17.609022 1324 raft_consensus.cc:493] T 0d297b6243c348ee82c146a8339cf6c4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.615685 1183 tablet_replica.cc:333] T 72475ea720c6429aa68de9478b374280 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:17.615666 1324 raft_consensus.cc:515] T 0d297b6243c348ee82c146a8339cf6c4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:17.617291 1324 leader_election.cc:290] T 0d297b6243c348ee82c146a8339cf6c4 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:17.617632 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet fc93352b24e345f99d15085e4886c748 (table default.Table_8aee8d27e02942fb91879229b190457f [id=c59468c8c5b7480d88cb4b7d447a5975]) successfully deleted
I20260202 02:52:17.617412 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0d297b6243c348ee82c146a8339cf6c4" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:17.621619 1183 raft_consensus.cc:2243] T 72475ea720c6429aa68de9478b374280 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:17.621827 1183 raft_consensus.cc:2272] T 72475ea720c6429aa68de9478b374280 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:17.622622 715 leader_election.cc:343] T 0d297b6243c348ee82c146a8339cf6c4 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:17.623536 1444 tablet.cc:1722] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:17.624534 1174 tablet_bootstrap.cc:492] T 724d47241f0342768fb1321ef37c3a7b P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:17.624717 1174 ts_tablet_manager.cc:1403] T 724d47241f0342768fb1321ef37c3a7b P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.002s
I20260202 02:52:17.626068 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 560465c063d94ffaab33f9f79a58a357 (table default.Table_3b335a0806154f98a907f551596ba353 [id=0da4db61b3fe4cf4846edd5530b9327f]) successfully deleted
W20260202 02:52:17.626386 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 80d443bdb66d4cc68ae283e08c410fe6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 80d443bdb66d4cc68ae283e08c410fe6 already in progress: creating tablet
W20260202 02:52:17.626794 714 leader_election.cc:343] T 0d297b6243c348ee82c146a8339cf6c4 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 0d297b6243c348ee82c146a8339cf6c4
I20260202 02:52:17.626972 714 leader_election.cc:304] T 0d297b6243c348ee82c146a8339cf6c4 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:17.627038 1174 raft_consensus.cc:359] T 724d47241f0342768fb1321ef37c3a7b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:17.627245 1174 raft_consensus.cc:385] T 724d47241f0342768fb1321ef37c3a7b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.627336 1174 raft_consensus.cc:740] T 724d47241f0342768fb1321ef37c3a7b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.627588 1174 consensus_queue.cc:260] T 724d47241f0342768fb1321ef37c3a7b P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:17.628175 1174 ts_tablet_manager.cc:1434] T 724d47241f0342768fb1321ef37c3a7b P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260202 02:52:17.628553 1174 tablet_bootstrap.cc:492] T cb0dcacdf0f1419da0b25f20cac1d62d P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:17.629209 1277 raft_consensus.cc:2749] T 0d297b6243c348ee82c146a8339cf6c4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:17.631464 1174 tablet_bootstrap.cc:654] T cb0dcacdf0f1419da0b25f20cac1d62d P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:17.633378 528 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d64f42d187bc4c9f9381c931449ec0cc has no consensus state
I20260202 02:52:17.634980 1324 raft_consensus.cc:493] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.635264 1324 raft_consensus.cc:515] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:17.636854 528 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_fe76707af2ff46d1b44c46cef9e88be4" } new_table_name: "default.Table_12d68a317c8e4d809dbbe77c095a1f6e" modify_external_catalogs: true
I20260202 02:52:17.638537 1173 tablet_bootstrap.cc:492] T add934bf70ae4de8a40066f06fd94795 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:17.638729 1173 ts_tablet_manager.cc:1403] T add934bf70ae4de8a40066f06fd94795 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.037s user 0.005s sys 0.000s
I20260202 02:52:17.639544 1173 raft_consensus.cc:359] T add934bf70ae4de8a40066f06fd94795 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.639779 1173 raft_consensus.cc:385] T add934bf70ae4de8a40066f06fd94795 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.639869 1173 raft_consensus.cc:740] T add934bf70ae4de8a40066f06fd94795 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.641707 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "23c38ef4e3c840808ec36f859cf6f6de" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:17.643448 1183 ts_tablet_manager.cc:1916] T 72475ea720c6429aa68de9478b374280 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:17.645139 1324 leader_election.cc:290] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:17.646395 1173 consensus_queue.cc:260] T add934bf70ae4de8a40066f06fd94795 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:17.645401 715 leader_election.cc:343] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:17.647094 1174 tablet_bootstrap.cc:492] T cb0dcacdf0f1419da0b25f20cac1d62d P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:17.648890 1174 ts_tablet_manager.cc:1403] T cb0dcacdf0f1419da0b25f20cac1d62d P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.005s
I20260202 02:52:17.649533 1173 ts_tablet_manager.cc:1434] T add934bf70ae4de8a40066f06fd94795 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.011s user 0.003s sys 0.000s
I20260202 02:52:17.649897 1174 raft_consensus.cc:359] T cb0dcacdf0f1419da0b25f20cac1d62d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.650187 1174 raft_consensus.cc:385] T cb0dcacdf0f1419da0b25f20cac1d62d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.650274 1174 raft_consensus.cc:740] T cb0dcacdf0f1419da0b25f20cac1d62d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.650601 1174 consensus_queue.cc:260] T cb0dcacdf0f1419da0b25f20cac1d62d P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.651202 1174 ts_tablet_manager.cc:1434] T cb0dcacdf0f1419da0b25f20cac1d62d P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:17.651585 1174 tablet_bootstrap.cc:492] T 560465c063d94ffaab33f9f79a58a357 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:17.653486 1174 tablet_bootstrap.cc:654] T 560465c063d94ffaab33f9f79a58a357 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.653571 1173 tablet_bootstrap.cc:492] T 575016c0e9f84e99b962c5f14ffa96cc P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:17.654558 922 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "23c38ef4e3c840808ec36f859cf6f6de" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
W20260202 02:52:17.654841 715 consensus_peers.cc:597] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:17.654839 922 raft_consensus.cc:2468] T 23c38ef4e3c840808ec36f859cf6f6de P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:17.663089 1324 raft_consensus.cc:493] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.663610 1324 raft_consensus.cc:515] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:17.667004 714 leader_election.cc:304] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:17.667405 1321 raft_consensus.cc:2804] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:17.667671 1321 raft_consensus.cc:493] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:17.667867 1321 raft_consensus.cc:3060] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:17.668597 1173 tablet_bootstrap.cc:654] T 575016c0e9f84e99b962c5f14ffa96cc P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.670028 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ea040c03420c4b1b98897f39160b4130" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:17.670698 715 leader_election.cc:343] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:17.671461 1321 raft_consensus.cc:515] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:17.672302 1321 leader_election.cc:290] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:17.673270 922 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ea040c03420c4b1b98897f39160b4130" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:17.673530 922 raft_consensus.cc:2468] T ea040c03420c4b1b98897f39160b4130 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:17.673913 921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "23c38ef4e3c840808ec36f859cf6f6de" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:17.673954 714 leader_election.cc:304] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:17.674067 921 raft_consensus.cc:3060] T 23c38ef4e3c840808ec36f859cf6f6de P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:17.674255 1321 raft_consensus.cc:2804] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:17.674340 1321 raft_consensus.cc:493] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:17.674427 1321 raft_consensus.cc:3060] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:17.677284 1183 ts_tablet_manager.cc:1929] T 72475ea720c6429aa68de9478b374280 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:17.677309 1321 raft_consensus.cc:515] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:17.677479 1183 log.cc:1199] T 72475ea720c6429aa68de9478b374280 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/72475ea720c6429aa68de9478b374280
I20260202 02:52:17.678014 921 raft_consensus.cc:2468] T 23c38ef4e3c840808ec36f859cf6f6de P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:17.679841 1174 tablet_bootstrap.cc:492] T 560465c063d94ffaab33f9f79a58a357 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:17.680017 1174 ts_tablet_manager.cc:1403] T 560465c063d94ffaab33f9f79a58a357 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.029s user 0.000s sys 0.008s
I20260202 02:52:17.680297 1321 leader_election.cc:290] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:17.680995 1174 raft_consensus.cc:359] T 560465c063d94ffaab33f9f79a58a357 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.681223 1174 raft_consensus.cc:385] T 560465c063d94ffaab33f9f79a58a357 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.681246 1324 leader_election.cc:290] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:17.681334 1174 raft_consensus.cc:740] T 560465c063d94ffaab33f9f79a58a357 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.678788 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ea040c03420c4b1b98897f39160b4130" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:17.681623 1174 consensus_queue.cc:260] T 560465c063d94ffaab33f9f79a58a357 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:17.682261 715 leader_election.cc:343] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:17.682870 1183 ts_tablet_manager.cc:1950] T 72475ea720c6429aa68de9478b374280 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:17.682904 714 leader_election.cc:304] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters:
I20260202 02:52:17.682190 1174 ts_tablet_manager.cc:1434] T 560465c063d94ffaab33f9f79a58a357 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260202 02:52:17.683738 1174 tablet_bootstrap.cc:492] T bf74f96bea2e4ab7adec42c3c5207e79 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:17.685659 1277 raft_consensus.cc:2804] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:17.686039 1277 raft_consensus.cc:697] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:17.686452 1175 tablet_bootstrap.cc:492] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:17.686369 1277 consensus_queue.cc:237] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:17.687204 1175 ts_tablet_manager.cc:1403] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.217s user 0.027s sys 0.023s
I20260202 02:52:17.687943 1174 tablet_bootstrap.cc:654] T bf74f96bea2e4ab7adec42c3c5207e79 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.688083 1175 raft_consensus.cc:359] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:17.689338 514 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet d06b77ff1a6f4f78b73082b00145fcf6 (table default.Table_5308cabde3e94be1ab7a0b4e15ad67a9 [id=915b211258ad498286448a5b2598a289], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet d06b77ff1a6f4f78b73082b00145fcf6 consensus state has no leader
I20260202 02:52:17.690028 1175 raft_consensus.cc:385] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.691569 1175 raft_consensus.cc:740] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.692013 1175 consensus_queue.cc:260] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.692855 1175 ts_tablet_manager.cc:1434] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260202 02:52:17.695221 921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ea040c03420c4b1b98897f39160b4130" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:17.695403 1175 tablet_bootstrap.cc:492] T f58a7b25815c4b5bac568aa11c19610c P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:17.695991 921 raft_consensus.cc:3060] T ea040c03420c4b1b98897f39160b4130 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:17.697538 1175 tablet_bootstrap.cc:654] T f58a7b25815c4b5bac568aa11c19610c P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.698477 1321 raft_consensus.cc:493] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.698657 1321 raft_consensus.cc:515] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:17.684285 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "23c38ef4e3c840808ec36f859cf6f6de" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:17.699800 921 raft_consensus.cc:2468] T ea040c03420c4b1b98897f39160b4130 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
W20260202 02:52:17.699810 715 leader_election.cc:343] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:17.699721 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bd7b0a0fcb7645fdb9edfeccb3d18268" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:17.700352 715 leader_election.cc:343] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:17.700479 1321 leader_election.cc:290] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:17.701196 714 leader_election.cc:304] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:17.702142 1277 raft_consensus.cc:2804] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:17.702159 921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bd7b0a0fcb7645fdb9edfeccb3d18268" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:17.702337 1277 raft_consensus.cc:697] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:17.702677 1277 consensus_queue.cc:237] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:17.703141 1183 tablet_replica.cc:333] T 672ea3f731ba42ff8b1ff47db039ed22 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:17.704041 921 raft_consensus.cc:2468] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:17.705505 714 leader_election.cc:304] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:17.706065 1324 raft_consensus.cc:2804] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:17.706215 1324 raft_consensus.cc:493] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:17.706372 1324 raft_consensus.cc:3060] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:17.706825 1183 raft_consensus.cc:2243] T 672ea3f731ba42ff8b1ff47db039ed22 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:17.707082 1183 raft_consensus.cc:2272] T 672ea3f731ba42ff8b1ff47db039ed22 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:17.707686 1183 ts_tablet_manager.cc:1916] T 672ea3f731ba42ff8b1ff47db039ed22 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:17.709074 1174 tablet_bootstrap.cc:492] T bf74f96bea2e4ab7adec42c3c5207e79 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:17.709275 1174 ts_tablet_manager.cc:1403] T bf74f96bea2e4ab7adec42c3c5207e79 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.026s user 0.004s sys 0.002s
I20260202 02:52:17.709569 1324 raft_consensus.cc:515] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:17.710258 1174 raft_consensus.cc:359] T bf74f96bea2e4ab7adec42c3c5207e79 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:17.710484 1174 raft_consensus.cc:385] T bf74f96bea2e4ab7adec42c3c5207e79 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.710562 1174 raft_consensus.cc:740] T bf74f96bea2e4ab7adec42c3c5207e79 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.710851 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bd7b0a0fcb7645fdb9edfeccb3d18268" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:17.710897 1174 consensus_queue.cc:260] T bf74f96bea2e4ab7adec42c3c5207e79 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
W20260202 02:52:17.711479 715 leader_election.cc:343] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:17.711542 921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bd7b0a0fcb7645fdb9edfeccb3d18268" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:17.711741 921 raft_consensus.cc:3060] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:17.712702 515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.190:33981, user_credentials={real_user=slave}} blocked reactor thread for 50746.9us
I20260202 02:52:17.714841 921 raft_consensus.cc:2468] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:17.716012 1174 ts_tablet_manager.cc:1434] T bf74f96bea2e4ab7adec42c3c5207e79 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20260202 02:52:17.716543 1174 tablet_bootstrap.cc:492] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:17.717237 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 72475ea720c6429aa68de9478b374280 (table default.Table_09d8f825c4c546839c0a820f1ddf0047 [id=ce957713e7fb49928d3fe57c3682f26f]) successfully deleted
I20260202 02:52:17.717352 1175 tablet_bootstrap.cc:492] T f58a7b25815c4b5bac568aa11c19610c P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:17.717501 1175 ts_tablet_manager.cc:1403] T f58a7b25815c4b5bac568aa11c19610c P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.004s
W20260202 02:52:17.717515 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet cb0dcacdf0f1419da0b25f20cac1d62d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cb0dcacdf0f1419da0b25f20cac1d62d already in progress: creating tablet
W20260202 02:52:17.717729 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 0d297b6243c348ee82c146a8339cf6c4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0d297b6243c348ee82c146a8339cf6c4 already in progress: creating tablet
W20260202 02:52:17.717963 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet bf74f96bea2e4ab7adec42c3c5207e79 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bf74f96bea2e4ab7adec42c3c5207e79 already in progress: creating tablet
I20260202 02:52:17.718348 1175 raft_consensus.cc:359] T f58a7b25815c4b5bac568aa11c19610c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.718538 1175 raft_consensus.cc:385] T f58a7b25815c4b5bac568aa11c19610c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.718638 1175 raft_consensus.cc:740] T f58a7b25815c4b5bac568aa11c19610c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.718868 1175 consensus_queue.cc:260] T f58a7b25815c4b5bac568aa11c19610c P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.722383 1174 tablet_bootstrap.cc:654] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.725641 1175 ts_tablet_manager.cc:1434] T f58a7b25815c4b5bac568aa11c19610c P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260202 02:52:17.730324 1175 tablet_bootstrap.cc:492] T c064ea0dade840b4840a5715c2d83f69 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:17.733501 524 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_c245b83c1987454fabbe8646d14252d6" } modify_external_catalogs: true
I20260202 02:52:17.733718 524 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_c245b83c1987454fabbe8646d14252d6" } modify_external_catalogs: true
I20260202 02:52:17.734176 523 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_e70b044a267b465d94fede069f2f0df6 [id=1229c6d37b994a669d3363fdf6f8c6c8] alter complete (version 1)
I20260202 02:52:17.734829 1183 ts_tablet_manager.cc:1929] T 672ea3f731ba42ff8b1ff47db039ed22 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:17.734957 1183 log.cc:1199] T 672ea3f731ba42ff8b1ff47db039ed22 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/672ea3f731ba42ff8b1ff47db039ed22
I20260202 02:52:17.735344 1183 ts_tablet_manager.cc:1950] T 672ea3f731ba42ff8b1ff47db039ed22 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
W20260202 02:52:17.735309 523 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.104.129:48435 (request call id 30) took 955 ms (client timeout 1000 ms). Trace:
W20260202 02:52:17.735404 523 rpcz_store.cc:269] 0202 02:52:16.779876 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:16.791356 (+ 11480us) service_pool.cc:225] Handling call
0202 02:52:16.799748 (+ 8392us) spinlock_profiling.cc:243] Waited 7.87 ms on lock 0x616000041254. stack: 00007fc023f2da86 00007fc023f2d840 00007fc022f083ad 0000000000984f94 0000000000984f5d 00007fc0332576bd 00007fc0332577b3 00007fc02fe1e521 00007fc037ae96fd 00007fc037c705d8 00007fc02e771ca6 00007fc02e7719d5 00007fc02aef055e 00007fc02aeef43e 00007fc02aef5675 00007fc02aef5f9f
0202 02:52:17.735231 (+935483us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:17.441607 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:17.441712 (+ 105us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:17.441718 (+ 6us) write_op.cc:435] Acquired schema lock
0202 02:52:17.441721 (+ 3us) tablet.cc:660] Decoding operations
0202 02:52:17.441819 (+ 98us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:17.441830 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:17.441835 (+ 5us) tablet.cc:683] Acquiring locks for 1 operations
0202 02:52:17.441888 (+ 53us) tablet.cc:699] Row locks acquired
0202 02:52:17.441890 (+ 2us) write_op.cc:260] PREPARE: finished
0202 02:52:17.441926 (+ 36us) write_op.cc:270] Start()
0202 02:52:17.441957 (+ 31us) write_op.cc:276] Timestamp: P: 1770000737441862 usec, L: 0
0202 02:52:17.441960 (+ 3us) op_driver.cc:348] REPLICATION: starting
0202 02:52:17.442366 (+ 406us) log.cc:844] Serialized 533 byte log entry
0202 02:52:17.487609 (+ 45243us) op_driver.cc:464] REPLICATION: finished
0202 02:52:17.490493 (+ 2884us) write_op.cc:301] APPLY: starting
0202 02:52:17.490503 (+ 10us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:17.490527 (+ 24us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:17.490564 (+ 37us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:17.490596 (+ 32us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:17.490618 (+ 22us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0202 02:52:17.490625 (+ 7us) write_op.cc:312] APPLY: finished
0202 02:52:17.490670 (+ 45us) log.cc:844] Serialized 23 byte log entry
0202 02:52:17.490716 (+ 46us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:17.490727 (+ 11us) write_op.cc:454] Released schema lock
0202 02:52:17.490732 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0202 02:52:17.490788 (+ 56us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:17.490792 (+ 4us) write_op.cc:454] Released schema lock
Related trace 'op':
0202 02:52:17.654974 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:17.655217 (+ 243us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:17.655225 (+ 8us) write_op.cc:435] Acquired schema lock
0202 02:52:17.655227 (+ 2us) tablet.cc:660] Decoding operations
0202 02:52:17.655346 (+ 119us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:17.655360 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:17.655365 (+ 5us) tablet.cc:683] Acquiring locks for 1 operations
0202 02:52:17.655431 (+ 66us) tablet.cc:699] Row locks acquired
0202 02:52:17.655434 (+ 3us) write_op.cc:260] PREPARE: finished
0202 02:52:17.655479 (+ 45us) write_op.cc:270] Start()
0202 02:52:17.655517 (+ 38us) write_op.cc:276] Timestamp: P: 1770000737655392 usec, L: 0
0202 02:52:17.655521 (+ 4us) op_driver.cc:348] REPLICATION: starting
0202 02:52:17.655735 (+ 214us) log.cc:844] Serialized 356 byte log entry
0202 02:52:17.729482 (+ 73747us) op_driver.cc:464] REPLICATION: finished
0202 02:52:17.730403 (+ 921us) write_op.cc:301] APPLY: starting
0202 02:52:17.730414 (+ 11us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:17.730446 (+ 32us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:17.730468 (+ 22us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:17.730511 (+ 43us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:17.730537 (+ 26us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0202 02:52:17.730544 (+ 7us) write_op.cc:312] APPLY: finished
0202 02:52:17.730596 (+ 52us) log.cc:844] Serialized 23 byte log entry
0202 02:52:17.730643 (+ 47us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:17.730655 (+ 12us) write_op.cc:454] Released schema lock
0202 02:52:17.730660 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0202 02:52:17.730724 (+ 64us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:17.730728 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":92,"reported_tablets":8,"spinlock_wait_cycles":18090368,"child_traces":[["op",{"apply.queue_time_us":2856,"apply.run_cpu_time_us":288,"apply.run_wall_time_us":287,"num_ops":1,"prepare.queue_time_us":115665,"prepare.run_cpu_time_us":699,"prepare.run_wall_time_us":934,"raft.queue_time_us":37,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":6,"replication_time_us":45635}],["op",{"apply.queue_time_us":890,"apply.run_cpu_time_us":313,"apply.run_wall_time_us":313,"num_ops":1,"prepare.queue_time_us":147832,"prepare.run_cpu_time_us":819,"prepare.run_wall_time_us":1390,"raft.queue_time_us":44,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":6,"replication_time_us":73945}]]}
I20260202 02:52:17.736639 526 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet fae7e3941c124cc2871390a0b6471f27
I20260202 02:52:17.737107 526 catalog_manager.cc:6724] ReplaceTablet: tablet fae7e3941c124cc2871390a0b6471f27 deleted and replaced by tablet a5622ef925bc46cca76f481ddfd12afd
I20260202 02:52:17.737485 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_0eb450e3edbb4a4ba0906567f831830c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:17.738040 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0eb450e3edbb4a4ba0906567f831830c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:17.739499 521 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_5e366ca4b9744597add5ca93f7fe7850" } modify_external_catalogs: true
I20260202 02:52:17.739663 521 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_5e366ca4b9744597add5ca93f7fe7850" } modify_external_catalogs: true
I20260202 02:52:17.740422 1174 tablet_bootstrap.cc:492] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:17.740605 1174 ts_tablet_manager.cc:1403] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.024s user 0.007s sys 0.000s
I20260202 02:52:17.740658 1175 tablet_bootstrap.cc:654] T c064ea0dade840b4840a5715c2d83f69 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.740907 522 master_service.cc:954] ReplaceTablet: received request to replace tablet f58a7b25815c4b5bac568aa11c19610c from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:17.741652 1174 raft_consensus.cc:359] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.741874 1174 raft_consensus.cc:385] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.741957 1174 raft_consensus.cc:740] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.742178 1174 consensus_queue.cc:260] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.742743 1174 ts_tablet_manager.cc:1434] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260202 02:52:17.743093 1174 tablet_bootstrap.cc:492] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
W20260202 02:52:17.743366 520 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 560465c063d94ffaab33f9f79a58a357 (table default.Table_3b335a0806154f98a907f551596ba353 [id=0da4db61b3fe4cf4846edd5530b9327f], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 560465c063d94ffaab33f9f79a58a357 consensus state has no leader
W20260202 02:52:17.744148 520 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.104.130:36691 (request call id 32) took 984 ms (client timeout 1000 ms). Trace:
I20260202 02:52:17.744621 1183 tablet_replica.cc:333] T 4de41396cffd4d32b86c701b538d1e01 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:17.744850 1183 raft_consensus.cc:2243] T 4de41396cffd4d32b86c701b538d1e01 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:17.745054 1183 raft_consensus.cc:2272] T 4de41396cffd4d32b86c701b538d1e01 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:17.745100 1174 tablet_bootstrap.cc:654] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.745329 527 master_service.cc:954] ReplaceTablet: received request to replace tablet bc1539e5c8aa4d6e98f687728b891530 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:17.745680 1183 ts_tablet_manager.cc:1916] T 4de41396cffd4d32b86c701b538d1e01 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:17.748900 1175 tablet_bootstrap.cc:492] T c064ea0dade840b4840a5715c2d83f69 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:17.749099 1175 ts_tablet_manager.cc:1403] T c064ea0dade840b4840a5715c2d83f69 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.003s
I20260202 02:52:17.751268 1174 tablet_bootstrap.cc:492] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:17.751448 1174 ts_tablet_manager.cc:1403] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.003s
I20260202 02:52:17.752297 1174 raft_consensus.cc:359] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:17.752573 1174 raft_consensus.cc:385] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.752640 1174 raft_consensus.cc:740] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.752894 1174 consensus_queue.cc:260] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:17.753389 1174 ts_tablet_manager.cc:1434] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260202 02:52:17.753741 1174 tablet_bootstrap.cc:492] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:17.755661 1174 tablet_bootstrap.cc:654] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.755652 519 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_024075b20add4ddcbc4b27601a94eed4" } new_table_name: "default.Table_8c47663f416b467cad37110bd09d85ad" modify_external_catalogs: true
I20260202 02:52:17.749883 1175 raft_consensus.cc:359] T c064ea0dade840b4840a5715c2d83f69 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.756857 1175 raft_consensus.cc:385] T c064ea0dade840b4840a5715c2d83f69 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.757274 1175 raft_consensus.cc:740] T c064ea0dade840b4840a5715c2d83f69 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.757573 1175 consensus_queue.cc:260] T c064ea0dade840b4840a5715c2d83f69 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.758982 1175 ts_tablet_manager.cc:1434] T c064ea0dade840b4840a5715c2d83f69 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20260202 02:52:17.759580 758 tablet_service.cc:1552] Processing DeleteTablet for tablet c1fc01252d95407e954cfcbf7e715912 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:17.759845 754 tablet_service.cc:1552] Processing DeleteTablet for tablet fae7e3941c124cc2871390a0b6471f27 with delete_type TABLET_DATA_DELETED (replaced by tablet a5622ef925bc46cca76f481ddfd12afd) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:17.759894 1180 tablet_replica.cc:333] T c1fc01252d95407e954cfcbf7e715912 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:17.760161 1180 raft_consensus.cc:2243] T c1fc01252d95407e954cfcbf7e715912 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:17.760385 1180 raft_consensus.cc:2272] T c1fc01252d95407e954cfcbf7e715912 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:17.760296 1175 tablet_bootstrap.cc:492] T b428356fba0c4fb5a6033c5d39740cc5 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:17.761000 1180 ts_tablet_manager.cc:1916] T c1fc01252d95407e954cfcbf7e715912 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:17.769083 752 tablet_service.cc:1552] Processing DeleteTablet for tablet 2ff6af17a6854451a1d7b904aa4f1f55 with delete_type TABLET_DATA_DELETED (replaced by tablet 78f122342dd144b48de89902faa0f94f) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:17.769865 1175 tablet_bootstrap.cc:654] T b428356fba0c4fb5a6033c5d39740cc5 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.770835 523 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 251455168e33430aafea8e0de42ca1b8 (table default.Table_ce2d44550fb5403a99372bc5d3d79265 [id=4f19275f416a425c9b9d100b2a83870a]). Expected version 2 got 0
I20260202 02:52:17.772168 1174 tablet_bootstrap.cc:492] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:17.772385 1174 ts_tablet_manager.cc:1403] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.003s
I20260202 02:52:17.773324 1174 raft_consensus.cc:359] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.773557 1174 raft_consensus.cc:385] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.773651 1174 raft_consensus.cc:740] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.773918 1174 consensus_queue.cc:260] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.774267 523 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_2ebdd65f856c4fd1881a573bcd3c46d4" } modify_external_catalogs: true
I20260202 02:52:17.774524 1174 ts_tablet_manager.cc:1434] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260202 02:52:17.775000 1174 tablet_bootstrap.cc:492] T 0b411e8fab074aa88f8380e3f5694340 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:17.775311 525 master_service.cc:954] ReplaceTablet: received request to replace tablet f93e8d3c4faf48eaa98c3cbf88d585e2 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:17.777045 1183 ts_tablet_manager.cc:1929] T 4de41396cffd4d32b86c701b538d1e01 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:17.777665 1175 tablet_bootstrap.cc:492] T b428356fba0c4fb5a6033c5d39740cc5 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:17.778137 1175 ts_tablet_manager.cc:1403] T b428356fba0c4fb5a6033c5d39740cc5 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.000s
I20260202 02:52:17.779292 1175 raft_consensus.cc:359] T b428356fba0c4fb5a6033c5d39740cc5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.780135 1183 log.cc:1199] T 4de41396cffd4d32b86c701b538d1e01 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/4de41396cffd4d32b86c701b538d1e01
I20260202 02:52:17.781246 1174 tablet_bootstrap.cc:654] T 0b411e8fab074aa88f8380e3f5694340 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.781585 1453 raft_consensus.cc:493] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.781795 1453 raft_consensus.cc:515] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.783301 1453 leader_election.cc:290] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:17.784407 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0b411e8fab074aa88f8380e3f5694340" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:17.785255 859 leader_election.cc:343] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:17.785580 1175 raft_consensus.cc:385] T b428356fba0c4fb5a6033c5d39740cc5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.774638 753 tablet_service.cc:1552] Processing DeleteTablet for tablet 098c5cdfcd0f4c52b6ae4f986f07705a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:16 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:17.786072 1183 ts_tablet_manager.cc:1950] T 4de41396cffd4d32b86c701b538d1e01 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:17.786253 1175 raft_consensus.cc:740] T b428356fba0c4fb5a6033c5d39740cc5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.786684 1175 consensus_queue.cc:260] T b428356fba0c4fb5a6033c5d39740cc5 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.792132 1180 ts_tablet_manager.cc:1929] T c1fc01252d95407e954cfcbf7e715912 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:17.792357 1180 log.cc:1199] T c1fc01252d95407e954cfcbf7e715912 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/c1fc01252d95407e954cfcbf7e715912
I20260202 02:52:17.793071 1180 ts_tablet_manager.cc:1950] T c1fc01252d95407e954cfcbf7e715912 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
W20260202 02:52:17.744242 520 rpcz_store.cc:269] 0202 02:52:16.759808 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:16.765767 (+ 5959us) service_pool.cc:225] Handling call
0202 02:52:17.744087 (+978320us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:17.160959 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:17.161053 (+ 94us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:17.161058 (+ 5us) write_op.cc:435] Acquired schema lock
0202 02:52:17.161060 (+ 2us) tablet.cc:660] Decoding operations
0202 02:52:17.161148 (+ 88us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:17.161156 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:17.161160 (+ 4us) tablet.cc:683] Acquiring locks for 2 operations
0202 02:52:17.161253 (+ 93us) tablet.cc:699] Row locks acquired
0202 02:52:17.161255 (+ 2us) write_op.cc:260] PREPARE: finished
0202 02:52:17.161284 (+ 29us) write_op.cc:270] Start()
0202 02:52:17.161309 (+ 25us) write_op.cc:276] Timestamp: P: 1770000737161237 usec, L: 0
0202 02:52:17.161311 (+ 2us) op_driver.cc:348] REPLICATION: starting
0202 02:52:17.161455 (+ 144us) log.cc:844] Serialized 913 byte log entry
0202 02:52:17.298427 (+136972us) op_driver.cc:464] REPLICATION: finished
0202 02:52:17.300297 (+ 1870us) write_op.cc:301] APPLY: starting
0202 02:52:17.300307 (+ 10us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:17.300872 (+ 565us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:17.300895 (+ 23us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:17.300959 (+ 64us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:17.300989 (+ 30us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0202 02:52:17.300996 (+ 7us) write_op.cc:312] APPLY: finished
0202 02:52:17.301048 (+ 52us) log.cc:844] Serialized 29 byte log entry
0202 02:52:17.301106 (+ 58us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:17.301119 (+ 13us) write_op.cc:454] Released schema lock
0202 02:52:17.301125 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0202 02:52:17.301184 (+ 59us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:17.301188 (+ 4us) write_op.cc:454] Released schema lock
Related trace 'op':
0202 02:52:17.442600 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:17.442706 (+ 106us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:17.442710 (+ 4us) write_op.cc:435] Acquired schema lock
0202 02:52:17.442712 (+ 2us) tablet.cc:660] Decoding operations
0202 02:52:17.442868 (+ 156us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:17.442878 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:17.442882 (+ 4us) tablet.cc:683] Acquiring locks for 1 operations
0202 02:52:17.442942 (+ 60us) tablet.cc:699] Row locks acquired
0202 02:52:17.442944 (+ 2us) write_op.cc:260] PREPARE: finished
0202 02:52:17.442979 (+ 35us) write_op.cc:270] Start()
0202 02:52:17.443006 (+ 27us) write_op.cc:276] Timestamp: P: 1770000737442916 usec, L: 0
0202 02:52:17.443009 (+ 3us) op_driver.cc:348] REPLICATION: starting
0202 02:52:17.443181 (+ 172us) log.cc:844] Serialized 356 byte log entry
0202 02:52:17.487644 (+ 44463us) op_driver.cc:464] REPLICATION: finished
0202 02:52:17.490843 (+ 3199us) write_op.cc:301] APPLY: starting
0202 02:52:17.490851 (+ 8us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:17.490876 (+ 25us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:17.490901 (+ 25us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:17.490958 (+ 57us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:17.490980 (+ 22us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0202 02:52:17.490987 (+ 7us) write_op.cc:312] APPLY: finished
0202 02:52:17.491048 (+ 61us) log.cc:844] Serialized 23 byte log entry
0202 02:52:17.491094 (+ 46us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:17.491105 (+ 11us) write_op.cc:454] Released schema lock
0202 02:52:17.491111 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0202 02:52:17.508474 (+ 17363us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:17.508483 (+ 9us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1574,"reported_tablets":16,"child_traces":[["op",{"apply.queue_time_us":1841,"apply.run_cpu_time_us":878,"apply.run_wall_time_us":879,"num_ops":2,"prepare.queue_time_us":41321,"prepare.run_cpu_time_us":583,"prepare.run_wall_time_us":633,"raft.queue_time_us":33,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":5,"replication_time_us":137102}],["op",{"apply.queue_time_us":3171,"apply.run_cpu_time_us":352,"apply.run_wall_time_us":17601,"num_ops":1,"prepare.queue_time_us":115791,"prepare.run_cpu_time_us":662,"prepare.run_wall_time_us":783,"raft.queue_time_us":40,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":5,"replication_time_us":44621}]]}
I20260202 02:52:17.793687 1180 tablet_replica.cc:333] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:17.793977 1180 raft_consensus.cc:2243] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:17.794230 520 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_5308cabde3e94be1ab7a0b4e15ad67a9" } new_table_name: "default.Table_c09bacb95cba44a3ad80c21c0a77b6e0" modify_external_catalogs: true
I20260202 02:52:17.794485 1180 raft_consensus.cc:2272] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:17.794526 1175 ts_tablet_manager.cc:1434] T b428356fba0c4fb5a6033c5d39740cc5 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.016s user 0.002s sys 0.000s
I20260202 02:52:17.795149 1180 ts_tablet_manager.cc:1916] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:17.795854 1175 tablet_bootstrap.cc:492] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:17.796232 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet c1fc01252d95407e954cfcbf7e715912 (table default.Table_f23c311557a14f4a8c66eaf4af7a8a92 [id=10880639cfc743fcafecc538fa2ecd85]) successfully deleted
I20260202 02:52:17.776558 523 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_2ebdd65f856c4fd1881a573bcd3c46d4" } modify_external_catalogs: true
I20260202 02:52:17.797652 1453 raft_consensus.cc:493] T c63a670d5c5e489897bc6dbb142b844d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.797825 1453 raft_consensus.cc:515] T c63a670d5c5e489897bc6dbb142b844d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.798444 1453 leader_election.cc:290] T c63a670d5c5e489897bc6dbb142b844d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:17.799086 1174 tablet_bootstrap.cc:492] T 0b411e8fab074aa88f8380e3f5694340 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:17.799134 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c63a670d5c5e489897bc6dbb142b844d" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:17.799271 1174 ts_tablet_manager.cc:1403] T 0b411e8fab074aa88f8380e3f5694340 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.024s user 0.005s sys 0.001s
W20260202 02:52:17.799675 859 leader_election.cc:343] T c63a670d5c5e489897bc6dbb142b844d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:17.800202 1174 raft_consensus.cc:359] T 0b411e8fab074aa88f8380e3f5694340 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.800535 1174 raft_consensus.cc:385] T 0b411e8fab074aa88f8380e3f5694340 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.800616 1174 raft_consensus.cc:740] T 0b411e8fab074aa88f8380e3f5694340 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.800884 1174 consensus_queue.cc:260] T 0b411e8fab074aa88f8380e3f5694340 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.801544 1174 ts_tablet_manager.cc:1434] T 0b411e8fab074aa88f8380e3f5694340 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:17.802069 1174 tablet_bootstrap.cc:492] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:17.803109 1183 tablet_replica.cc:333] T 13309b20a96240b5aa9fbb691cbed981 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:17.803561 1183 raft_consensus.cc:2243] T 13309b20a96240b5aa9fbb691cbed981 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:17.803869 1174 tablet_bootstrap.cc:654] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.804118 1183 raft_consensus.cc:2272] T 13309b20a96240b5aa9fbb691cbed981 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:17.804415 1175 tablet_bootstrap.cc:654] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.805401 1040 tablet_service.cc:1552] Processing DeleteTablet for tablet fae7e3941c124cc2871390a0b6471f27 with delete_type TABLET_DATA_DELETED (replaced by tablet a5622ef925bc46cca76f481ddfd12afd) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:17.806403 1040 tablet_service.cc:1552] Processing DeleteTablet for tablet add934bf70ae4de8a40066f06fd94795 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:17.807821 1183 ts_tablet_manager.cc:1916] T 13309b20a96240b5aa9fbb691cbed981 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:17.810446 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 672ea3f731ba42ff8b1ff47db039ed22 (table default.Table_85da33c295954de58e2c6290005ac865 [id=ed3c3f07e9ef4a568d36ebd62e0bddc5]) successfully deleted
I20260202 02:52:17.811313 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 4de41396cffd4d32b86c701b538d1e01 (table default.Table_d142ff2cb10e491997bf0e302d6a9fa5 [id=a0b8442c02974d6596db002c9292be0e]) successfully deleted
I20260202 02:52:17.811535 1174 tablet_bootstrap.cc:492] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
W20260202 02:52:17.811609 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 9884c90cf1d245b2a7eaaf00d71b1c3e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9884c90cf1d245b2a7eaaf00d71b1c3e already in progress: creating tablet
I20260202 02:52:17.811717 1174 ts_tablet_manager.cc:1403] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
W20260202 02:52:17.811823 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet c064ea0dade840b4840a5715c2d83f69 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c064ea0dade840b4840a5715c2d83f69 already in progress: creating tablet
W20260202 02:52:17.812068 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 526f2c7b95184a0987e7aa94efc8475a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 526f2c7b95184a0987e7aa94efc8475a already in progress: creating tablet
I20260202 02:52:17.812455 1175 tablet_bootstrap.cc:492] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:17.812635 1174 raft_consensus.cc:359] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.812839 1174 raft_consensus.cc:385] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.812944 1174 raft_consensus.cc:740] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.813122 1175 ts_tablet_manager.cc:1403] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.000s
I20260202 02:52:17.813266 1174 consensus_queue.cc:260] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.813860 1174 ts_tablet_manager.cc:1434] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:17.814213 1174 tablet_bootstrap.cc:492] T b76d1837813c46e8a0925b49939d8884 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:17.815317 714 leader_election.cc:304] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:17.815690 1322 raft_consensus.cc:2804] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:17.815826 1322 raft_consensus.cc:697] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:17.816090 1322 consensus_queue.cc:237] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:17.816234 1174 tablet_bootstrap.cc:654] T b76d1837813c46e8a0925b49939d8884 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.813275 1180 ts_tablet_manager.cc:1929] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:17.817755 1180 log.cc:1199] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/fae7e3941c124cc2871390a0b6471f27
I20260202 02:52:17.818205 1180 ts_tablet_manager.cc:1950] T fae7e3941c124cc2871390a0b6471f27 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:17.819532 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet fae7e3941c124cc2871390a0b6471f27 (table default.Table_3d26f7a34d0c48379e3cebda66e1ef13 [id=45e6d83589fa493195b06f633e295b00]) successfully deleted
W20260202 02:52:17.819855 513 catalog_manager.cc:4952] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 2ff6af17a6854451a1d7b904aa4f1f55 because the tablet was not found. No further retry: Not found: Tablet not found: 2ff6af17a6854451a1d7b904aa4f1f55
I20260202 02:52:17.820020 1174 tablet_bootstrap.cc:492] T b76d1837813c46e8a0925b49939d8884 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
W20260202 02:52:17.820071 513 catalog_manager.cc:4952] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 098c5cdfcd0f4c52b6ae4f986f07705a because the tablet was not found. No further retry: Not found: Tablet not found: 098c5cdfcd0f4c52b6ae4f986f07705a
I20260202 02:52:17.820209 1174 ts_tablet_manager.cc:1403] T b76d1837813c46e8a0925b49939d8884 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260202 02:52:17.821578 1174 raft_consensus.cc:359] T b76d1837813c46e8a0925b49939d8884 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.821818 1174 raft_consensus.cc:385] T b76d1837813c46e8a0925b49939d8884 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.821900 1174 raft_consensus.cc:740] T b76d1837813c46e8a0925b49939d8884 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.822139 1174 consensus_queue.cc:260] T b76d1837813c46e8a0925b49939d8884 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.822748 1174 ts_tablet_manager.cc:1434] T b76d1837813c46e8a0925b49939d8884 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260202 02:52:17.823102 1174 tablet_bootstrap.cc:492] T 583e547d0a1943f09f4efe82070fc777 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:17.823240 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0b411e8fab074aa88f8380e3f5694340" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:17.823546 779 raft_consensus.cc:2468] T 0b411e8fab074aa88f8380e3f5694340 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
I20260202 02:52:17.826288 1175 raft_consensus.cc:359] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.826560 1175 raft_consensus.cc:385] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.826634 1175 raft_consensus.cc:740] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.826803 898 tablet_service.cc:1552] Processing DeleteTablet for tablet fae7e3941c124cc2871390a0b6471f27 with delete_type TABLET_DATA_DELETED (replaced by tablet a5622ef925bc46cca76f481ddfd12afd) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:17.826901 1175 consensus_queue.cc:260] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.827087 903 tablet_service.cc:1552] Processing DeleteTablet for tablet ea040c03420c4b1b98897f39160b4130 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:17.812275 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet c184d44d7f434b39a997a7387e5d3de0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c184d44d7f434b39a997a7387e5d3de0 already in progress: creating tablet
I20260202 02:52:17.827296 900 tablet_service.cc:1552] Processing DeleteTablet for tablet add934bf70ae4de8a40066f06fd94795 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:14 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:17.827479 901 tablet_service.cc:1552] Processing DeleteTablet for tablet 23c38ef4e3c840808ec36f859cf6f6de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:17.827713 896 tablet_service.cc:1552] Processing DeleteTablet for tablet bf5ca99708044482970cb854d2eced18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:17.827734 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet eb0ac55fbb534b29a86d6b3e8aa72ebc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet eb0ac55fbb534b29a86d6b3e8aa72ebc already in progress: creating tablet
I20260202 02:52:17.827935 1182 tablet_replica.cc:333] T fae7e3941c124cc2871390a0b6471f27 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:17.828168 1182 raft_consensus.cc:2243] T fae7e3941c124cc2871390a0b6471f27 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:17.829835 1173 tablet_bootstrap.cc:492] T 575016c0e9f84e99b962c5f14ffa96cc P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:17.830004 1173 ts_tablet_manager.cc:1403] T 575016c0e9f84e99b962c5f14ffa96cc P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.178s user 0.016s sys 0.014s
I20260202 02:52:17.830302 1174 tablet_bootstrap.cc:654] T 583e547d0a1943f09f4efe82070fc777 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.830811 1173 raft_consensus.cc:359] T 575016c0e9f84e99b962c5f14ffa96cc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.831017 1173 raft_consensus.cc:385] T 575016c0e9f84e99b962c5f14ffa96cc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.831118 1173 raft_consensus.cc:740] T 575016c0e9f84e99b962c5f14ffa96cc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.831393 1173 consensus_queue.cc:260] T 575016c0e9f84e99b962c5f14ffa96cc P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.833444 1173 ts_tablet_manager.cc:1434] T 575016c0e9f84e99b962c5f14ffa96cc P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260202 02:52:17.834434 1173 tablet_bootstrap.cc:492] T f24c0b99574d437a99b273204f105460 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:17.836097 1453 raft_consensus.cc:493] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.836287 1453 raft_consensus.cc:515] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.837069 1453 leader_election.cc:290] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:17.837467 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c63a670d5c5e489897bc6dbb142b844d" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:17.838416 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0d3633f5f9f54115bc78b9c1b78aba67" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:17.839088 859 leader_election.cc:343] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:17.839601 1182 raft_consensus.cc:2272] T fae7e3941c124cc2871390a0b6471f27 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:17.840260 1182 ts_tablet_manager.cc:1916] T fae7e3941c124cc2871390a0b6471f27 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:17.842371 899 tablet_service.cc:1552] Processing DeleteTablet for tablet 0d297b6243c348ee82c146a8339cf6c4 with delete_type TABLET_DATA_DELETED (replaced by tablet d64f42d187bc4c9f9381c931449ec0cc) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:17.842665 896 tablet_service.cc:1552] Processing DeleteTablet for tablet 098c5cdfcd0f4c52b6ae4f986f07705a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:16 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:17.847955 1182 ts_tablet_manager.cc:1929] T fae7e3941c124cc2871390a0b6471f27 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:17.848228 1182 log.cc:1199] T fae7e3941c124cc2871390a0b6471f27 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/fae7e3941c124cc2871390a0b6471f27
I20260202 02:52:17.848734 1182 ts_tablet_manager.cc:1950] T fae7e3941c124cc2871390a0b6471f27 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:17.848788 896 tablet_service.cc:1552] Processing DeleteTablet for tablet 575016c0e9f84e99b962c5f14ffa96cc with delete_type TABLET_DATA_DELETED (replaced by tablet ff6f5710de4b4c48948aacdf5be29e8b) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:17.849771 1182 tablet_replica.cc:333] T ea040c03420c4b1b98897f39160b4130 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:17.849885 1359 tablet.cc:1722] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:17.850251 1175 ts_tablet_manager.cc:1434] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.037s user 0.002s sys 0.000s
I20260202 02:52:17.850436 1182 raft_consensus.cc:2243] T ea040c03420c4b1b98897f39160b4130 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:17.850598 1182 raft_consensus.cc:2272] T ea040c03420c4b1b98897f39160b4130 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:17.850795 1175 tablet_bootstrap.cc:492] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:17.852968 1175 tablet_bootstrap.cc:654] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.854677 1174 tablet_bootstrap.cc:492] T 583e547d0a1943f09f4efe82070fc777 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:17.854861 1174 ts_tablet_manager.cc:1403] T 583e547d0a1943f09f4efe82070fc777 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.032s user 0.009s sys 0.004s
I20260202 02:52:17.855111 1182 ts_tablet_manager.cc:1916] T ea040c03420c4b1b98897f39160b4130 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:17.855875 1174 raft_consensus.cc:359] T 583e547d0a1943f09f4efe82070fc777 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.856112 1174 raft_consensus.cc:385] T 583e547d0a1943f09f4efe82070fc777 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.856216 1174 raft_consensus.cc:740] T 583e547d0a1943f09f4efe82070fc777 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.856492 1174 consensus_queue.cc:260] T 583e547d0a1943f09f4efe82070fc777 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.857141 1174 ts_tablet_manager.cc:1434] T 583e547d0a1943f09f4efe82070fc777 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:17.857482 1174 tablet_bootstrap.cc:492] T d2fa4c3d6ec44047abf5986e502c1590 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:17.859710 1174 tablet_bootstrap.cc:654] T d2fa4c3d6ec44047abf5986e502c1590 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.860272 1173 tablet_bootstrap.cc:654] T f24c0b99574d437a99b273204f105460 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.861670 1183 ts_tablet_manager.cc:1929] T 13309b20a96240b5aa9fbb691cbed981 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:17.862008 1183 log.cc:1199] T 13309b20a96240b5aa9fbb691cbed981 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/13309b20a96240b5aa9fbb691cbed981
I20260202 02:52:17.862536 1183 ts_tablet_manager.cc:1950] T 13309b20a96240b5aa9fbb691cbed981 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:17.865093 1183 tablet_replica.cc:333] T fae7e3941c124cc2871390a0b6471f27 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:17.866001 1175 tablet_bootstrap.cc:492] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:17.866202 1175 ts_tablet_manager.cc:1403] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.000s
I20260202 02:52:17.866614 1183 raft_consensus.cc:2243] T fae7e3941c124cc2871390a0b6471f27 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:17.867116 1175 raft_consensus.cc:359] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.867340 1175 raft_consensus.cc:385] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.867318 1173 tablet_bootstrap.cc:492] T f24c0b99574d437a99b273204f105460 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:17.867444 1175 raft_consensus.cc:740] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.867622 1173 ts_tablet_manager.cc:1403] T f24c0b99574d437a99b273204f105460 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.033s user 0.003s sys 0.001s
I20260202 02:52:17.867784 1175 consensus_queue.cc:260] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.868475 1175 ts_tablet_manager.cc:1434] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260202 02:52:17.868919 1175 tablet_bootstrap.cc:492] T 70306ed9dec54286885d965f2c012da4 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:17.871023 1173 raft_consensus.cc:359] T f24c0b99574d437a99b273204f105460 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.871398 1173 raft_consensus.cc:385] T f24c0b99574d437a99b273204f105460 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.871516 1173 raft_consensus.cc:740] T f24c0b99574d437a99b273204f105460 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.871840 1175 tablet_bootstrap.cc:654] T 70306ed9dec54286885d965f2c012da4 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.873523 1174 tablet_bootstrap.cc:492] T d2fa4c3d6ec44047abf5986e502c1590 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:17.873742 1174 ts_tablet_manager.cc:1403] T d2fa4c3d6ec44047abf5986e502c1590 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.003s
I20260202 02:52:17.874439 1321 raft_consensus.cc:493] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.874886 1321 raft_consensus.cc:515] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.874799 1174 raft_consensus.cc:359] T d2fa4c3d6ec44047abf5986e502c1590 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.875379 1174 raft_consensus.cc:385] T d2fa4c3d6ec44047abf5986e502c1590 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.875476 1174 raft_consensus.cc:740] T d2fa4c3d6ec44047abf5986e502c1590 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.876159 1174 consensus_queue.cc:260] T d2fa4c3d6ec44047abf5986e502c1590 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.876427 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "63a9029bc39c4fbd8ac4bf4bc37d6b4e" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:17.876569 1173 consensus_queue.cc:260] T f24c0b99574d437a99b273204f105460 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:17.877326 715 leader_election.cc:343] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:17.879340 1175 tablet_bootstrap.cc:492] T 70306ed9dec54286885d965f2c012da4 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:17.879555 1175 ts_tablet_manager.cc:1403] T 70306ed9dec54286885d965f2c012da4 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260202 02:52:17.879674 1173 ts_tablet_manager.cc:1434] T f24c0b99574d437a99b273204f105460 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.012s user 0.002s sys 0.000s
I20260202 02:52:17.880481 1175 raft_consensus.cc:359] T 70306ed9dec54286885d965f2c012da4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.880736 1175 raft_consensus.cc:385] T 70306ed9dec54286885d965f2c012da4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.880827 1175 raft_consensus.cc:740] T 70306ed9dec54286885d965f2c012da4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.881143 1175 consensus_queue.cc:260] T 70306ed9dec54286885d965f2c012da4 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.881748 1321 leader_election.cc:290] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:17.882418 1324 leader_election.cc:290] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:17.883073 857 leader_election.cc:304] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:17.883596 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0d3633f5f9f54115bc78b9c1b78aba67" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:17.884099 1173 tablet_bootstrap.cc:492] T ffd8aa2bc6e344b78dce518411997fb5 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:17.886132 1183 raft_consensus.cc:2272] T fae7e3941c124cc2871390a0b6471f27 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:17.888453 1173 tablet_bootstrap.cc:654] T ffd8aa2bc6e344b78dce518411997fb5 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:17.894429 515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.190:33981, user_credentials={real_user=slave}} blocked reactor thread for 49202.4us
I20260202 02:52:17.895279 758 tablet_service.cc:1505] Processing CreateTablet for tablet d1e0e74f32a24729b250c5751677cfb0 (DEFAULT_TABLE table=default.Table_ba9c16aaa48849db9fece59168c16886 [id=f1f6c97fc88340da8bab0347ad6430ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.895677 752 tablet_service.cc:1505] Processing CreateTablet for tablet d21fca9a5b234c26879717f51c8ea0f0 (DEFAULT_TABLE table=default.Table_6cd4b2d180cf4fdebad15ee5ea5ea7de [id=a77fa0c30d27405294adaef676bc76f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.895624 1040 tablet_service.cc:1505] Processing CreateTablet for tablet 4e79cd48fefb4ce29ef48beacfc4d26f (DEFAULT_TABLE table=default.Table_6de529f984b247d79857c52705d4974a [id=2a625eeef55b43f19cd4bbeb61e85bca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.896016 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1e0e74f32a24729b250c5751677cfb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.896168 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e79cd48fefb4ce29ef48beacfc4d26f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.896481 754 tablet_service.cc:1505] Processing CreateTablet for tablet 1f54c1dfe0a942b3bafd85522953a032 (DEFAULT_TABLE table=default.Table_95c7a47b60374ca2959b0feb63df471f [id=891d9355855640a28939b84b2dd30537]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.897126 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f54c1dfe0a942b3bafd85522953a032. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.900767 750 tablet_service.cc:1505] Processing CreateTablet for tablet 94bada52db6a47179415d302b541d01f (DEFAULT_TABLE table=default.Table_1fd940660d9b4ef292d7aaa5e0e336cb [id=32fe1d8734de41cfae07a4662814987e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.901539 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94bada52db6a47179415d302b541d01f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.886045 1175 ts_tablet_manager.cc:1434] T 70306ed9dec54286885d965f2c012da4 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260202 02:52:17.887943 1182 ts_tablet_manager.cc:1929] T ea040c03420c4b1b98897f39160b4130 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:17.904790 1182 log.cc:1199] T ea040c03420c4b1b98897f39160b4130 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/ea040c03420c4b1b98897f39160b4130
I20260202 02:52:17.904758 749 tablet_service.cc:1505] Processing CreateTablet for tablet 929eb48881b141718873d0e359e768a8 (DEFAULT_TABLE table=default.Table_b0e344dc1a824cd9b41aa8991b7911dd [id=8e640d1261014706a067f522bda7ea61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.905243 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 929eb48881b141718873d0e359e768a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.905277 1175 tablet_bootstrap.cc:492] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:17.905375 1182 ts_tablet_manager.cc:1950] T ea040c03420c4b1b98897f39160b4130 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:17.907153 1182 tablet_replica.cc:333] T 23c38ef4e3c840808ec36f859cf6f6de P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:17.908701 755 tablet_service.cc:1505] Processing CreateTablet for tablet ffd80badce1048f3ac95366cad05ae53 (DEFAULT_TABLE table=default.Table_38c5bca682e343c1be1ebdc253c9e848 [id=6f44bec7f8ea4d2abed8c0fd603c6569]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.909008 1040 tablet_service.cc:1505] Processing CreateTablet for tablet d21fca9a5b234c26879717f51c8ea0f0 (DEFAULT_TABLE table=default.Table_6cd4b2d180cf4fdebad15ee5ea5ea7de [id=a77fa0c30d27405294adaef676bc76f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.909152 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffd80badce1048f3ac95366cad05ae53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.909459 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d21fca9a5b234c26879717f51c8ea0f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.907393 1182 raft_consensus.cc:2243] T 23c38ef4e3c840808ec36f859cf6f6de P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:17.907666 1175 tablet_bootstrap.cc:654] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.913022 1338 raft_consensus.cc:2804] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:17.913195 1338 raft_consensus.cc:493] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:17.913295 1338 raft_consensus.cc:3060] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:17.913399 753 tablet_service.cc:1505] Processing CreateTablet for tablet 4e79cd48fefb4ce29ef48beacfc4d26f (DEFAULT_TABLE table=default.Table_6de529f984b247d79857c52705d4974a [id=2a625eeef55b43f19cd4bbeb61e85bca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.914134 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e79cd48fefb4ce29ef48beacfc4d26f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.916610 1182 raft_consensus.cc:2272] T 23c38ef4e3c840808ec36f859cf6f6de P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:17.917491 1182 ts_tablet_manager.cc:1916] T 23c38ef4e3c840808ec36f859cf6f6de P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:17.918813 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d21fca9a5b234c26879717f51c8ea0f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.921162 1338 raft_consensus.cc:515] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.922327 1338 leader_election.cc:290] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:17.923122 1178 raft_consensus.cc:493] T add934bf70ae4de8a40066f06fd94795 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.923305 1178 raft_consensus.cc:515] T add934bf70ae4de8a40066f06fd94795 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.924028 1178 leader_election.cc:290] T add934bf70ae4de8a40066f06fd94795 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:17.925055 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0b411e8fab074aa88f8380e3f5694340" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
W20260202 02:52:17.926092 859 leader_election.cc:343] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:17.928628 857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 45573us
I20260202 02:52:17.930068 1041 tablet_service.cc:1505] Processing CreateTablet for tablet 1f54c1dfe0a942b3bafd85522953a032 (DEFAULT_TABLE table=default.Table_95c7a47b60374ca2959b0feb63df471f [id=891d9355855640a28939b84b2dd30537]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.930584 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f54c1dfe0a942b3bafd85522953a032. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:17.930567 857 leader_election.cc:343] T c63a670d5c5e489897bc6dbb142b844d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:17.930733 857 leader_election.cc:304] T c63a670d5c5e489897bc6dbb142b844d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:17.931859 1043 tablet_service.cc:1505] Processing CreateTablet for tablet 94bada52db6a47179415d302b541d01f (DEFAULT_TABLE table=default.Table_1fd940660d9b4ef292d7aaa5e0e336cb [id=32fe1d8734de41cfae07a4662814987e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.932379 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94bada52db6a47179415d302b541d01f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.934211 1038 tablet_service.cc:1505] Processing CreateTablet for tablet d1e0e74f32a24729b250c5751677cfb0 (DEFAULT_TABLE table=default.Table_ba9c16aaa48849db9fece59168c16886 [id=f1f6c97fc88340da8bab0347ad6430ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.934993 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1e0e74f32a24729b250c5751677cfb0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:17.937465 998 leader_election.cc:343] T add934bf70ae4de8a40066f06fd94795 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Not found: Tablet not found: add934bf70ae4de8a40066f06fd94795
I20260202 02:52:17.938315 1183 ts_tablet_manager.cc:1916] T fae7e3941c124cc2871390a0b6471f27 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:17.943749 1043 tablet_service.cc:1505] Processing CreateTablet for tablet 929eb48881b141718873d0e359e768a8 (DEFAULT_TABLE table=default.Table_b0e344dc1a824cd9b41aa8991b7911dd [id=8e640d1261014706a067f522bda7ea61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.944432 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 929eb48881b141718873d0e359e768a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.952085 1338 raft_consensus.cc:2749] T c63a670d5c5e489897bc6dbb142b844d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:17.952652 857 leader_election.cc:343] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:17.952818 857 leader_election.cc:304] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
W20260202 02:52:17.953385 859 consensus_peers.cc:597] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5 -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:17.953708 1038 tablet_service.cc:1505] Processing CreateTablet for tablet ffd80badce1048f3ac95366cad05ae53 (DEFAULT_TABLE table=default.Table_38c5bca682e343c1be1ebdc253c9e848 [id=6f44bec7f8ea4d2abed8c0fd603c6569]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.954245 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffd80badce1048f3ac95366cad05ae53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.954614 1338 raft_consensus.cc:2749] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:17.877333 1174 ts_tablet_manager.cc:1434] T d2fa4c3d6ec44047abf5986e502c1590 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260202 02:52:17.958218 1174 tablet_bootstrap.cc:492] T 723921dfe54c4e66af5cab76a25fee0a P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:17.960813 1338 raft_consensus.cc:493] T c064ea0dade840b4840a5715c2d83f69 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.961020 1338 raft_consensus.cc:515] T c064ea0dade840b4840a5715c2d83f69 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.961670 1183 ts_tablet_manager.cc:1929] T fae7e3941c124cc2871390a0b6471f27 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:17.962070 1338 leader_election.cc:290] T c064ea0dade840b4840a5715c2d83f69 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:17.965332 1175 tablet_bootstrap.cc:492] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:17.965631 1175 ts_tablet_manager.cc:1403] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.060s user 0.003s sys 0.006s
I20260202 02:52:17.966642 1175 raft_consensus.cc:359] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.966985 1175 raft_consensus.cc:385] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:17.967142 1175 raft_consensus.cc:740] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:17.967448 1175 consensus_queue.cc:260] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.968431 1175 ts_tablet_manager.cc:1434] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260202 02:52:17.968871 1175 tablet_bootstrap.cc:492] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:17.962347 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c064ea0dade840b4840a5715c2d83f69" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:17.970952 1175 tablet_bootstrap.cc:654] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.971176 1183 log.cc:1199] T fae7e3941c124cc2871390a0b6471f27 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/fae7e3941c124cc2871390a0b6471f27
W20260202 02:52:17.971261 859 leader_election.cc:343] T c064ea0dade840b4840a5715c2d83f69 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:17.971333 1338 raft_consensus.cc:493] T 70306ed9dec54286885d965f2c012da4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.971570 1338 raft_consensus.cc:515] T 70306ed9dec54286885d965f2c012da4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.971738 1183 ts_tablet_manager.cc:1950] T fae7e3941c124cc2871390a0b6471f27 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:17.972903 1183 tablet_replica.cc:333] T add934bf70ae4de8a40066f06fd94795 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:17.973052 1338 leader_election.cc:290] T 70306ed9dec54286885d965f2c012da4 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:17.973162 1183 raft_consensus.cc:2243] T add934bf70ae4de8a40066f06fd94795 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:17.968750 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet fae7e3941c124cc2871390a0b6471f27 (table default.Table_3d26f7a34d0c48379e3cebda66e1ef13 [id=45e6d83589fa493195b06f633e295b00]) successfully deleted
I20260202 02:52:17.973433 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 13309b20a96240b5aa9fbb691cbed981 (table default.Table_5fa327d665484811934aa3b11188a76d [id=f7c5b24c065443d89a2b1f9c34409c9d]) successfully deleted
I20260202 02:52:17.973352 1183 raft_consensus.cc:2272] T add934bf70ae4de8a40066f06fd94795 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:17.974130 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "70306ed9dec54286885d965f2c012da4" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:17.974943 859 leader_election.cc:343] T 70306ed9dec54286885d965f2c012da4 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:17.974987 1183 ts_tablet_manager.cc:1916] T add934bf70ae4de8a40066f06fd94795 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:17.976723 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet ea040c03420c4b1b98897f39160b4130 (table default.Table_9bac792e43464105846e5482edbccacc [id=0a3c23955dde47d9982c4659d9a73e30]) successfully deleted
W20260202 02:52:17.976748 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet fe7b6004e5094c0dbe5847144a2489b9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fe7b6004e5094c0dbe5847144a2489b9 already in progress: creating tablet
W20260202 02:52:17.977073 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet add934bf70ae4de8a40066f06fd94795 because the tablet was not found. No further retry: Not found: Tablet not found: add934bf70ae4de8a40066f06fd94795
I20260202 02:52:17.977835 1182 ts_tablet_manager.cc:1929] T 23c38ef4e3c840808ec36f859cf6f6de P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260202 02:52:17.978467 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 560465c063d94ffaab33f9f79a58a357 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 560465c063d94ffaab33f9f79a58a357 already in progress: creating tablet
I20260202 02:52:17.978320 896 tablet_service.cc:1505] Processing CreateTablet for tablet 4e79cd48fefb4ce29ef48beacfc4d26f (DEFAULT_TABLE table=default.Table_6de529f984b247d79857c52705d4974a [id=2a625eeef55b43f19cd4bbeb61e85bca]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:17.978742 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 80d443bdb66d4cc68ae283e08c410fe6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 80d443bdb66d4cc68ae283e08c410fe6 already in progress: creating tablet
I20260202 02:52:17.979048 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e79cd48fefb4ce29ef48beacfc4d26f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.979281 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet fae7e3941c124cc2871390a0b6471f27 (table default.Table_3d26f7a34d0c48379e3cebda66e1ef13 [id=45e6d83589fa493195b06f633e295b00]) successfully deleted
W20260202 02:52:17.981603 528 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c63a670d5c5e489897bc6dbb142b844d consensus state has no leader
I20260202 02:52:17.983224 1178 raft_consensus.cc:493] T 575016c0e9f84e99b962c5f14ffa96cc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.983350 1041 tablet_service.cc:1505] Processing CreateTablet for tablet 622a113af04a41c6852c29fc8a8acd78 (DEFAULT_TABLE table=default.Table_5fa327d665484811934aa3b11188a76d [id=f7c5b24c065443d89a2b1f9c34409c9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.983417 1178 raft_consensus.cc:515] T 575016c0e9f84e99b962c5f14ffa96cc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.983780 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 622a113af04a41c6852c29fc8a8acd78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.984109 1178 leader_election.cc:290] T 575016c0e9f84e99b962c5f14ffa96cc P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:17.985442 998 leader_election.cc:343] T 575016c0e9f84e99b962c5f14ffa96cc P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Not found: Tablet not found: 575016c0e9f84e99b962c5f14ffa96cc
I20260202 02:52:17.985476 1037 tablet_service.cc:1505] Processing CreateTablet for tablet d64f42d187bc4c9f9381c931449ec0cc (DEFAULT_TABLE table=default.Table_0c83932cab13463f9cca28129f18b576 [id=7c76848b98614fbf8b2950e5d55cc609]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.985992 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d64f42d187bc4c9f9381c931449ec0cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.988703 1034 tablet_service.cc:1505] Processing CreateTablet for tablet d67e91a022f1401ea724dff898c3b81b (DEFAULT_TABLE table=default.Table_c245b83c1987454fabbe8646d14252d6 [id=08498a654b6a4a9ca3f968d70daad9b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.989202 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d67e91a022f1401ea724dff898c3b81b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.960318 1174 tablet_bootstrap.cc:654] T 723921dfe54c4e66af5cab76a25fee0a P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:17.964605 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0b411e8fab074aa88f8380e3f5694340" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:17.994690 779 raft_consensus.cc:3060] T 0b411e8fab074aa88f8380e3f5694340 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:17.997502 1338 raft_consensus.cc:493] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.997787 1338 raft_consensus.cc:515] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.969816 750 tablet_service.cc:1505] Processing CreateTablet for tablet 622a113af04a41c6852c29fc8a8acd78 (DEFAULT_TABLE table=default.Table_5fa327d665484811934aa3b11188a76d [id=f7c5b24c065443d89a2b1f9c34409c9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.998135 752 tablet_service.cc:1505] Processing CreateTablet for tablet 25765fb687294e9db1289f76530dd877 (DEFAULT_TABLE table=default.Table_024075b20add4ddcbc4b27601a94eed4 [id=ea0c52a72da8422ba14345fbfc242d27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.998312 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 622a113af04a41c6852c29fc8a8acd78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:17.969930 1324 raft_consensus.cc:493] T 560465c063d94ffaab33f9f79a58a357 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:17.998575 1324 raft_consensus.cc:515] T 560465c063d94ffaab33f9f79a58a357 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:17.998977 1338 leader_election.cc:290] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:17.999222 1324 leader_election.cc:290] T 560465c063d94ffaab33f9f79a58a357 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:17.999293 1174 tablet_bootstrap.cc:492] T 723921dfe54c4e66af5cab76a25fee0a P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:17.977891 753 tablet_service.cc:1505] Processing CreateTablet for tablet d64f42d187bc4c9f9381c931449ec0cc (DEFAULT_TABLE table=default.Table_0c83932cab13463f9cca28129f18b576 [id=7c76848b98614fbf8b2950e5d55cc609]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:17.999464 1174 ts_tablet_manager.cc:1403] T 723921dfe54c4e66af5cab76a25fee0a P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.041s user 0.003s sys 0.003s
I20260202 02:52:17.999791 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d64f42d187bc4c9f9381c931449ec0cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.000511 1174 raft_consensus.cc:359] T 723921dfe54c4e66af5cab76a25fee0a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.000742 1174 raft_consensus.cc:385] T 723921dfe54c4e66af5cab76a25fee0a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:18.000844 1174 raft_consensus.cc:740] T 723921dfe54c4e66af5cab76a25fee0a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:18.000891 1338 raft_consensus.cc:493] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:18.001039 1338 raft_consensus.cc:515] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.001112 1174 consensus_queue.cc:260] T 723921dfe54c4e66af5cab76a25fee0a P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.001559 1277 raft_consensus.cc:493] T cb0dcacdf0f1419da0b25f20cac1d62d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:18.001641 1338 leader_election.cc:290] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:18.001725 1277 raft_consensus.cc:515] T cb0dcacdf0f1419da0b25f20cac1d62d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.002353 1277 leader_election.cc:290] T cb0dcacdf0f1419da0b25f20cac1d62d P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:18.003093 1174 ts_tablet_manager.cc:1434] T 723921dfe54c4e66af5cab76a25fee0a P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260202 02:52:17.978128 754 tablet_service.cc:1505] Processing CreateTablet for tablet d67e91a022f1401ea724dff898c3b81b (DEFAULT_TABLE table=default.Table_c245b83c1987454fabbe8646d14252d6 [id=08498a654b6a4a9ca3f968d70daad9b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.003417 1174 tablet_bootstrap.cc:492] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:18.003574 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d67e91a022f1401ea724dff898c3b81b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.005704 1174 tablet_bootstrap.cc:654] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:18.008104 1182 log.cc:1199] T 23c38ef4e3c840808ec36f859cf6f6de P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/23c38ef4e3c840808ec36f859cf6f6de
I20260202 02:52:17.978348 749 tablet_service.cc:1505] Processing CreateTablet for tablet 3711910e050c4515bed4294d1271d5a9 (DEFAULT_TABLE table=default.Table_9451b9f5148f4d9aab65983dea967e32 [id=9ef504467a544f299b48499e49adf27f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.008502 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3711910e050c4515bed4294d1271d5a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.008518 1182 ts_tablet_manager.cc:1950] T 23c38ef4e3c840808ec36f859cf6f6de P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:18.009492 1182 tablet_replica.cc:333] T bf5ca99708044482970cb854d2eced18 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:18.009696 1182 raft_consensus.cc:2243] T bf5ca99708044482970cb854d2eced18 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:18.009882 1182 raft_consensus.cc:2272] T bf5ca99708044482970cb854d2eced18 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:18.008265 899 tablet_service.cc:1505] Processing CreateTablet for tablet d21fca9a5b234c26879717f51c8ea0f0 (DEFAULT_TABLE table=default.Table_6cd4b2d180cf4fdebad15ee5ea5ea7de [id=a77fa0c30d27405294adaef676bc76f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.010577 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d21fca9a5b234c26879717f51c8ea0f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.010887 896 tablet_service.cc:1505] Processing CreateTablet for tablet 1f54c1dfe0a942b3bafd85522953a032 (DEFAULT_TABLE table=default.Table_95c7a47b60374ca2959b0feb63df471f [id=891d9355855640a28939b84b2dd30537]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.011242 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f54c1dfe0a942b3bafd85522953a032. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.012894 1174 tablet_bootstrap.cc:492] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:18.013077 1174 ts_tablet_manager.cc:1403] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260202 02:52:18.013841 1174 raft_consensus.cc:359] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.014151 1174 raft_consensus.cc:385] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:18.014241 1174 raft_consensus.cc:740] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:18.014511 1174 consensus_queue.cc:260] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.015103 1174 ts_tablet_manager.cc:1434] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:18.015415 1174 tablet_bootstrap.cc:492] T fe7b6004e5094c0dbe5847144a2489b9 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:17.986624 755 tablet_service.cc:1505] Processing CreateTablet for tablet f6746ccbf37e477196a9706396d1e3fd (DEFAULT_TABLE table=default.Table_69b88340567c4887947ed8458ffd1cba [id=0c8385b71ceb44199472e1ce640b54e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.016113 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6746ccbf37e477196a9706396d1e3fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.017442 1174 tablet_bootstrap.cc:654] T fe7b6004e5094c0dbe5847144a2489b9 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:18.024395 903 tablet_service.cc:1505] Processing CreateTablet for tablet ffd80badce1048f3ac95366cad05ae53 (DEFAULT_TABLE table=default.Table_38c5bca682e343c1be1ebdc253c9e848 [id=6f44bec7f8ea4d2abed8c0fd603c6569]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.025028 898 tablet_service.cc:1505] Processing CreateTablet for tablet 622a113af04a41c6852c29fc8a8acd78 (DEFAULT_TABLE table=default.Table_5fa327d665484811934aa3b11188a76d [id=f7c5b24c065443d89a2b1f9c34409c9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.025450 901 tablet_service.cc:1505] Processing CreateTablet for tablet 94bada52db6a47179415d302b541d01f (DEFAULT_TABLE table=default.Table_1fd940660d9b4ef292d7aaa5e0e336cb [id=32fe1d8734de41cfae07a4662814987e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.025481 900 tablet_service.cc:1505] Processing CreateTablet for tablet 929eb48881b141718873d0e359e768a8 (DEFAULT_TABLE table=default.Table_b0e344dc1a824cd9b41aa8991b7911dd [id=8e640d1261014706a067f522bda7ea61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.026103 897 tablet_service.cc:1505] Processing CreateTablet for tablet d64f42d187bc4c9f9381c931449ec0cc (DEFAULT_TABLE table=default.Table_0c83932cab13463f9cca28129f18b576 [id=7c76848b98614fbf8b2950e5d55cc609]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.026727 903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffd80badce1048f3ac95366cad05ae53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.029986 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 622a113af04a41c6852c29fc8a8acd78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.030453 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d64f42d187bc4c9f9381c931449ec0cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.030987 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25765fb687294e9db1289f76530dd877. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.031201 896 tablet_service.cc:1505] Processing CreateTablet for tablet d1e0e74f32a24729b250c5751677cfb0 (DEFAULT_TABLE table=default.Table_ba9c16aaa48849db9fece59168c16886 [id=f1f6c97fc88340da8bab0347ad6430ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.033994 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 929eb48881b141718873d0e359e768a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.036671 1324 raft_consensus.cc:493] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:18.036849 1324 raft_consensus.cc:515] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.037496 1324 leader_election.cc:290] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:18.038475 527 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet bc1539e5c8aa4d6e98f687728b891530
I20260202 02:52:18.038977 527 catalog_manager.cc:6724] ReplaceTablet: tablet bc1539e5c8aa4d6e98f687728b891530 deleted and replaced by tablet 84689b18ad274ba0a1522441980b62ce
I20260202 02:52:18.040407 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94bada52db6a47179415d302b541d01f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.030674 895 tablet_service.cc:1505] Processing CreateTablet for tablet d67e91a022f1401ea724dff898c3b81b (DEFAULT_TABLE table=default.Table_c245b83c1987454fabbe8646d14252d6 [id=08498a654b6a4a9ca3f968d70daad9b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.043221 1338 raft_consensus.cc:493] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:18.043452 1338 raft_consensus.cc:515] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.044183 1338 leader_election.cc:290] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:18.047477 1174 tablet_bootstrap.cc:492] T fe7b6004e5094c0dbe5847144a2489b9 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:18.047654 1174 ts_tablet_manager.cc:1403] T fe7b6004e5094c0dbe5847144a2489b9 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.032s user 0.007s sys 0.005s
I20260202 02:52:18.048476 1174 raft_consensus.cc:359] T fe7b6004e5094c0dbe5847144a2489b9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.048650 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d67e91a022f1401ea724dff898c3b81b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.048667 1174 raft_consensus.cc:385] T fe7b6004e5094c0dbe5847144a2489b9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:18.048749 1174 raft_consensus.cc:740] T fe7b6004e5094c0dbe5847144a2489b9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:18.048985 1174 consensus_queue.cc:260] T fe7b6004e5094c0dbe5847144a2489b9 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.051435 1174 ts_tablet_manager.cc:1434] T fe7b6004e5094c0dbe5847144a2489b9 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260202 02:52:18.051872 1174 tablet_bootstrap.cc:492] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:18.053987 1174 tablet_bootstrap.cc:654] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:18.059439 779 raft_consensus.cc:2468] T 0b411e8fab074aa88f8380e3f5694340 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
I20260202 02:52:18.062673 1174 tablet_bootstrap.cc:492] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:18.062974 1174 ts_tablet_manager.cc:1403] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20260202 02:52:18.064620 1174 raft_consensus.cc:359] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.065436 1174 raft_consensus.cc:385] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:18.065590 1174 raft_consensus.cc:740] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:18.065929 1174 consensus_queue.cc:260] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.066716 1174 ts_tablet_manager.cc:1434] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260202 02:52:18.067164 1174 tablet_bootstrap.cc:492] T d06b77ff1a6f4f78b73082b00145fcf6 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:18.069697 1174 tablet_bootstrap.cc:654] T d06b77ff1a6f4f78b73082b00145fcf6 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:18.074270 750 tablet_service.cc:1552] Processing DeleteTablet for tablet bc1539e5c8aa4d6e98f687728b891530 with delete_type TABLET_DATA_DELETED (replaced by tablet 84689b18ad274ba0a1522441980b62ce) from {username='slave'} at 127.0.0.1:42950
W20260202 02:52:18.076367 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet bc1539e5c8aa4d6e98f687728b891530 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bc1539e5c8aa4d6e98f687728b891530 already in progress: creating tablet
I20260202 02:52:18.076727 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1e0e74f32a24729b250c5751677cfb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.078500 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_fe06d3b0b66447e29403a54ab14c476c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:18.079540 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fe06d3b0b66447e29403a54ab14c476c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:18.082700 519 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 25765fb687294e9db1289f76530dd877 consensus state has no leader
I20260202 02:52:18.083201 519 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_e749897cf0214764baf40d6c8d5fc316" } modify_external_catalogs: true
I20260202 02:52:18.083407 519 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_e749897cf0214764baf40d6c8d5fc316" } modify_external_catalogs: true
I20260202 02:52:18.084400 522 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet f58a7b25815c4b5bac568aa11c19610c
I20260202 02:52:18.084913 522 catalog_manager.cc:6724] ReplaceTablet: tablet f58a7b25815c4b5bac568aa11c19610c deleted and replaced by tablet cb634d4bae554ef3b39eeaa9958adc6c
I20260202 02:52:18.085022 528 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet b76d1837813c46e8a0925b49939d8884 (table default.Table_3919d090261e4b90b0672f2b863588b2 [id=fe7b88b74c864c2f98365c0956117bf1]). Expected version 1 got 0
I20260202 02:52:18.085237 528 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 583e547d0a1943f09f4efe82070fc777 (table default.Table_2669ca9662b942c08bcbfd7641cfe2e3 [id=fb813e4c5b0849f8a488c7a22d1defae]). Expected version 1 got 0
I20260202 02:52:18.085400 528 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet c63a670d5c5e489897bc6dbb142b844d (table default.Table_12d68a317c8e4d809dbbe77c095a1f6e [id=7219522ffc3849faab94d754a42a8511]). Expected version 1 got 0
I20260202 02:52:18.085633 528 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 63a9029bc39c4fbd8ac4bf4bc37d6b4e (table default.Table_6c46c4d0140e47cb8edc42ccd2748741 [id=fa95e04db2d443b5972de98c7c0e6457]). Expected version 1 got 0
I20260202 02:52:18.085794 528 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet d06b77ff1a6f4f78b73082b00145fcf6 (table default.Table_5308cabde3e94be1ab7a0b4e15ad67a9 [id=915b211258ad498286448a5b2598a289]). Expected version 1 got 0
I20260202 02:52:18.086354 528 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 267bd4752c5f4b99a3fe778ba8f38a2f (table default.Table_dda472e63fac4873ab3ccefcac3635cd [id=40105de1d8074d3e983ff62557d14f48]). Expected version 1 got 0
I20260202 02:52:18.086553 528 catalog_manager.cc:5654] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130). New cstate: current_term: 1 leader_uuid: "1521a6d262b7451bbef8debe970dd6b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: HEALTHY } } }
I20260202 02:52:18.086969 528 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet d2d27ceea1bb4122b18f0fcf4d2d1bf4 (table default.Table_10aa0dafec0547069eb92ecf12b3c780 [id=1f9c48c8e2484856a66a0719268f22ae]). Expected version 1 got 0
I20260202 02:52:18.087236 528 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet c12ff20ca6b44ddba0c936a28cfb2eb9 (table default.Table_c9d046b019d94f46bee8e3fd28d253ca [id=bdf2f19a439d4be3aa2362cfde38ab7f]). Expected version 1 got 0
I20260202 02:52:18.087412 528 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 723921dfe54c4e66af5cab76a25fee0a (table default.Table_bdfbf5f796b949e196b1c1609fbb3655 [id=c30593053ed7490588ff567e30984807]). Expected version 1 got 0
I20260202 02:52:18.087574 528 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet ff6f5710de4b4c48948aacdf5be29e8b (table default.Table_4720e764353243a495245b92e5b913fb [id=f2276e1c2ca04a70804053da61dd3447]). Expected version 1 got 0
I20260202 02:52:18.088012 528 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 5986676f5484498da927b4c6da3f47ff (table default.Table_8cb3cbab1a654f88a07d6d480fc72e52 [id=d8485f23f63b493781bea557dbe8a0c1]). Expected version 1 got 0
I20260202 02:52:18.088161 528 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet c0937b6910534ddc8a2217f0cbd539c5 (table default.Table_e0149b31be0c49d19600de4849d6577e [id=46b5d47377474ff9bac2acdb26b08c65]). Expected version 1 got 0
I20260202 02:52:18.090093 901 tablet_service.cc:1505] Processing CreateTablet for tablet f6746ccbf37e477196a9706396d1e3fd (DEFAULT_TABLE table=default.Table_69b88340567c4887947ed8458ffd1cba [id=0c8385b71ceb44199472e1ce640b54e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.090548 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6746ccbf37e477196a9706396d1e3fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.090528 1174 tablet_bootstrap.cc:492] T d06b77ff1a6f4f78b73082b00145fcf6 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:18.090968 1174 ts_tablet_manager.cc:1403] T d06b77ff1a6f4f78b73082b00145fcf6 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.024s user 0.008s sys 0.007s
W20260202 02:52:18.091248 520 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d06b77ff1a6f4f78b73082b00145fcf6 consensus state has no leader
I20260202 02:52:18.091639 521 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 75b97d7e848241cf8d894b0ae92a8f8f
I20260202 02:52:18.091663 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_92b7f492c09948f784c64ccdcaadbb86"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:18.091768 1174 raft_consensus.cc:359] T d06b77ff1a6f4f78b73082b00145fcf6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.091967 1174 raft_consensus.cc:385] T d06b77ff1a6f4f78b73082b00145fcf6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:18.092056 1174 raft_consensus.cc:740] T d06b77ff1a6f4f78b73082b00145fcf6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
W20260202 02:52:18.092283 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_92b7f492c09948f784c64ccdcaadbb86 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:18.092299 1174 consensus_queue.cc:260] T d06b77ff1a6f4f78b73082b00145fcf6 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.092468 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_733044f900d44218a06c976c8701f122"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:18.092993 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_733044f900d44218a06c976c8701f122 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:18.093014 1174 ts_tablet_manager.cc:1434] T d06b77ff1a6f4f78b73082b00145fcf6 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:18.093825 1174 tablet_bootstrap.cc:492] T c184d44d7f434b39a997a7387e5d3de0 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:18.085245 522 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_6c46c4d0140e47cb8edc42ccd2748741" } new_table_name: "default.Table_ee295f1a8e36417d95a66ecbc7a680ab" modify_external_catalogs: true
I20260202 02:52:18.097435 1174 tablet_bootstrap.cc:654] T c184d44d7f434b39a997a7387e5d3de0 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:18.098578 901 tablet_service.cc:1505] Processing CreateTablet for tablet 25765fb687294e9db1289f76530dd877 (DEFAULT_TABLE table=default.Table_024075b20add4ddcbc4b27601a94eed4 [id=ea0c52a72da8422ba14345fbfc242d27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.099313 896 tablet_service.cc:1505] Processing CreateTablet for tablet 3711910e050c4515bed4294d1271d5a9 (DEFAULT_TABLE table=default.Table_9451b9f5148f4d9aab65983dea967e32 [id=9ef504467a544f299b48499e49adf27f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.099696 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3711910e050c4515bed4294d1271d5a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.100610 1173 tablet_bootstrap.cc:492] T ffd8aa2bc6e344b78dce518411997fb5 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:18.100800 1173 ts_tablet_manager.cc:1403] T ffd8aa2bc6e344b78dce518411997fb5 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.217s user 0.008s sys 0.013s
I20260202 02:52:18.104311 1324 raft_consensus.cc:493] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:18.104312 895 tablet_service.cc:1552] Processing DeleteTablet for tablet f58a7b25815c4b5bac568aa11c19610c with delete_type TABLET_DATA_DELETED (replaced by tablet cb634d4bae554ef3b39eeaa9958adc6c) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:18.104537 1324 raft_consensus.cc:515] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.104913 895 tablet_service.cc:1552] Processing DeleteTablet for tablet 75b97d7e848241cf8d894b0ae92a8f8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:18.105269 1324 leader_election.cc:290] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:18.106011 754 tablet_service.cc:1552] Processing DeleteTablet for tablet f58a7b25815c4b5bac568aa11c19610c with delete_type TABLET_DATA_DELETED (replaced by tablet cb634d4bae554ef3b39eeaa9958adc6c) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:18.106024 525 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet f93e8d3c4faf48eaa98c3cbf88d585e2
I20260202 02:52:18.106334 752 tablet_service.cc:1552] Processing DeleteTablet for tablet 75b97d7e848241cf8d894b0ae92a8f8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:18.106618 525 catalog_manager.cc:6724] ReplaceTablet: tablet f93e8d3c4faf48eaa98c3cbf88d585e2 deleted and replaced by tablet e491be9ecd124b87bd9aef704e7da266
I20260202 02:52:18.107018 525 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_dda472e63fac4873ab3ccefcac3635cd" } modify_external_catalogs: true
I20260202 02:52:18.107203 525 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_dda472e63fac4873ab3ccefcac3635cd" } modify_external_catalogs: true
I20260202 02:52:18.107355 1338 raft_consensus.cc:493] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:18.107542 1338 raft_consensus.cc:515] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.108261 1338 leader_election.cc:290] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:18.108908 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_d4ad0ff5676e4af2a187497eda81d7a2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:18.110025 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d4ad0ff5676e4af2a187497eda81d7a2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:18.110754 920 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "415d14533fd54f20adb00455a7bcad5f" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:18.110961 920 raft_consensus.cc:2468] T 415d14533fd54f20adb00455a7bcad5f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:18.111507 920 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c12ff20ca6b44ddba0c936a28cfb2eb9" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:18.111704 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cb0dcacdf0f1419da0b25f20cac1d62d" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:18.111830 920 raft_consensus.cc:2468] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:18.112073 1042 tablet_service.cc:1505] Processing CreateTablet for tablet 25765fb687294e9db1289f76530dd877 (DEFAULT_TABLE table=default.Table_024075b20add4ddcbc4b27601a94eed4 [id=ea0c52a72da8422ba14345fbfc242d27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.108289 922 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "63a9029bc39c4fbd8ac4bf4bc37d6b4e" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:18.113009 922 raft_consensus.cc:2468] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:18.113163 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "70306ed9dec54286885d965f2c012da4" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:18.113577 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8095e0c6ff0041fc8b8c8b0778a20b00" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:18.113937 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "723921dfe54c4e66af5cab76a25fee0a" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:18.114058 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 23c38ef4e3c840808ec36f859cf6f6de (table default.Table_9fd715491a6a4bbc8685138d0194362d [id=98ddcbfb5ad14ae480a54845ec7c45a4]) successfully deleted
I20260202 02:52:18.114138 778 raft_consensus.cc:2468] T 723921dfe54c4e66af5cab76a25fee0a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
I20260202 02:52:18.114435 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c064ea0dade840b4840a5715c2d83f69" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:18.114595 714 leader_election.cc:343] T 560465c063d94ffaab33f9f79a58a357 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 560465c063d94ffaab33f9f79a58a357
W20260202 02:52:18.114995 714 leader_election.cc:343] T cb0dcacdf0f1419da0b25f20cac1d62d P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: cb0dcacdf0f1419da0b25f20cac1d62d
I20260202 02:52:18.115381 714 leader_election.cc:304] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters:
I20260202 02:52:18.115736 1324 raft_consensus.cc:2804] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:18.115854 1324 raft_consensus.cc:493] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:18.115975 1324 raft_consensus.cc:3060] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:18.115953 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d2fa4c3d6ec44047abf5986e502c1590" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:18.116209 776 raft_consensus.cc:2468] T d2fa4c3d6ec44047abf5986e502c1590 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
I20260202 02:52:18.116441 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "583e547d0a1943f09f4efe82070fc777" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:18.116607 779 raft_consensus.cc:2468] T 583e547d0a1943f09f4efe82070fc777 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
I20260202 02:52:18.117167 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 575016c0e9f84e99b962c5f14ffa96cc with delete_type TABLET_DATA_DELETED (replaced by tablet ff6f5710de4b4c48948aacdf5be29e8b) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:18.117198 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet f58a7b25815c4b5bac568aa11c19610c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f58a7b25815c4b5bac568aa11c19610c already in progress: creating tablet
W20260202 02:52:18.117441 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 75b97d7e848241cf8d894b0ae92a8f8f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 75b97d7e848241cf8d894b0ae92a8f8f already in progress: creating tablet
I20260202 02:52:18.112007 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8095e0c6ff0041fc8b8c8b0778a20b00" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:18.118752 1324 raft_consensus.cc:515] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.119994 1324 leader_election.cc:290] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:18.119077 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "560465c063d94ffaab33f9f79a58a357" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:18.120690 859 leader_election.cc:343] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:18.121145 715 leader_election.cc:343] T cb0dcacdf0f1419da0b25f20cac1d62d P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:18.121392 715 leader_election.cc:304] T cb0dcacdf0f1419da0b25f20cac1d62d P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
W20260202 02:52:18.121506 522 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 63a9029bc39c4fbd8ac4bf4bc37d6b4e consensus state has no leader
I20260202 02:52:18.121752 519 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet 7fbb4a11065a463d8869e82ecb162424
I20260202 02:52:18.121838 1277 raft_consensus.cc:2749] T cb0dcacdf0f1419da0b25f20cac1d62d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:18.122115 519 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_1fd940660d9b4ef292d7aaa5e0e336cb" } new_table_name: "default.Table_086abcdd67e94b38ac7f5d98777f5c6b" modify_external_catalogs: true
I20260202 02:52:18.122298 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_2acd399bc33146fe9f1b6e2a15e0b101"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:18.122812 857 leader_election.cc:304] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
W20260202 02:52:18.122946 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2acd399bc33146fe9f1b6e2a15e0b101 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:18.098975 900 tablet_service.cc:1552] Processing DeleteTablet for tablet bc1539e5c8aa4d6e98f687728b891530 with delete_type TABLET_DATA_DELETED (replaced by tablet 84689b18ad274ba0a1522441980b62ce) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:18.124025 1338 raft_consensus.cc:2804] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:18.121779 522 master_service.cc:954] ReplaceTablet: received request to replace tablet 0b411e8fab074aa88f8380e3f5694340 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:18.124176 1338 raft_consensus.cc:697] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:18.124457 1338 consensus_queue.cc:237] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.125905 1039 tablet_service.cc:1552] Processing DeleteTablet for tablet f58a7b25815c4b5bac568aa11c19610c with delete_type TABLET_DATA_DELETED (replaced by tablet cb634d4bae554ef3b39eeaa9958adc6c) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:18.126565 715 leader_election.cc:343] T 560465c063d94ffaab33f9f79a58a357 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:18.126734 715 leader_election.cc:304] T 560465c063d94ffaab33f9f79a58a357 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
W20260202 02:52:18.127246 715 consensus_peers.cc:597] T c050a2882f764dda8df2a7df6547f149 P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:18.129777 1324 raft_consensus.cc:2749] T 560465c063d94ffaab33f9f79a58a357 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:18.130024 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_f9071fc3e9d14dfd95877b77b0ac405e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:18.130841 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f9071fc3e9d14dfd95877b77b0ac405e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:18.123529 1510 tablet.cc:1722] T c0937b6910534ddc8a2217f0cbd539c5 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:18.131625 714 leader_election.cc:304] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters:
I20260202 02:52:18.132218 1324 raft_consensus.cc:2804] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:18.132354 1324 raft_consensus.cc:493] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:18.132477 1324 raft_consensus.cc:3060] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:18.132877 857 leader_election.cc:343] T 70306ed9dec54286885d965f2c012da4 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:18.134215 857 leader_election.cc:304] T 70306ed9dec54286885d965f2c012da4 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:18.135607 1453 raft_consensus.cc:2749] T 70306ed9dec54286885d965f2c012da4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:18.136097 857 leader_election.cc:343] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:18.136656 857 leader_election.cc:304] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:18.141418 1324 raft_consensus.cc:515] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.141913 1453 raft_consensus.cc:2749] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:18.142792 857 leader_election.cc:304] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters:
I20260202 02:52:18.143412 1453 raft_consensus.cc:2804] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:18.143527 1453 raft_consensus.cc:493] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:18.143610 1453 raft_consensus.cc:3060] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:18.137797 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_80912cc76f934f139da7f866546ae97a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:18.145586 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_80912cc76f934f139da7f866546ae97a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:18.146616 1453 raft_consensus.cc:515] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.148545 1453 leader_election.cc:290] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:18.148898 1173 raft_consensus.cc:359] T ffd8aa2bc6e344b78dce518411997fb5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:18.149121 1173 raft_consensus.cc:385] T ffd8aa2bc6e344b78dce518411997fb5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:18.149092 752 tablet_service.cc:1552] Processing DeleteTablet for tablet f93e8d3c4faf48eaa98c3cbf88d585e2 with delete_type TABLET_DATA_DELETED (replaced by tablet e491be9ecd124b87bd9aef704e7da266) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:18.149200 1173 raft_consensus.cc:740] T ffd8aa2bc6e344b78dce518411997fb5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:18.149405 1173 consensus_queue.cc:260] T ffd8aa2bc6e344b78dce518411997fb5 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:18.149911 1173 ts_tablet_manager.cc:1434] T ffd8aa2bc6e344b78dce518411997fb5 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.049s user 0.001s sys 0.001s
I20260202 02:52:18.150233 1173 tablet_bootstrap.cc:492] T 73f8db8f30f14d369d2227c9be7c4647 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:18.150260 1324 leader_election.cc:290] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:18.148833 1035 tablet_service.cc:1505] Processing CreateTablet for tablet f6746ccbf37e477196a9706396d1e3fd (DEFAULT_TABLE table=default.Table_69b88340567c4887947ed8458ffd1cba [id=0c8385b71ceb44199472e1ce640b54e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.148867 1061 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "723921dfe54c4e66af5cab76a25fee0a" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:18.152606 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet bc1539e5c8aa4d6e98f687728b891530 with delete_type TABLET_DATA_DELETED (replaced by tablet 84689b18ad274ba0a1522441980b62ce) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:18.153064 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25765fb687294e9db1289f76530dd877. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.154826 523 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet e491be9ecd124b87bd9aef704e7da266
I20260202 02:52:18.155263 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_52fba283eb604f81acbbc5ea3762d1f9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:18.155906 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_52fba283eb604f81acbbc5ea3762d1f9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:18.156404 999 leader_election.cc:343] T add934bf70ae4de8a40066f06fd94795 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: add934bf70ae4de8a40066f06fd94795
I20260202 02:52:18.156533 999 leader_election.cc:304] T add934bf70ae4de8a40066f06fd94795 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
W20260202 02:52:18.156733 999 raft_consensus.cc:2686] T add934bf70ae4de8a40066f06fd94795 P b1f05820b6034228bdad884cc348716d: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260202 02:52:18.157192 1058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "583e547d0a1943f09f4efe82070fc777" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:18.157572 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "415d14533fd54f20adb00455a7bcad5f" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:18.157897 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c12ff20ca6b44ddba0c936a28cfb2eb9" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:18.158134 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d2fa4c3d6ec44047abf5986e502c1590" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:18.158347 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 75b97d7e848241cf8d894b0ae92a8f8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:18.158522 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "415d14533fd54f20adb00455a7bcad5f" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:18.158849 1039 tablet_service.cc:1552] Processing DeleteTablet for tablet f93e8d3c4faf48eaa98c3cbf88d585e2 with delete_type TABLET_DATA_DELETED (replaced by tablet e491be9ecd124b87bd9aef704e7da266) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:18.159137 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6746ccbf37e477196a9706396d1e3fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.164738 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_f92d6341db3143e490f6d64418ef6ff2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:18.165436 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f92d6341db3143e490f6d64418ef6ff2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:18.167230 1174 tablet_bootstrap.cc:492] T c184d44d7f434b39a997a7387e5d3de0 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:18.167424 1174 ts_tablet_manager.cc:1403] T c184d44d7f434b39a997a7387e5d3de0 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.074s user 0.020s sys 0.014s
W20260202 02:52:18.167934 859 leader_election.cc:343] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:18.168061 519 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 94bada52db6a47179415d302b541d01f consensus state has no leader
W20260202 02:52:18.168196 859 leader_election.cc:343] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:18.168354 519 master_service.cc:954] ReplaceTablet: received request to replace tablet eaa47f3388144e8b852804c167f232ce from {username='slave'} at 127.0.0.1:47430
W20260202 02:52:18.168458 859 leader_election.cc:343] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:18.168797 715 leader_election.cc:343] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:18.169128 715 leader_election.cc:343] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:18.169449 715 leader_election.cc:343] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:18.169447 1174 raft_consensus.cc:359] T c184d44d7f434b39a997a7387e5d3de0 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:18.169786 1174 raft_consensus.cc:385] T c184d44d7f434b39a997a7387e5d3de0 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:18.169870 1174 raft_consensus.cc:740] T c184d44d7f434b39a997a7387e5d3de0 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:18.170095 1174 consensus_queue.cc:260] T c184d44d7f434b39a997a7387e5d3de0 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:18.170179 527 rpcz_store.cc:275] Call kudu.master.MasterService.GetMasterRegistration from 127.0.0.1:47402 (request call id 12) took 1725 ms. Trace:
I20260202 02:52:18.170284 527 rpcz_store.cc:276] 0202 02:52:16.444610 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:18.169911 (+1725301us) service_pool.cc:225] Handling call
0202 02:52:18.170142 (+ 231us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:18.170600 1174 ts_tablet_manager.cc:1434] T c184d44d7f434b39a997a7387e5d3de0 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260202 02:52:18.170754 527 rpcz_store.cc:275] Call kudu.master.MasterService.GetMasterRegistration from 127.0.0.1:47398 (request call id 2) took 1513 ms. Trace:
I20260202 02:52:18.170853 527 rpcz_store.cc:276] 0202 02:52:16.656932 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:18.170487 (+1513555us) service_pool.cc:225] Handling call
0202 02:52:18.170723 (+ 236us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:18.170953 1174 tablet_bootstrap.cc:492] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:18.171592 1173 tablet_bootstrap.cc:654] T 73f8db8f30f14d369d2227c9be7c4647 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:18.172452 1277 raft_consensus.cc:493] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:18.172616 1277 raft_consensus.cc:515] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:18.173310 1277 leader_election.cc:290] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:18.175081 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25765fb687294e9db1289f76530dd877. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.175681 1174 tablet_bootstrap.cc:654] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:18.098707 1182 ts_tablet_manager.cc:1916] T bf5ca99708044482970cb854d2eced18 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:18.177825 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ff6f5710de4b4c48948aacdf5be29e8b" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:18.179297 715 leader_election.cc:343] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:18.180344 494 master_runner.cc:186] Error getting master registration for 127.0.104.188:39077: Network error: Client connection negotiation failed: client connection to 127.0.104.188:39077: connect: Connection refused (error 111),
I20260202 02:52:18.180557 494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:52:18.181514 1059 raft_consensus.cc:3060] T 251455168e33430aafea8e0de42ca1b8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:18.183142 1174 tablet_bootstrap.cc:492] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:18.183331 1174 ts_tablet_manager.cc:1403] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.004s
I20260202 02:52:18.184047 1277 raft_consensus.cc:493] T 724d47241f0342768fb1321ef37c3a7b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:18.184191 1277 raft_consensus.cc:515] T 724d47241f0342768fb1321ef37c3a7b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:18.184284 1174 raft_consensus.cc:359] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.184526 1174 raft_consensus.cc:385] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:18.184628 1174 raft_consensus.cc:740] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:18.184913 1277 leader_election.cc:290] T 724d47241f0342768fb1321ef37c3a7b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:18.185359 1183 ts_tablet_manager.cc:1929] T add934bf70ae4de8a40066f06fd94795 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:18.185510 1183 log.cc:1199] T add934bf70ae4de8a40066f06fd94795 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/add934bf70ae4de8a40066f06fd94795
I20260202 02:52:18.185601 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c12ff20ca6b44ddba0c936a28cfb2eb9" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:18.185627 1057 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "723921dfe54c4e66af5cab76a25fee0a" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:18.186022 426 master_runner.cc:186] Error getting master registration for 127.0.104.188:39077: Network error: Client connection negotiation failed: client connection to 127.0.104.188:39077: connect: Connection refused (error 111),
I20260202 02:52:18.186213 426 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:52:18.186476 1183 ts_tablet_manager.cc:1950] T add934bf70ae4de8a40066f06fd94795 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
W20260202 02:52:18.187136 715 leader_election.cc:343] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:18.187115 1057 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "724d47241f0342768fb1321ef37c3a7b" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:18.187291 1040 tablet_service.cc:1552] Processing DeleteTablet for tablet f24c0b99574d437a99b273204f105460 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:18.188117 715 leader_election.cc:343] T 724d47241f0342768fb1321ef37c3a7b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:18.189229 1174 consensus_queue.cc:260] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.189680 1183 tablet_replica.cc:333] T 575016c0e9f84e99b962c5f14ffa96cc P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:18.189886 1183 raft_consensus.cc:2243] T 575016c0e9f84e99b962c5f14ffa96cc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:18.190076 1183 raft_consensus.cc:2272] T 575016c0e9f84e99b962c5f14ffa96cc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:18.190626 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet add934bf70ae4de8a40066f06fd94795 (table default.Table_9fb69280bd6048d898c8e92f2ef9f61e [id=7cf1e30412cb48ab962f526efee72121]) successfully deleted
W20260202 02:52:18.191185 859 leader_election.cc:343] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:18.191402 1174 ts_tablet_manager.cc:1434] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.008s user 0.000s sys 0.003s
I20260202 02:52:18.191752 1174 tablet_bootstrap.cc:492] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:18.193986 1183 ts_tablet_manager.cc:1916] T 575016c0e9f84e99b962c5f14ffa96cc P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:18.195217 1174 tablet_bootstrap.cc:654] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:18.199386 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_9451b9f5148f4d9aab65983dea967e32" } new_table_name: "default.Table_ab29f752f1e140da9038d9e11ee61f39" modify_external_catalogs: true
I20260202 02:52:18.199934 1277 raft_consensus.cc:493] T fe7b6004e5094c0dbe5847144a2489b9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:18.200114 1277 raft_consensus.cc:515] T fe7b6004e5094c0dbe5847144a2489b9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.200758 1277 leader_election.cc:290] T fe7b6004e5094c0dbe5847144a2489b9 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:18.202198 1057 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fe7b6004e5094c0dbe5847144a2489b9" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:18.202823 521 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet ffd8aa2bc6e344b78dce518411997fb5 (table default.Table_ab8d74f30cd14c61b85950a82d0f3787 [id=3ccb047c30974c0f9cbe778971e61644]). Expected version 3 got 0
W20260202 02:52:18.204612 715 leader_election.cc:343] T fe7b6004e5094c0dbe5847144a2489b9 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:18.212870 1174 tablet_bootstrap.cc:492] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:18.213078 1174 ts_tablet_manager.cc:1403] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.021s user 0.006s sys 0.003s
I20260202 02:52:18.148792 1038 tablet_service.cc:1505] Processing CreateTablet for tablet 3711910e050c4515bed4294d1271d5a9 (DEFAULT_TABLE table=default.Table_9451b9f5148f4d9aab65983dea967e32 [id=9ef504467a544f299b48499e49adf27f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.213922 1174 raft_consensus.cc:359] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.214164 1174 raft_consensus.cc:385] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:18.214242 1174 raft_consensus.cc:740] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:18.214346 1453 raft_consensus.cc:493] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:18.214497 1174 consensus_queue.cc:260] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.214079 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3711910e050c4515bed4294d1271d5a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.214594 1453 raft_consensus.cc:515] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.218397 1057 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b76d1837813c46e8a0925b49939d8884" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:18.218516 1174 ts_tablet_manager.cc:1434] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260202 02:52:18.220129 1174 tablet_bootstrap.cc:492] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:18.224195 1174 tablet_bootstrap.cc:654] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:18.228498 859 leader_election.cc:343] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:18.230974 526 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet c0937b6910534ddc8a2217f0cbd539c5 (table default.Table_e0149b31be0c49d19600de4849d6577e [id=46b5d47377474ff9bac2acdb26b08c65]). Expected version 1 got 0
I20260202 02:52:18.231308 1174 tablet_bootstrap.cc:492] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:18.231279 526 catalog_manager.cc:5654] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f442ef85029461091de847118c34c0d (127.0.104.129). New cstate: current_term: 1 leader_uuid: "1f442ef85029461091de847118c34c0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } }
I20260202 02:52:18.231559 1174 ts_tablet_manager.cc:1403] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.000s
I20260202 02:52:18.231703 526 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet 5986676f5484498da927b4c6da3f47ff (table default.Table_8cb3cbab1a654f88a07d6d480fc72e52 [id=d8485f23f63b493781bea557dbe8a0c1]). Expected version 1 got 0
I20260202 02:52:18.232191 526 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet d2d27ceea1bb4122b18f0fcf4d2d1bf4 (table default.Table_10aa0dafec0547069eb92ecf12b3c780 [id=1f9c48c8e2484856a66a0719268f22ae]). Expected version 1 got 0
I20260202 02:52:18.232918 752 tablet_service.cc:1505] Processing CreateTablet for tablet 6154460838eb4e508f3646d3e3e4cf21 (DEFAULT_TABLE table=default.Table_2ca7a7f464b54d398f58e0ca6a5e269b [id=7d8f7ac4c2ee4b72b2ff25aff060010f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.233335 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6154460838eb4e508f3646d3e3e4cf21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.233331 1042 tablet_service.cc:1505] Processing CreateTablet for tablet 6154460838eb4e508f3646d3e3e4cf21 (DEFAULT_TABLE table=default.Table_2ca7a7f464b54d398f58e0ca6a5e269b [id=7d8f7ac4c2ee4b72b2ff25aff060010f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.233748 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6154460838eb4e508f3646d3e3e4cf21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.237468 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_91411bac2a2a45a395b81a4dab2617f7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:18.238173 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_91411bac2a2a45a395b81a4dab2617f7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:18.239785 1035 tablet_service.cc:1505] Processing CreateTablet for tablet 2f0e8c5e2fd44370ae150a206f4e6cbc (DEFAULT_TABLE table=default.Table_04337834a3ce43ae8f6012b818d70d9f [id=3d1bb64023a145bb9fdcb2378a844a37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.239935 526 catalog_manager.cc:5654] T c050a2882f764dda8df2a7df6547f149 P 1f442ef85029461091de847118c34c0d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f442ef85029461091de847118c34c0d (127.0.104.129). New cstate: current_term: 1 leader_uuid: "1f442ef85029461091de847118c34c0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } }
I20260202 02:52:18.240448 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f0e8c5e2fd44370ae150a206f4e6cbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.240577 526 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet ff6f5710de4b4c48948aacdf5be29e8b (table default.Table_4720e764353243a495245b92e5b913fb [id=f2276e1c2ca04a70804053da61dd3447]). Expected version 1 got 0
I20260202 02:52:18.240917 528 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_b3001bd4a86843ada9ee50b8e4915a9a [id=36a229921ecc4185adbfdfba372a38ff] alter complete (version 1)
I20260202 02:52:18.240926 524 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet d67e91a022f1401ea724dff898c3b81b
I20260202 02:52:18.243136 1175 tablet_bootstrap.cc:492] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:18.243129 754 tablet_service.cc:1505] Processing CreateTablet for tablet 2f0e8c5e2fd44370ae150a206f4e6cbc (DEFAULT_TABLE table=default.Table_04337834a3ce43ae8f6012b818d70d9f [id=3d1bb64023a145bb9fdcb2378a844a37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.243556 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f0e8c5e2fd44370ae150a206f4e6cbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.243892 1277 raft_consensus.cc:493] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:18.244053 1277 raft_consensus.cc:515] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.244812 1277 leader_election.cc:290] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:18.245100 1175 ts_tablet_manager.cc:1403] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.275s user 0.004s sys 0.000s
I20260202 02:52:18.246028 1175 raft_consensus.cc:359] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:18.246227 1175 raft_consensus.cc:385] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:18.246307 1175 raft_consensus.cc:740] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:18.246536 1175 consensus_queue.cc:260] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:18.247128 1175 ts_tablet_manager.cc:1434] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:18.247475 1175 tablet_bootstrap.cc:492] T eaa47f3388144e8b852804c167f232ce P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:18.249347 1453 leader_election.cc:290] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:18.250720 1175 tablet_bootstrap.cc:654] T eaa47f3388144e8b852804c167f232ce P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:18.257395 755 tablet_service.cc:1505] Processing CreateTablet for tablet 59990791123c4d4eb620090a1e8e31e1 (DEFAULT_TABLE table=default.Table_26642341d3d54c0fa9af19fef3a5b06d [id=9ff2eb6ee89943918019f5e899ac362b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.257923 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59990791123c4d4eb620090a1e8e31e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.260082 1277 raft_consensus.cc:493] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:18.260303 1277 raft_consensus.cc:515] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.261076 1277 leader_election.cc:290] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:18.262862 1182 ts_tablet_manager.cc:1929] T bf5ca99708044482970cb854d2eced18 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:18.262997 1182 log.cc:1199] T bf5ca99708044482970cb854d2eced18 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/bf5ca99708044482970cb854d2eced18
I20260202 02:52:18.263378 1182 ts_tablet_manager.cc:1950] T bf5ca99708044482970cb854d2eced18 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:18.257409 752 tablet_service.cc:1505] Processing CreateTablet for tablet e1284fde67bc4d65b067a54c416ccc33 (DEFAULT_TABLE table=default.Table_4b9f73d0a12f4218872a70ed1e421519 [id=e304e70f12c3479694cd69693ae14cef]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:18.264919 857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 79814.6us
I20260202 02:52:18.265108 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1284fde67bc4d65b067a54c416ccc33. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:18.265197 857 leader_election.cc:343] T c064ea0dade840b4840a5715c2d83f69 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:18.265332 857 leader_election.cc:304] T c064ea0dade840b4840a5715c2d83f69 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:18.265836 1338 raft_consensus.cc:2749] T c064ea0dade840b4840a5715c2d83f69 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:18.266049 1338 consensus_queue.cc:1048] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:18.267024 857 leader_election.cc:304] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:18.268394 1338 raft_consensus.cc:2804] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:18.268522 1338 raft_consensus.cc:493] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:18.268611 1338 raft_consensus.cc:3060] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:18.271322 1338 raft_consensus.cc:515] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.272404 1338 leader_election.cc:290] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:18.272938 857 leader_election.cc:304] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:18.277515 524 master_service.cc:954] ReplaceTablet: received request to replace tablet 526f2c7b95184a0987e7aa94efc8475a from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:18.277655 1057 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "761f5a51335a4de5977f5f7fef0d4260" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:18.278162 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d2fa4c3d6ec44047abf5986e502c1590" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:18.278384 1338 raft_consensus.cc:2804] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:18.278447 1173 tablet_bootstrap.cc:492] T 73f8db8f30f14d369d2227c9be7c4647 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:18.278528 1338 raft_consensus.cc:493] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:18.278509 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f58a7b25815c4b5bac568aa11c19610c" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:18.278625 1338 raft_consensus.cc:3060] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:18.278908 715 leader_election.cc:343] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:18.279523 715 leader_election.cc:343] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:18.279585 1173 ts_tablet_manager.cc:1403] T 73f8db8f30f14d369d2227c9be7c4647 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.128s user 0.001s sys 0.004s
I20260202 02:52:18.280527 714 leader_election.cc:304] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:18.280992 1173 raft_consensus.cc:359] T 73f8db8f30f14d369d2227c9be7c4647 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.281275 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet d67e91a022f1401ea724dff898c3b81b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:18.281414 1183 ts_tablet_manager.cc:1929] T 575016c0e9f84e99b962c5f14ffa96cc P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:18.281601 1183 log.cc:1199] T 575016c0e9f84e99b962c5f14ffa96cc P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/575016c0e9f84e99b962c5f14ffa96cc
I20260202 02:52:18.281967 1183 ts_tablet_manager.cc:1950] T 575016c0e9f84e99b962c5f14ffa96cc P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:18.275951 1182 tablet_replica.cc:333] T f58a7b25815c4b5bac568aa11c19610c P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:18.282776 1182 raft_consensus.cc:2243] T f58a7b25815c4b5bac568aa11c19610c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:18.282964 1182 raft_consensus.cc:2272] T f58a7b25815c4b5bac568aa11c19610c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:18.283273 859 leader_election.cc:343] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:18.283471 1183 tablet_replica.cc:333] T f24c0b99574d437a99b273204f105460 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:18.283553 1182 ts_tablet_manager.cc:1916] T f58a7b25815c4b5bac568aa11c19610c P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:18.283650 1183 raft_consensus.cc:2243] T f24c0b99574d437a99b273204f105460 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:18.284169 1183 raft_consensus.cc:2272] T f24c0b99574d437a99b273204f105460 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:18.284358 1173 raft_consensus.cc:385] T 73f8db8f30f14d369d2227c9be7c4647 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:18.284430 1173 raft_consensus.cc:740] T 73f8db8f30f14d369d2227c9be7c4647 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:18.284644 1173 consensus_queue.cc:260] T 73f8db8f30f14d369d2227c9be7c4647 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.285048 1038 tablet_service.cc:1505] Processing CreateTablet for tablet e1284fde67bc4d65b067a54c416ccc33 (DEFAULT_TABLE table=default.Table_4b9f73d0a12f4218872a70ed1e421519 [id=e304e70f12c3479694cd69693ae14cef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.285550 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1284fde67bc4d65b067a54c416ccc33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.285511 755 tablet_service.cc:1505] Processing CreateTablet for tablet a5622ef925bc46cca76f481ddfd12afd (DEFAULT_TABLE table=default.Table_3d26f7a34d0c48379e3cebda66e1ef13 [id=45e6d83589fa493195b06f633e295b00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.285924 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5622ef925bc46cca76f481ddfd12afd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.287856 1040 tablet_service.cc:1505] Processing CreateTablet for tablet 59990791123c4d4eb620090a1e8e31e1 (DEFAULT_TABLE table=default.Table_26642341d3d54c0fa9af19fef3a5b06d [id=9ff2eb6ee89943918019f5e899ac362b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.288275 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59990791123c4d4eb620090a1e8e31e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.288568 754 tablet_service.cc:1552] Processing DeleteTablet for tablet d67e91a022f1401ea724dff898c3b81b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:18.288897 1277 raft_consensus.cc:2804] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:18.289006 1277 raft_consensus.cc:493] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:18.289098 1277 raft_consensus.cc:3060] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:18.289515 1039 tablet_service.cc:1505] Processing CreateTablet for tablet d0671586e45e45ffb7fd9e1be6e2bbf8 (DEFAULT_TABLE table=default.Table_addc8f2e088548c99dbfaf1ae14135d0 [id=b2fe4f8ae50642e9a6ea05759313b31b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.289921 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0671586e45e45ffb7fd9e1be6e2bbf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.289968 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 575016c0e9f84e99b962c5f14ffa96cc (table default.Table_4720e764353243a495245b92e5b913fb [id=f2276e1c2ca04a70804053da61dd3447]) successfully deleted
I20260202 02:52:18.290297 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "723921dfe54c4e66af5cab76a25fee0a" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:18.290498 779 raft_consensus.cc:3060] T 723921dfe54c4e66af5cab76a25fee0a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:18.291812 1277 raft_consensus.cc:515] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.292513 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b76d1837813c46e8a0925b49939d8884" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:18.292745 776 raft_consensus.cc:2468] T b76d1837813c46e8a0925b49939d8884 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
I20260202 02:52:18.293248 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet bf5ca99708044482970cb854d2eced18 (table default.Table_34c95b3937dd4587a9bd81a7c805f562 [id=8227929c83a441af86babfb3cff579bd]) successfully deleted
I20260202 02:52:18.293289 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d2fa4c3d6ec44047abf5986e502c1590" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:18.293471 777 raft_consensus.cc:3060] T d2fa4c3d6ec44047abf5986e502c1590 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:18.293622 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 0d297b6243c348ee82c146a8339cf6c4 because the tablet was not found. No further retry: Not found: Tablet not found: 0d297b6243c348ee82c146a8339cf6c4
W20260202 02:52:18.293831 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 098c5cdfcd0f4c52b6ae4f986f07705a because the tablet was not found. No further retry: Not found: Tablet not found: 098c5cdfcd0f4c52b6ae4f986f07705a
I20260202 02:52:18.281296 1338 raft_consensus.cc:515] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:18.294036 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 575016c0e9f84e99b962c5f14ffa96cc because the tablet was not found. No further retry: Not found: Tablet not found: 575016c0e9f84e99b962c5f14ffa96cc
I20260202 02:52:18.294674 1338 leader_election.cc:290] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:18.294768 1277 leader_election.cc:290] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:18.296201 777 raft_consensus.cc:2468] T d2fa4c3d6ec44047abf5986e502c1590 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
I20260202 02:52:18.296221 1521 raft_consensus.cc:493] T d06b77ff1a6f4f78b73082b00145fcf6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:18.297125 1183 ts_tablet_manager.cc:1916] T f24c0b99574d437a99b273204f105460 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:18.296389 1521 raft_consensus.cc:515] T d06b77ff1a6f4f78b73082b00145fcf6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.298190 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "63a9029bc39c4fbd8ac4bf4bc37d6b4e" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:18.298462 1521 leader_election.cc:290] T d06b77ff1a6f4f78b73082b00145fcf6 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:18.298808 715 leader_election.cc:343] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:18.299659 921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "415d14533fd54f20adb00455a7bcad5f" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:18.299988 921 raft_consensus.cc:3060] T 415d14533fd54f20adb00455a7bcad5f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:18.300153 779 raft_consensus.cc:2468] T 723921dfe54c4e66af5cab76a25fee0a P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
I20260202 02:52:18.301622 920 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c12ff20ca6b44ddba0c936a28cfb2eb9" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:18.301852 920 raft_consensus.cc:3060] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:18.301940 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "583e547d0a1943f09f4efe82070fc777" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:18.302619 1277 raft_consensus.cc:493] T bf74f96bea2e4ab7adec42c3c5207e79 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:18.302801 1277 raft_consensus.cc:515] T bf74f96bea2e4ab7adec42c3c5207e79 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:18.303972 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf74f96bea2e4ab7adec42c3c5207e79" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:18.302898 752 tablet_service.cc:1505] Processing CreateTablet for tablet d0671586e45e45ffb7fd9e1be6e2bbf8 (DEFAULT_TABLE table=default.Table_addc8f2e088548c99dbfaf1ae14135d0 [id=b2fe4f8ae50642e9a6ea05759313b31b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.303989 918 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "761f5a51335a4de5977f5f7fef0d4260" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:18.304270 918 raft_consensus.cc:2468] T 761f5a51335a4de5977f5f7fef0d4260 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
W20260202 02:52:18.304522 715 leader_election.cc:343] T bf74f96bea2e4ab7adec42c3c5207e79 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:18.305717 918 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f58a7b25815c4b5bac568aa11c19610c" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:18.306772 1174 raft_consensus.cc:359] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.307374 918 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "63a9029bc39c4fbd8ac4bf4bc37d6b4e" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:18.307585 918 raft_consensus.cc:3060] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:18.301707 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ff6f5710de4b4c48948aacdf5be29e8b" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:18.309406 923 raft_consensus.cc:2468] T ff6f5710de4b4c48948aacdf5be29e8b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:18.307237 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0671586e45e45ffb7fd9e1be6e2bbf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.309350 1277 leader_election.cc:290] T bf74f96bea2e4ab7adec42c3c5207e79 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:18.309648 1324 raft_consensus.cc:493] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:18.310087 1324 raft_consensus.cc:515] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.310277 1064 raft_consensus.cc:3060] T 73f8db8f30f14d369d2227c9be7c4647 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:18.310720 1324 leader_election.cc:290] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:18.311715 901 tablet_service.cc:1552] Processing DeleteTablet for tablet f93e8d3c4faf48eaa98c3cbf88d585e2 with delete_type TABLET_DATA_DELETED (replaced by tablet e491be9ecd124b87bd9aef704e7da266) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:18.311187 1174 raft_consensus.cc:385] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:18.312206 921 raft_consensus.cc:2468] T 415d14533fd54f20adb00455a7bcad5f P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:18.313038 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bbf31e5619ed4f60b4d7c8b19ec8a549" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:18.312839 1174 raft_consensus.cc:740] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
W20260202 02:52:18.313618 715 leader_election.cc:343] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:18.313990 901 tablet_service.cc:1505] Processing CreateTablet for tablet 6154460838eb4e508f3646d3e3e4cf21 (DEFAULT_TABLE table=default.Table_2ca7a7f464b54d398f58e0ca6a5e269b [id=7d8f7ac4c2ee4b72b2ff25aff060010f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.314268 920 raft_consensus.cc:2468] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:18.314462 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6154460838eb4e508f3646d3e3e4cf21. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:18.315819 859 leader_election.cc:343] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:18.316545 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d06b77ff1a6f4f78b73082b00145fcf6" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:18.319362 859 leader_election.cc:343] T d06b77ff1a6f4f78b73082b00145fcf6 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:18.321434 896 tablet_service.cc:1552] Processing DeleteTablet for tablet d67e91a022f1401ea724dff898c3b81b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:18.321974 900 tablet_service.cc:1505] Processing CreateTablet for tablet 2f0e8c5e2fd44370ae150a206f4e6cbc (DEFAULT_TABLE table=default.Table_04337834a3ce43ae8f6012b818d70d9f [id=3d1bb64023a145bb9fdcb2378a844a37]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:18.322072 999 leader_election.cc:343] T 575016c0e9f84e99b962c5f14ffa96cc P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 575016c0e9f84e99b962c5f14ffa96cc
I20260202 02:52:18.322225 999 leader_election.cc:304] T 575016c0e9f84e99b962c5f14ffa96cc P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
W20260202 02:52:18.322405 999 raft_consensus.cc:2686] T 575016c0e9f84e99b962c5f14ffa96cc P b1f05820b6034228bdad884cc348716d: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260202 02:52:18.322446 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f0e8c5e2fd44370ae150a206f4e6cbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.322630 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "583e547d0a1943f09f4efe82070fc777" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:18.322880 779 raft_consensus.cc:3060] T 583e547d0a1943f09f4efe82070fc777 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:18.324127 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet f58a7b25815c4b5bac568aa11c19610c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f58a7b25815c4b5bac568aa11c19610c already in progress: creating tablet
I20260202 02:52:18.324782 895 tablet_service.cc:1505] Processing CreateTablet for tablet e1284fde67bc4d65b067a54c416ccc33 (DEFAULT_TABLE table=default.Table_4b9f73d0a12f4218872a70ed1e421519 [id=e304e70f12c3479694cd69693ae14cef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.325263 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1284fde67bc4d65b067a54c416ccc33. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:18.325438 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet bc1539e5c8aa4d6e98f687728b891530 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bc1539e5c8aa4d6e98f687728b891530 already in progress: creating tablet
W20260202 02:52:18.325691 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 75b97d7e848241cf8d894b0ae92a8f8f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 75b97d7e848241cf8d894b0ae92a8f8f already in progress: creating tablet
I20260202 02:52:18.325788 779 raft_consensus.cc:2468] T 583e547d0a1943f09f4efe82070fc777 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
I20260202 02:52:18.327237 1175 tablet_bootstrap.cc:492] T eaa47f3388144e8b852804c167f232ce P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:18.327405 1175 ts_tablet_manager.cc:1403] T eaa47f3388144e8b852804c167f232ce P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.080s user 0.007s sys 0.002s
I20260202 02:52:18.328263 1175 raft_consensus.cc:359] T eaa47f3388144e8b852804c167f232ce P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.328864 898 tablet_service.cc:1505] Processing CreateTablet for tablet d0671586e45e45ffb7fd9e1be6e2bbf8 (DEFAULT_TABLE table=default.Table_addc8f2e088548c99dbfaf1ae14135d0 [id=b2fe4f8ae50642e9a6ea05759313b31b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.329324 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0671586e45e45ffb7fd9e1be6e2bbf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.329658 896 tablet_service.cc:1505] Processing CreateTablet for tablet 59990791123c4d4eb620090a1e8e31e1 (DEFAULT_TABLE table=default.Table_26642341d3d54c0fa9af19fef3a5b06d [id=9ff2eb6ee89943918019f5e899ac362b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.330094 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59990791123c4d4eb620090a1e8e31e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.328855 1175 raft_consensus.cc:385] T eaa47f3388144e8b852804c167f232ce P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:18.336588 1175 raft_consensus.cc:740] T eaa47f3388144e8b852804c167f232ce P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:18.336964 1175 consensus_queue.cc:260] T eaa47f3388144e8b852804c167f232ce P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.337811 1175 ts_tablet_manager.cc:1434] T eaa47f3388144e8b852804c167f232ce P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.010s user 0.003s sys 0.000s
I20260202 02:52:18.320470 918 raft_consensus.cc:2468] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:18.348177 1174 consensus_queue.cc:260] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.350131 1174 ts_tablet_manager.cc:1434] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.118s user 0.004s sys 0.000s
I20260202 02:52:18.350553 1174 tablet_bootstrap.cc:492] T c064ea0dade840b4840a5715c2d83f69 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:18.351377 1338 raft_consensus.cc:493] T bc1539e5c8aa4d6e98f687728b891530 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:18.351697 1338 raft_consensus.cc:515] T bc1539e5c8aa4d6e98f687728b891530 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:18.352787 1338 leader_election.cc:290] T bc1539e5c8aa4d6e98f687728b891530 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:18.353482 1175 tablet_bootstrap.cc:492] T 526f2c7b95184a0987e7aa94efc8475a P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:18.355315 1175 tablet_bootstrap.cc:654] T 526f2c7b95184a0987e7aa94efc8475a P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:18.357702 524 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ee295f1a8e36417d95a66ecbc7a680ab" } new_table_name: "default.Table_90344989f4234a11a30653bdc34fe899" modify_external_catalogs: true
I20260202 02:52:18.360318 857 leader_election.cc:304] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:18.360638 1521 raft_consensus.cc:493] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:18.360817 1521 raft_consensus.cc:515] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:18.361438 1521 leader_election.cc:290] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:18.361609 1521 raft_consensus.cc:2804] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:18.361683 1521 raft_consensus.cc:493] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:18.361769 1521 raft_consensus.cc:3060] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:18.363618 521 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_f9071fc3e9d14dfd95877b77b0ac405e" } modify_external_catalogs: true
I20260202 02:52:18.363814 521 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_f9071fc3e9d14dfd95877b77b0ac405e" } modify_external_catalogs: true
I20260202 02:52:18.364789 1521 raft_consensus.cc:515] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:18.363231 513 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.188:39077, user_credentials={real_user=slave}} blocked reactor thread for 73094.5us
I20260202 02:52:18.365957 1521 leader_election.cc:290] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:18.366109 523 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_c9d046b019d94f46bee8e3fd28d253ca" } new_table_name: "default.Table_959d3e3da2814b3a986d29864e47246c" modify_external_catalogs: true
I20260202 02:52:18.366940 1057 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "78f122342dd144b48de89902faa0f94f" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:18.367241 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b76d1837813c46e8a0925b49939d8884" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
W20260202 02:52:18.367923 859 leader_election.cc:343] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:18.368257 859 leader_election.cc:343] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:18.369568 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet d67e91a022f1401ea724dff898c3b81b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d67e91a022f1401ea724dff898c3b81b already in progress: creating tablet
I20260202 02:52:18.372253 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bc1539e5c8aa4d6e98f687728b891530" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:18.372527 1042 tablet_service.cc:1505] Processing CreateTablet for tablet a5622ef925bc46cca76f481ddfd12afd (DEFAULT_TABLE table=default.Table_3d26f7a34d0c48379e3cebda66e1ef13 [id=45e6d83589fa493195b06f633e295b00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.372850 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 2b8ba9dd450948ada395207ae1c5b325 (DEFAULT_TABLE table=default.Table_614c0ff5900b4d64b9c0af8c88a33ecb [id=29f81d15b06d4f8abe2aba83493cfa6a]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:18.374225 859 leader_election.cc:343] T bc1539e5c8aa4d6e98f687728b891530 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:18.374310 1173 ts_tablet_manager.cc:1434] T 73f8db8f30f14d369d2227c9be7c4647 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.091s user 0.000s sys 0.002s
W20260202 02:52:18.358707 714 leader_election.cc:343] T 724d47241f0342768fb1321ef37c3a7b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 724d47241f0342768fb1321ef37c3a7b
I20260202 02:52:18.374634 714 leader_election.cc:304] T 724d47241f0342768fb1321ef37c3a7b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:18.358660 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d06b77ff1a6f4f78b73082b00145fcf6" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:18.375033 1324 raft_consensus.cc:2749] T 724d47241f0342768fb1321ef37c3a7b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:18.375341 778 raft_consensus.cc:2468] T d06b77ff1a6f4f78b73082b00145fcf6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
W20260202 02:52:18.376219 714 leader_election.cc:343] T fe7b6004e5094c0dbe5847144a2489b9 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: fe7b6004e5094c0dbe5847144a2489b9
I20260202 02:52:18.376369 714 leader_election.cc:304] T fe7b6004e5094c0dbe5847144a2489b9 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:18.376605 1324 raft_consensus.cc:2749] T fe7b6004e5094c0dbe5847144a2489b9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:18.376986 1173 tablet_bootstrap.cc:492] T c050a2882f764dda8df2a7df6547f149 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:18.377139 714 leader_election.cc:304] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:18.377413 1324 raft_consensus.cc:2804] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:18.377493 1324 raft_consensus.cc:493] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:18.377583 1324 raft_consensus.cc:3060] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:18.380108 1324 raft_consensus.cc:515] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:18.381026 714 leader_election.cc:343] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Illegal state: Tablet not RUNNING: STOPPED
I20260202 02:52:18.381234 714 leader_election.cc:304] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:18.381811 1321 raft_consensus.cc:2749] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:18.382359 714 leader_election.cc:304] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:18.382696 1321 raft_consensus.cc:2804] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:18.382781 1321 raft_consensus.cc:493] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:18.382874 1321 raft_consensus.cc:3060] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:18.384806 1036 tablet_service.cc:1505] Processing CreateTablet for tablet 20ea39bc5d3d46f5a4ba427277d6c6a3 (DEFAULT_TABLE table=default.Table_3b335a0806154f98a907f551596ba353 [id=0da4db61b3fe4cf4846edd5530b9327f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.385401 1043 tablet_service.cc:1505] Processing CreateTablet for tablet 91ec01d25f8f441cbe3783cacbee51db (DEFAULT_TABLE table=default.Table_b18a63e4731b418abcb84c36082553c1 [id=255f4d94812f45d0b4423150098460ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.385485 1321 raft_consensus.cc:515] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:18.385922 1034 tablet_service.cc:1505] Processing CreateTablet for tablet 79f694b740954d429145fba9157338ca (DEFAULT_TABLE table=default.Table_662111850b514df5ad7c69aa21783397 [id=d595db36d702401ab81d5401ad3a76e2]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:18.386096 527 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3711910e050c4515bed4294d1271d5a9 consensus state has no leader
I20260202 02:52:18.386179 1321 leader_election.cc:290] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:18.386751 527 master_service.cc:954] ReplaceTablet: received request to replace tablet 251455168e33430aafea8e0de42ca1b8 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:18.386823 714 leader_election.cc:304] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:18.387203 714 leader_election.cc:304] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:18.387820 1324 leader_election.cc:290] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:18.388183 1277 raft_consensus.cc:2804] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:18.388888 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ff6f5710de4b4c48948aacdf5be29e8b" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:18.389088 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5622ef925bc46cca76f481ddfd12afd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.389822 1324 raft_consensus.cc:2804] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:18.389911 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91ec01d25f8f441cbe3783cacbee51db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.389961 1324 raft_consensus.cc:697] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:18.390218 1324 consensus_queue.cc:237] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.390745 525 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 267bd4752c5f4b99a3fe778ba8f38a2f
I20260202 02:52:18.391623 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_9181c8b857b540ff846c397f9f1a5484"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:18.392102 522 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 0b411e8fab074aa88f8380e3f5694340
W20260202 02:52:18.392495 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9181c8b857b540ff846c397f9f1a5484 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:18.392577 522 catalog_manager.cc:6724] ReplaceTablet: tablet 0b411e8fab074aa88f8380e3f5694340 deleted and replaced by tablet 28a900bcaa8b41718a285e595886d805
I20260202 02:52:18.392935 522 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ab8d74f30cd14c61b85950a82d0f3787" } modify_external_catalogs: true
I20260202 02:52:18.393110 522 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ab8d74f30cd14c61b85950a82d0f3787" } modify_external_catalogs: true
I20260202 02:52:18.393231 752 tablet_service.cc:1505] Processing CreateTablet for tablet 2b8ba9dd450948ada395207ae1c5b325 (DEFAULT_TABLE table=default.Table_614c0ff5900b4d64b9c0af8c88a33ecb [id=29f81d15b06d4f8abe2aba83493cfa6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.393824 519 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet eaa47f3388144e8b852804c167f232ce
I20260202 02:52:18.394073 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_9dfc2d7b50e94228b7aad2a3f575f625"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:18.394289 519 catalog_manager.cc:6724] ReplaceTablet: tablet eaa47f3388144e8b852804c167f232ce deleted and replaced by tablet 38fdbc6df7ec40b9afe4e0dfe0119e3e
I20260202 02:52:18.394646 519 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_5c1272b44a5b4a84918e3c6c84dca87e" } new_table_name: "default.Table_99b75b61234e40289fc183ba92b187e8" modify_external_catalogs: true
W20260202 02:52:18.394709 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9dfc2d7b50e94228b7aad2a3f575f625 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:18.395638 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79f694b740954d429145fba9157338ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.395663 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b8ba9dd450948ada395207ae1c5b325. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.396397 1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20ea39bc5d3d46f5a4ba427277d6c6a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.397223 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b8ba9dd450948ada395207ae1c5b325. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.399757 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet 267bd4752c5f4b99a3fe778ba8f38a2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:18.401051 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet 0b411e8fab074aa88f8380e3f5694340 with delete_type TABLET_DATA_DELETED (replaced by tablet 28a900bcaa8b41718a285e595886d805) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:18.401695 750 tablet_service.cc:1505] Processing CreateTablet for tablet 79f694b740954d429145fba9157338ca (DEFAULT_TABLE table=default.Table_662111850b514df5ad7c69aa21783397 [id=d595db36d702401ab81d5401ad3a76e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.401993 754 tablet_service.cc:1505] Processing CreateTablet for tablet 91ec01d25f8f441cbe3783cacbee51db (DEFAULT_TABLE table=default.Table_b18a63e4731b418abcb84c36082553c1 [id=255f4d94812f45d0b4423150098460ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.402215 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79f694b740954d429145fba9157338ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.402411 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91ec01d25f8f441cbe3783cacbee51db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.402377 1338 raft_consensus.cc:493] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:18.402532 1338 raft_consensus.cc:515] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.403229 1338 leader_election.cc:290] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:18.408030 1338 raft_consensus.cc:493] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:18.408360 1338 raft_consensus.cc:515] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.409024 1338 leader_election.cc:290] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:18.413156 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "761f5a51335a4de5977f5f7fef0d4260" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:18.413765 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f93e8d3c4faf48eaa98c3cbf88d585e2" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:18.414209 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "267bd4752c5f4b99a3fe778ba8f38a2f" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:18.414705 715 leader_election.cc:343] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:18.414791 528 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_ab8d74f30cd14c61b85950a82d0f3787 [id=3ccb047c30974c0f9cbe778971e61644] alter complete (version 3)
W20260202 02:52:18.414970 715 leader_election.cc:343] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:18.416630 1182 ts_tablet_manager.cc:1929] T f58a7b25815c4b5bac568aa11c19610c P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:18.419147 1174 tablet_bootstrap.cc:654] T c064ea0dade840b4840a5715c2d83f69 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:18.421165 1182 log.cc:1199] T f58a7b25815c4b5bac568aa11c19610c P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/f58a7b25815c4b5bac568aa11c19610c
I20260202 02:52:18.422515 1040 tablet_service.cc:1552] Processing DeleteTablet for tablet eaa47f3388144e8b852804c167f232ce with delete_type TABLET_DATA_DELETED (replaced by tablet 38fdbc6df7ec40b9afe4e0dfe0119e3e) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:18.422992 1182 ts_tablet_manager.cc:1950] T f58a7b25815c4b5bac568aa11c19610c P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
W20260202 02:52:18.421332 715 consensus_peers.cc:597] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:18.427161 1182 tablet_replica.cc:333] T bc1539e5c8aa4d6e98f687728b891530 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
W20260202 02:52:18.428764 524 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 63a9029bc39c4fbd8ac4bf4bc37d6b4e consensus state has no leader
I20260202 02:52:18.429293 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_2a4b8e3cee324b6da94baaeaba0c5f26"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:18.429955 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2a4b8e3cee324b6da94baaeaba0c5f26 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:18.430742 521 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet a67521711a994af5938a9ebd9fe5cb02
I20260202 02:52:18.431246 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_b6e7efe67b0e4efbb7e7c51ee5f1caa7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:18.431926 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b6e7efe67b0e4efbb7e7c51ee5f1caa7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:18.432046 1182 raft_consensus.cc:2243] T bc1539e5c8aa4d6e98f687728b891530 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:18.432054 755 tablet_service.cc:1505] Processing CreateTablet for tablet 20ea39bc5d3d46f5a4ba427277d6c6a3 (DEFAULT_TABLE table=default.Table_3b335a0806154f98a907f551596ba353 [id=0da4db61b3fe4cf4846edd5530b9327f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.432267 1182 raft_consensus.cc:2272] T bc1539e5c8aa4d6e98f687728b891530 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:18.432512 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20ea39bc5d3d46f5a4ba427277d6c6a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.432862 1182 ts_tablet_manager.cc:1916] T bc1539e5c8aa4d6e98f687728b891530 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:18.434048 1321 raft_consensus.cc:493] T bf5ca99708044482970cb854d2eced18 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:18.434245 1321 raft_consensus.cc:515] T bf5ca99708044482970cb854d2eced18 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.435372 1321 leader_election.cc:290] T bf5ca99708044482970cb854d2eced18 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:18.417384 857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 57104.4us
I20260202 02:52:18.435511 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf5ca99708044482970cb854d2eced18" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:18.436216 715 leader_election.cc:343] T bf5ca99708044482970cb854d2eced18 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:18.430749 1175 tablet_bootstrap.cc:492] T 526f2c7b95184a0987e7aa94efc8475a P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:18.442181 1539 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.188:39077 every 8 connection(s)
I20260202 02:52:18.442548 1246 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.188:39077
I20260202 02:52:18.444608 857 leader_election.cc:304] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:18.446710 1246 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-2/data/info.pb
I20260202 02:52:18.449059 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskYXxklX/build/asan/bin/kudu as pid 1246
W20260202 02:52:18.450537 523 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c12ff20ca6b44ddba0c936a28cfb2eb9 consensus state has no leader
I20260202 02:52:18.452628 525 master_service.cc:954] ReplaceTablet: received request to replace tablet 0186572e7bb9499a932a636ab280f544 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:18.454094 1175 ts_tablet_manager.cc:1403] T 526f2c7b95184a0987e7aa94efc8475a P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.084s user 0.000s sys 0.004s
I20260202 02:52:18.454727 523 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_f92d6341db3143e490f6d64418ef6ff2" } modify_external_catalogs: true
I20260202 02:52:18.454932 523 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_f92d6341db3143e490f6d64418ef6ff2" } modify_external_catalogs: true
I20260202 02:52:18.455060 1175 raft_consensus.cc:359] T 526f2c7b95184a0987e7aa94efc8475a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:18.455266 1175 raft_consensus.cc:385] T 526f2c7b95184a0987e7aa94efc8475a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:18.455336 1175 raft_consensus.cc:740] T 526f2c7b95184a0987e7aa94efc8475a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:18.455585 1175 consensus_queue.cc:260] T 526f2c7b95184a0987e7aa94efc8475a P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:18.455835 857 leader_election.cc:304] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:18.456637 1338 raft_consensus.cc:2804] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:18.458119 857 leader_election.cc:304] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:18.458410 1521 raft_consensus.cc:2804] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:18.458717 1453 raft_consensus.cc:2804] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:18.467774 1173 tablet_bootstrap.cc:654] T c050a2882f764dda8df2a7df6547f149 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:18.478452 1175 ts_tablet_manager.cc:1434] T 526f2c7b95184a0987e7aa94efc8475a P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.024s user 0.001s sys 0.001s
I20260202 02:52:18.478965 1175 tablet_bootstrap.cc:492] T 0186572e7bb9499a932a636ab280f544 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:18.481266 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_7b5f2de229cb4b2f8bd174ccfd6b9553"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:18.483213 526 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_ce2d44550fb5403a99372bc5d3d79265 [id=4f19275f416a425c9b9d100b2a83870a] alter complete (version 2)
I20260202 02:52:18.484274 1540 sys_catalog.cc:263] Verifying existing consensus state
I20260202 02:52:18.487293 1183 ts_tablet_manager.cc:1929] T f24c0b99574d437a99b273204f105460 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:18.489177 1540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb: Bootstrap starting.
I20260202 02:52:18.490561 1183 log.cc:1199] T f24c0b99574d437a99b273204f105460 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/f24c0b99574d437a99b273204f105460
W20260202 02:52:18.491909 528 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet ff6f5710de4b4c48948aacdf5be29e8b (table default.Table_4720e764353243a495245b92e5b913fb [id=f2276e1c2ca04a70804053da61dd3447], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet ff6f5710de4b4c48948aacdf5be29e8b consensus state has no leader
I20260202 02:52:18.492369 750 tablet_service.cc:1552] Processing DeleteTablet for tablet 0b411e8fab074aa88f8380e3f5694340 with delete_type TABLET_DATA_DELETED (replaced by tablet 28a900bcaa8b41718a285e595886d805) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:18.492496 749 tablet_service.cc:1552] Processing DeleteTablet for tablet eaa47f3388144e8b852804c167f232ce with delete_type TABLET_DATA_DELETED (replaced by tablet 38fdbc6df7ec40b9afe4e0dfe0119e3e) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:18.492688 1180 tablet_replica.cc:333] T 0b411e8fab074aa88f8380e3f5694340 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:18.493778 1180 raft_consensus.cc:2243] T 0b411e8fab074aa88f8380e3f5694340 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
W20260202 02:52:18.493880 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7b5f2de229cb4b2f8bd174ccfd6b9553 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:18.494071 1180 raft_consensus.cc:2272] T 0b411e8fab074aa88f8380e3f5694340 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:18.494625 1183 ts_tablet_manager.cc:1950] T f24c0b99574d437a99b273204f105460 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:18.494648 1180 ts_tablet_manager.cc:1916] T 0b411e8fab074aa88f8380e3f5694340 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:18.495558 528 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_7ef6ef5b916e42709217b7c44fb489d0" } new_table_name: "default.Table_61aa40074c044f6d9802711543c14e20" modify_external_catalogs: true
I20260202 02:52:18.506538 752 tablet_service.cc:1552] Processing DeleteTablet for tablet 267bd4752c5f4b99a3fe778ba8f38a2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:18.506574 1540 log.cc:826] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb: Log is configured to *not* fsync() on all Append() calls
I20260202 02:52:18.513474 1180 ts_tablet_manager.cc:1929] T 0b411e8fab074aa88f8380e3f5694340 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:18.513568 527 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 251455168e33430aafea8e0de42ca1b8
I20260202 02:52:18.513728 1180 log.cc:1199] T 0b411e8fab074aa88f8380e3f5694340 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/0b411e8fab074aa88f8380e3f5694340
I20260202 02:52:18.514221 527 catalog_manager.cc:6724] ReplaceTablet: tablet 251455168e33430aafea8e0de42ca1b8 deleted and replaced by tablet fe53a8bdbd13417f9ce195c54a52a93c
I20260202 02:52:18.514501 755 tablet_service.cc:1552] Processing DeleteTablet for tablet 251455168e33430aafea8e0de42ca1b8 with delete_type TABLET_DATA_DELETED (replaced by tablet fe53a8bdbd13417f9ce195c54a52a93c) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:18.515190 1180 ts_tablet_manager.cc:1950] T 0b411e8fab074aa88f8380e3f5694340 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
W20260202 02:52:18.516115 519 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b428356fba0c4fb5a6033c5d39740cc5 consensus state has no leader
I20260202 02:52:18.516399 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 0b411e8fab074aa88f8380e3f5694340 (table default.Table_052de59e60de4473a2d13185440ab818 [id=cd07f9746ec94d9ebe956a7ca0d19b1c]) successfully deleted
W20260202 02:52:18.516794 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet eaa47f3388144e8b852804c167f232ce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet eaa47f3388144e8b852804c167f232ce already in progress: creating tablet
W20260202 02:52:18.517175 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 267bd4752c5f4b99a3fe778ba8f38a2f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 267bd4752c5f4b99a3fe778ba8f38a2f already in progress: creating tablet
I20260202 02:52:18.517333 1180 tablet_replica.cc:333] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:18.517969 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet 251455168e33430aafea8e0de42ca1b8 with delete_type TABLET_DATA_DELETED (replaced by tablet fe53a8bdbd13417f9ce195c54a52a93c) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:18.517757 1180 raft_consensus.cc:2243] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
W20260202 02:52:18.522631 859 leader_election.cc:343] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:18.525332 1182 ts_tablet_manager.cc:1929] T bc1539e5c8aa4d6e98f687728b891530 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:18.525593 1182 log.cc:1199] T bc1539e5c8aa4d6e98f687728b891530 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/bc1539e5c8aa4d6e98f687728b891530
I20260202 02:52:18.526033 1182 ts_tablet_manager.cc:1950] T bc1539e5c8aa4d6e98f687728b891530 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:18.526574 1182 tablet_replica.cc:333] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:18.526861 1182 raft_consensus.cc:2243] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:18.526937 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_6133c617ff8d40eca5cbde96533acd82"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:18.527686 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6133c617ff8d40eca5cbde96533acd82 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:18.530367 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet f24c0b99574d437a99b273204f105460 (table default.Table_f1e417bf19b94bc491c3937276c669fd [id=9a8a252de32848288c9b3752c7d3feb7]) successfully deleted
W20260202 02:52:18.530696 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet d67e91a022f1401ea724dff898c3b81b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d67e91a022f1401ea724dff898c3b81b already in progress: creating tablet
W20260202 02:52:18.530967 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 267bd4752c5f4b99a3fe778ba8f38a2f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 267bd4752c5f4b99a3fe778ba8f38a2f already in progress: creating tablet
I20260202 02:52:18.535179 522 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet ffd8aa2bc6e344b78dce518411997fb5
I20260202 02:52:18.536175 755 tablet_service.cc:1552] Processing DeleteTablet for tablet ffd8aa2bc6e344b78dce518411997fb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:18.536221 1180 raft_consensus.cc:2272] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:18.536926 1180 ts_tablet_manager.cc:1916] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:18.537595 1175 tablet_bootstrap.cc:654] T 0186572e7bb9499a932a636ab280f544 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:18.538472 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet ffd8aa2bc6e344b78dce518411997fb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:18.538739 1182 raft_consensus.cc:2272] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:18.539192 1533 tablet.cc:1722] T 251455168e33430aafea8e0de42ca1b8 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:18.543632 1183 tablet_replica.cc:333] T 251455168e33430aafea8e0de42ca1b8 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:18.545511 1183 raft_consensus.cc:2243] T 251455168e33430aafea8e0de42ca1b8 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:18.545531 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_84d6693620cc468e82d5cb1ec6cc69ea"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:18.545989 1183 raft_consensus.cc:2272] T 251455168e33430aafea8e0de42ca1b8 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:18.546491 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_84d6693620cc468e82d5cb1ec6cc69ea in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:18.547520 1453 raft_consensus.cc:697] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
W20260202 02:52:18.550575 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 0b411e8fab074aa88f8380e3f5694340 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0b411e8fab074aa88f8380e3f5694340 already in progress: creating tablet
W20260202 02:52:18.551220 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet eaa47f3388144e8b852804c167f232ce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet eaa47f3388144e8b852804c167f232ce already in progress: creating tablet
I20260202 02:52:18.551991 1338 raft_consensus.cc:697] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
W20260202 02:52:18.552501 857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 83891.5us
I20260202 02:52:18.553422 857 leader_election.cc:304] T d06b77ff1a6f4f78b73082b00145fcf6 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:18.554095 1548 raft_consensus.cc:2804] T d06b77ff1a6f4f78b73082b00145fcf6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:18.554260 1548 raft_consensus.cc:493] T d06b77ff1a6f4f78b73082b00145fcf6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:18.554445 1548 raft_consensus.cc:3060] T d06b77ff1a6f4f78b73082b00145fcf6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:18.555804 1277 raft_consensus.cc:697] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:18.556847 1277 consensus_queue.cc:237] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.558854 1173 tablet_bootstrap.cc:492] T c050a2882f764dda8df2a7df6547f149 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:18.559048 1173 ts_tablet_manager.cc:1403] T c050a2882f764dda8df2a7df6547f149 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.184s user 0.005s sys 0.007s
I20260202 02:52:18.558313 714 leader_election.cc:304] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:18.559863 1173 raft_consensus.cc:359] T c050a2882f764dda8df2a7df6547f149 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.560055 1173 raft_consensus.cc:385] T c050a2882f764dda8df2a7df6547f149 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:18.560137 1173 raft_consensus.cc:740] T c050a2882f764dda8df2a7df6547f149 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:18.561523 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_2107cef8ed314cad9f2c7c115e2baedc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:18.562142 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2107cef8ed314cad9f2c7c115e2baedc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:18.563234 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bc1539e5c8aa4d6e98f687728b891530" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:18.563490 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "78f122342dd144b48de89902faa0f94f" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:18.564051 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_1a25d92824e340579a0b522dab94fce6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:18.564563 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b76d1837813c46e8a0925b49939d8884" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
W20260202 02:52:18.564705 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1a25d92824e340579a0b522dab94fce6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:18.564756 779 raft_consensus.cc:3060] T b76d1837813c46e8a0925b49939d8884 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:18.567405 779 raft_consensus.cc:2468] T b76d1837813c46e8a0925b49939d8884 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
I20260202 02:52:18.568905 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f93e8d3c4faf48eaa98c3cbf88d585e2" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:18.570147 1174 tablet_bootstrap.cc:492] T c064ea0dade840b4840a5715c2d83f69 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:18.570369 1174 ts_tablet_manager.cc:1403] T c064ea0dade840b4840a5715c2d83f69 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.220s user 0.010s sys 0.006s
I20260202 02:52:18.571225 1174 raft_consensus.cc:359] T c064ea0dade840b4840a5715c2d83f69 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.571520 1174 raft_consensus.cc:385] T c064ea0dade840b4840a5715c2d83f69 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:18.571636 1174 raft_consensus.cc:740] T c064ea0dade840b4840a5715c2d83f69 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:18.571985 1174 consensus_queue.cc:260] T c064ea0dade840b4840a5715c2d83f69 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.572365 1324 raft_consensus.cc:2804] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:18.572492 1324 raft_consensus.cc:697] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:18.572755 1324 consensus_queue.cc:237] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.571866 1173 consensus_queue.cc:260] T c050a2882f764dda8df2a7df6547f149 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.579429 520 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_0eb450e3edbb4a4ba0906567f831830c" } modify_external_catalogs: true
I20260202 02:52:18.579658 520 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_0eb450e3edbb4a4ba0906567f831830c" } modify_external_catalogs: true
I20260202 02:52:18.583299 1173 ts_tablet_manager.cc:1434] T c050a2882f764dda8df2a7df6547f149 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.024s user 0.001s sys 0.001s
I20260202 02:52:18.583709 1173 tablet_bootstrap.cc:492] T 97bd7223d41d40a6bcdc60b1f63abc67 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
W20260202 02:52:18.584244 859 leader_election.cc:343] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:18.548624 1453 consensus_queue.cc:237] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.587702 1533 tablet.cc:1722] T 251455168e33430aafea8e0de42ca1b8 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260202 02:52:18.587769 859 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 55885.1us
W20260202 02:52:18.587949 528 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bd7b0a0fcb7645fdb9edfeccb3d18268 consensus state has no leader
I20260202 02:52:18.589608 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "267bd4752c5f4b99a3fe778ba8f38a2f" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:18.591346 755 tablet_service.cc:1552] Processing DeleteTablet for tablet bf5ca99708044482970cb854d2eced18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:18.591581 752 tablet_service.cc:1552] Processing DeleteTablet for tablet ea040c03420c4b1b98897f39160b4130 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:18.591719 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 4f2a9404f7424dc78b5bd062ebd8cb53 with delete_type TABLET_DATA_DELETED (replaced by tablet 526f2c7b95184a0987e7aa94efc8475a) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:18.593086 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 560465c063d94ffaab33f9f79a58a357 with delete_type TABLET_DATA_DELETED (replaced by tablet 20ea39bc5d3d46f5a4ba427277d6c6a3) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:18.593452 754 tablet_service.cc:1552] Processing DeleteTablet for tablet cb0dcacdf0f1419da0b25f20cac1d62d with delete_type TABLET_DATA_DELETED (replaced by tablet d67e91a022f1401ea724dff898c3b81b) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:18.593784 754 tablet_service.cc:1552] Processing DeleteTablet for tablet aa526a69cf914dd08142074636c956b7 with delete_type TABLET_DATA_DELETED (replaced by tablet c184d44d7f434b39a997a7387e5d3de0) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:18.593856 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 23c38ef4e3c840808ec36f859cf6f6de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:18.593971 753 tablet_service.cc:1552] Processing DeleteTablet for tablet 724d47241f0342768fb1321ef37c3a7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:16 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:18.594122 750 tablet_service.cc:1552] Processing DeleteTablet for tablet fae7e3941c124cc2871390a0b6471f27 with delete_type TABLET_DATA_DELETED (replaced by tablet a5622ef925bc46cca76f481ddfd12afd) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:18.594334 758 tablet_service.cc:1552] Processing DeleteTablet for tablet 952d760159844ad08269654264e6381b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:18.594141 759 tablet_service.cc:1552] Processing DeleteTablet for tablet bf74f96bea2e4ab7adec42c3c5207e79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:16 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:18.594604 752 tablet_service.cc:1552] Processing DeleteTablet for tablet 0d297b6243c348ee82c146a8339cf6c4 with delete_type TABLET_DATA_DELETED (replaced by tablet d64f42d187bc4c9f9381c931449ec0cc) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:18.594831 755 tablet_service.cc:1552] Processing DeleteTablet for tablet c1fc01252d95407e954cfcbf7e715912 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:18.595422 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet f58a7b25815c4b5bac568aa11c19610c (table default.Table_983bf3e0c5424c2588a0e754af1a5687 [id=c97e407e519d4a538860f66931f6aefd]) successfully deleted
W20260202 02:52:18.595700 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 75b97d7e848241cf8d894b0ae92a8f8f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 75b97d7e848241cf8d894b0ae92a8f8f already in progress: creating tablet
I20260202 02:52:18.596833 755 tablet_service.cc:1552] Processing DeleteTablet for tablet 415d14533fd54f20adb00455a7bcad5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:16 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:18.598412 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet bc1539e5c8aa4d6e98f687728b891530 (table default.Table_f1e417bf19b94bc491c3937276c669fd [id=9a8a252de32848288c9b3752c7d3feb7]) successfully deleted
I20260202 02:52:18.599231 526 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_95c7a47b60374ca2959b0feb63df471f" } new_table_name: "default.Table_a22f252ee8d44be5a9d7202afb726cfc" modify_external_catalogs: true
I20260202 02:52:18.601606 528 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_04337834a3ce43ae8f6012b818d70d9f" } modify_external_catalogs: true
I20260202 02:52:18.601918 528 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_04337834a3ce43ae8f6012b818d70d9f" } modify_external_catalogs: true
I20260202 02:52:18.605559 1173 tablet_bootstrap.cc:654] T 97bd7223d41d40a6bcdc60b1f63abc67 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:18.607470 1521 raft_consensus.cc:697] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:18.607242 1338 consensus_queue.cc:237] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.614599 1548 raft_consensus.cc:515] T d06b77ff1a6f4f78b73082b00145fcf6 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.616238 1180 ts_tablet_manager.cc:1929] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260202 02:52:18.616523 1180 log.cc:1199] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/251455168e33430aafea8e0de42ca1b8
I20260202 02:52:18.617060 1180 ts_tablet_manager.cc:1950] T 251455168e33430aafea8e0de42ca1b8 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:18.618121 527 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet d2d27ceea1bb4122b18f0fcf4d2d1bf4 (table default.Table_10aa0dafec0547069eb92ecf12b3c780 [id=1f9c48c8e2484856a66a0719268f22ae]). Expected version 1 got 0
I20260202 02:52:18.618217 1180 tablet_replica.cc:333] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:18.618336 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 251455168e33430aafea8e0de42ca1b8 (table default.Table_ce2d44550fb5403a99372bc5d3d79265 [id=4f19275f416a425c9b9d100b2a83870a]) successfully deleted
I20260202 02:52:18.618398 527 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet b428356fba0c4fb5a6033c5d39740cc5 (table default.Table_99b75b61234e40289fc183ba92b187e8 [id=e87761e6b64a4476a05fcbcf73f0a2f1]). Expected version 1 got 0
I20260202 02:52:18.618405 1180 raft_consensus.cc:2243] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:18.618770 1180 raft_consensus.cc:2272] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:18.619072 527 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 78f122342dd144b48de89902faa0f94f (table default.Table_5eefa56618e84f8d92f8d87132ccd7d4 [id=b4ab2ba86e1a49338a98d855ded99f5d]). Expected version 1 got 0
I20260202 02:52:18.619367 1109 heartbeater.cc:344] Connected to a master server at 127.0.104.188:39077
I20260202 02:52:18.620054 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d06b77ff1a6f4f78b73082b00145fcf6" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
W20260202 02:52:18.624301 715 consensus_peers.cc:597] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:18.624786 755 tablet_service.cc:1505] Processing CreateTablet for tablet c25e329023c04f609d8a743b4f7a95a4 (DEFAULT_TABLE table=default.Table_52fba283eb604f81acbbc5ea3762d1f9 [id=81e199e091b149a0a103eb3b8a232983]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.625069 1180 ts_tablet_manager.cc:1916] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:18.625560 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c25e329023c04f609d8a743b4f7a95a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.626263 752 tablet_service.cc:1505] Processing CreateTablet for tablet 68823eff1bc04108ba83487605aec0ea (DEFAULT_TABLE table=default.Table_d4ad0ff5676e4af2a187497eda81d7a2 [id=0769ae34ed3d479a9f7010b34c35a110]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.626689 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68823eff1bc04108ba83487605aec0ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.627521 1543 raft_consensus.cc:493] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:18.635742 755 tablet_service.cc:1505] Processing CreateTablet for tablet 8774c6bc0fe9414ea2f17ed859d9392a (DEFAULT_TABLE table=default.Table_0eb450e3edbb4a4ba0906567f831830c [id=8918ec9e2cdf4f2995b2b4baf21297d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.636282 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8774c6bc0fe9414ea2f17ed859d9392a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.637336 1173 tablet_bootstrap.cc:492] T 97bd7223d41d40a6bcdc60b1f63abc67 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:18.637605 1173 ts_tablet_manager.cc:1403] T 97bd7223d41d40a6bcdc60b1f63abc67 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.054s user 0.000s sys 0.004s
I20260202 02:52:18.638490 1173 raft_consensus.cc:359] T 97bd7223d41d40a6bcdc60b1f63abc67 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.638829 1173 raft_consensus.cc:385] T 97bd7223d41d40a6bcdc60b1f63abc67 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:18.638963 1173 raft_consensus.cc:740] T 97bd7223d41d40a6bcdc60b1f63abc67 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:18.639288 1173 consensus_queue.cc:260] T 97bd7223d41d40a6bcdc60b1f63abc67 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.640990 1182 ts_tablet_manager.cc:1916] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:18.625701 1521 consensus_queue.cc:237] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.642966 1042 tablet_service.cc:1505] Processing CreateTablet for tablet c25e329023c04f609d8a743b4f7a95a4 (DEFAULT_TABLE table=default.Table_52fba283eb604f81acbbc5ea3762d1f9 [id=81e199e091b149a0a103eb3b8a232983]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.643435 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c25e329023c04f609d8a743b4f7a95a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.646286 1173 ts_tablet_manager.cc:1434] T 97bd7223d41d40a6bcdc60b1f63abc67 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260202 02:52:18.646435 1036 tablet_service.cc:1505] Processing CreateTablet for tablet 68823eff1bc04108ba83487605aec0ea (DEFAULT_TABLE table=default.Table_d4ad0ff5676e4af2a187497eda81d7a2 [id=0769ae34ed3d479a9f7010b34c35a110]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.646869 1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68823eff1bc04108ba83487605aec0ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.647388 1173 tablet_bootstrap.cc:492] T c0937b6910534ddc8a2217f0cbd539c5 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:18.649179 1173 tablet_bootstrap.cc:654] T c0937b6910534ddc8a2217f0cbd539c5 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:18.649899 752 tablet_service.cc:1505] Processing CreateTablet for tablet 042d1ebf357b4abe8814a407891ce265 (DEFAULT_TABLE table=default.Table_fe06d3b0b66447e29403a54ab14c476c [id=cab83305ba6648e5b307441e71671a82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.650332 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 042d1ebf357b4abe8814a407891ce265. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.650959 1174 ts_tablet_manager.cc:1434] T c064ea0dade840b4840a5715c2d83f69 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.080s user 0.001s sys 0.001s
I20260202 02:52:18.651422 1178 raft_consensus.cc:493] T ffd8aa2bc6e344b78dce518411997fb5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:18.651587 1178 raft_consensus.cc:515] T ffd8aa2bc6e344b78dce518411997fb5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:18.652036 824 heartbeater.cc:344] Connected to a master server at 127.0.104.188:39077
I20260202 02:52:18.652287 1178 leader_election.cc:290] T ffd8aa2bc6e344b78dce518411997fb5 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:18.652654 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 8774c6bc0fe9414ea2f17ed859d9392a (DEFAULT_TABLE table=default.Table_0eb450e3edbb4a4ba0906567f831830c [id=8918ec9e2cdf4f2995b2b4baf21297d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.653074 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8774c6bc0fe9414ea2f17ed859d9392a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.653208 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ffd8aa2bc6e344b78dce518411997fb5" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:18.653780 998 leader_election.cc:343] T ffd8aa2bc6e344b78dce518411997fb5 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: Tablet not RUNNING: STOPPED
I20260202 02:52:18.656569 1174 tablet_bootstrap.cc:492] T bc1539e5c8aa4d6e98f687728b891530 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:18.657255 1183 ts_tablet_manager.cc:1916] T 251455168e33430aafea8e0de42ca1b8 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:18.658257 1174 tablet_bootstrap.cc:654] T bc1539e5c8aa4d6e98f687728b891530 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:18.660810 1034 tablet_service.cc:1505] Processing CreateTablet for tablet 042d1ebf357b4abe8814a407891ce265 (DEFAULT_TABLE table=default.Table_fe06d3b0b66447e29403a54ab14c476c [id=cab83305ba6648e5b307441e71671a82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.661265 1173 tablet_bootstrap.cc:492] T c0937b6910534ddc8a2217f0cbd539c5 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:18.661408 1173 ts_tablet_manager.cc:1403] T c0937b6910534ddc8a2217f0cbd539c5 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.000s
I20260202 02:52:18.662231 1173 raft_consensus.cc:359] T c0937b6910534ddc8a2217f0cbd539c5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.662441 1173 raft_consensus.cc:385] T c0937b6910534ddc8a2217f0cbd539c5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:18.662534 1173 raft_consensus.cc:740] T c0937b6910534ddc8a2217f0cbd539c5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:18.666580 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_f93d759396c04d25ac602a609ecc4962"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:18.667117 1173 consensus_queue.cc:260] T c0937b6910534ddc8a2217f0cbd539c5 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:18.667912 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f93d759396c04d25ac602a609ecc4962 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:18.669289 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 042d1ebf357b4abe8814a407891ce265. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.671046 519 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet c63a670d5c5e489897bc6dbb142b844d (table default.Table_12d68a317c8e4d809dbbe77c095a1f6e [id=7219522ffc3849faab94d754a42a8511]). Expected version 1 got 0
I20260202 02:52:18.671236 519 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet 723921dfe54c4e66af5cab76a25fee0a (table default.Table_bdfbf5f796b949e196b1c1609fbb3655 [id=c30593053ed7490588ff567e30984807]). Expected version 1 got 0
I20260202 02:52:18.671995 519 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet b76d1837813c46e8a0925b49939d8884 (table default.Table_3919d090261e4b90b0672f2b863588b2 [id=fe7b88b74c864c2f98365c0956117bf1]). Expected version 1 got 0
I20260202 02:52:18.672197 519 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet d06b77ff1a6f4f78b73082b00145fcf6 (table default.Table_c09bacb95cba44a3ad80c21c0a77b6e0 [id=915b211258ad498286448a5b2598a289]). Expected version 2 got 0
I20260202 02:52:18.672250 1173 ts_tablet_manager.cc:1434] T c0937b6910534ddc8a2217f0cbd539c5 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20260202 02:52:18.672597 1173 tablet_bootstrap.cc:492] T 61a9203035f2495f8f99924303ebf120 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:18.674153 1174 tablet_bootstrap.cc:492] T bc1539e5c8aa4d6e98f687728b891530 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:18.676266 1183 ts_tablet_manager.cc:1929] T 251455168e33430aafea8e0de42ca1b8 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260202 02:52:18.676855 1174 ts_tablet_manager.cc:1403] T bc1539e5c8aa4d6e98f687728b891530 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.000s
I20260202 02:52:18.677752 1174 raft_consensus.cc:359] T bc1539e5c8aa4d6e98f687728b891530 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:18.677986 1174 raft_consensus.cc:385] T bc1539e5c8aa4d6e98f687728b891530 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:18.678083 1174 raft_consensus.cc:740] T bc1539e5c8aa4d6e98f687728b891530 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:18.680568 519 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet 583e547d0a1943f09f4efe82070fc777 (table default.Table_2669ca9662b942c08bcbfd7641cfe2e3 [id=fb813e4c5b0849f8a488c7a22d1defae]). Expected version 1 got 0
I20260202 02:52:18.681185 1174 consensus_queue.cc:260] T bc1539e5c8aa4d6e98f687728b891530 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:18.681201 519 catalog_manager.cc:5654] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f442ef85029461091de847118c34c0d (127.0.104.129). New cstate: current_term: 1 leader_uuid: "1f442ef85029461091de847118c34c0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } }
I20260202 02:52:18.681890 519 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet c12ff20ca6b44ddba0c936a28cfb2eb9 (table default.Table_959d3e3da2814b3a986d29864e47246c [id=bdf2f19a439d4be3aa2362cfde38ab7f]). Expected version 2 got 0
I20260202 02:52:18.682263 519 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet d2d27ceea1bb4122b18f0fcf4d2d1bf4 (table default.Table_10aa0dafec0547069eb92ecf12b3c780 [id=1f9c48c8e2484856a66a0719268f22ae]). Expected version 1 got 0
I20260202 02:52:18.682600 519 catalog_manager.cc:5654] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f442ef85029461091de847118c34c0d (127.0.104.129). New cstate: current_term: 1 leader_uuid: "1f442ef85029461091de847118c34c0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: HEALTHY } } }
I20260202 02:52:18.683151 519 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet bd7b0a0fcb7645fdb9edfeccb3d18268 (table default.Table_61aa40074c044f6d9802711543c14e20 [id=1ffcc11561524fe8a7b09a7ddc92d5b2]). Expected version 1 got 0
I20260202 02:52:18.684160 1548 leader_election.cc:290] T d06b77ff1a6f4f78b73082b00145fcf6 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:18.684403 1548 raft_consensus.cc:493] T 526f2c7b95184a0987e7aa94efc8475a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:18.683852 519 catalog_manager.cc:5654] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f442ef85029461091de847118c34c0d (127.0.104.129). New cstate: current_term: 1 leader_uuid: "1f442ef85029461091de847118c34c0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } }
I20260202 02:52:18.684929 519 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet 63a9029bc39c4fbd8ac4bf4bc37d6b4e (table default.Table_90344989f4234a11a30653bdc34fe899 [id=fa95e04db2d443b5972de98c7c0e6457]). Expected version 3 got 0
I20260202 02:52:18.692759 1548 raft_consensus.cc:515] T 526f2c7b95184a0987e7aa94efc8475a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:18.693495 1548 leader_election.cc:290] T 526f2c7b95184a0987e7aa94efc8475a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:18.695140 1174 ts_tablet_manager.cc:1434] T bc1539e5c8aa4d6e98f687728b891530 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.018s user 0.001s sys 0.001s
I20260202 02:52:18.695582 1544 raft_consensus.cc:493] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260202 02:52:18.695853 859 leader_election.cc:343] T d06b77ff1a6f4f78b73082b00145fcf6 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:18.695829 918 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bbf31e5619ed4f60b4d7c8b19ec8a549" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
W20260202 02:52:18.695989 859 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 75340.4us
I20260202 02:52:18.696099 759 tablet_service.cc:1505] Processing CreateTablet for tablet af5aa68e47bb420eac300754d274fa73 (DEFAULT_TABLE table=default.Table_91411bac2a2a45a395b81a4dab2617f7 [id=feb8b9f926434cec8b119843413f8c9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.696555 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af5aa68e47bb420eac300754d274fa73. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:18.699745 513 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet ff6f5710de4b4c48948aacdf5be29e8b (table default.Table_4720e764353243a495245b92e5b913fb [id=f2276e1c2ca04a70804053da61dd3447], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet ff6f5710de4b4c48948aacdf5be29e8b consensus state has no leader
I20260202 02:52:18.702857 1174 tablet_bootstrap.cc:492] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:18.709348 1544 raft_consensus.cc:515] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.710386 1174 tablet_bootstrap.cc:654] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:18.641965 1543 raft_consensus.cc:515] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.718716 921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "761f5a51335a4de5977f5f7fef0d4260" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:18.718950 921 raft_consensus.cc:3060] T 761f5a51335a4de5977f5f7fef0d4260 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:18.724823 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bbf31e5619ed4f60b4d7c8b19ec8a549" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:18.725013 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9884c90cf1d245b2a7eaaf00d71b1c3e" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:18.723260 1544 leader_election.cc:290] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:18.724229 1543 leader_election.cc:290] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:18.725395 918 raft_consensus.cc:2468] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
W20260202 02:52:18.725494 859 leader_election.cc:343] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:18.726285 859 leader_election.cc:343] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:18.722544 921 raft_consensus.cc:2468] T 761f5a51335a4de5977f5f7fef0d4260 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:18.726843 1043 tablet_service.cc:1505] Processing CreateTablet for tablet 28a900bcaa8b41718a285e595886d805 (DEFAULT_TABLE table=default.Table_052de59e60de4473a2d13185440ab818 [id=cd07f9746ec94d9ebe956a7ca0d19b1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.728691 1039 tablet_service.cc:1505] Processing CreateTablet for tablet 08d8d781bb4e4acf915f11afb4fb2e22 (DEFAULT_TABLE table=default.Table_2acd399bc33146fe9f1b6e2a15e0b101 [id=487a204c7845401ab7391ae8464fdf1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.731434 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 97bd7223d41d40a6bcdc60b1f63abc67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:18.731642 1183 log.cc:1199] T 251455168e33430aafea8e0de42ca1b8 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/251455168e33430aafea8e0de42ca1b8
I20260202 02:52:18.731689 1059 raft_consensus.cc:3060] T c0937b6910534ddc8a2217f0cbd539c5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:18.732159 1183 ts_tablet_manager.cc:1950] T 251455168e33430aafea8e0de42ca1b8 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:18.733546 1183 tablet_replica.cc:333] T ffd8aa2bc6e344b78dce518411997fb5 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:18.733867 1183 raft_consensus.cc:2243] T ffd8aa2bc6e344b78dce518411997fb5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:18.734205 1183 raft_consensus.cc:2272] T ffd8aa2bc6e344b78dce518411997fb5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:18.734589 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "526f2c7b95184a0987e7aa94efc8475a" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:18.734241 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 38301e13e17c41fcae912c4b38c26a45 (DEFAULT_TABLE table=default.Table_80912cc76f934f139da7f866546ae97a [id=2f3bf7e0cf4c41ac8a2554360b57214a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.735219 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28a900bcaa8b41718a285e595886d805. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.735469 1183 ts_tablet_manager.cc:1916] T ffd8aa2bc6e344b78dce518411997fb5 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:18.735921 1036 tablet_service.cc:1505] Processing CreateTablet for tablet 38fdbc6df7ec40b9afe4e0dfe0119e3e (DEFAULT_TABLE table=default.Table_87277c5d80184738b96b773f026b32a5 [id=52687dbd0e554cfb81d88e8fcdd4dd5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.735802 1034 tablet_service.cc:1505] Processing CreateTablet for tablet af5aa68e47bb420eac300754d274fa73 (DEFAULT_TABLE table=default.Table_91411bac2a2a45a395b81a4dab2617f7 [id=feb8b9f926434cec8b119843413f8c9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.736444 1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38fdbc6df7ec40b9afe4e0dfe0119e3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.736656 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af5aa68e47bb420eac300754d274fa73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.736863 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38301e13e17c41fcae912c4b38c26a45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.740141 1035 tablet_service.cc:1505] Processing CreateTablet for tablet cb634d4bae554ef3b39eeaa9958adc6c (DEFAULT_TABLE table=default.Table_983bf3e0c5424c2588a0e754af1a5687 [id=c97e407e519d4a538860f66931f6aefd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.740688 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb634d4bae554ef3b39eeaa9958adc6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.741897 920 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ff6f5710de4b4c48948aacdf5be29e8b" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:18.742456 901 tablet_service.cc:1505] Processing CreateTablet for tablet a5622ef925bc46cca76f481ddfd12afd (DEFAULT_TABLE table=default.Table_3d26f7a34d0c48379e3cebda66e1ef13 [id=45e6d83589fa493195b06f633e295b00]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:18.742504 859 leader_election.cc:343] T 526f2c7b95184a0987e7aa94efc8475a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:18.742717 920 raft_consensus.cc:3060] T ff6f5710de4b4c48948aacdf5be29e8b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:18.742897 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5622ef925bc46cca76f481ddfd12afd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.744499 1040 tablet_service.cc:1505] Processing CreateTablet for tablet a1f39e5b02f747819e636cddb4292eb6 (DEFAULT_TABLE table=default.Table_92b7f492c09948f784c64ccdcaadbb86 [id=abceac85d3524c96834546f67f41f933]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.744815 1038 tablet_service.cc:1505] Processing CreateTablet for tablet 9c1271ce0f194021b588b82ac9d3db79 (DEFAULT_TABLE table=default.Table_733044f900d44218a06c976c8701f122 [id=9f912bc02bec49a8b79e4e7e921595b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.744936 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1f39e5b02f747819e636cddb4292eb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.745215 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c1271ce0f194021b588b82ac9d3db79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.745586 920 raft_consensus.cc:2468] T ff6f5710de4b4c48948aacdf5be29e8b P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:18.745772 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08d8d781bb4e4acf915f11afb4fb2e22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.746050 1175 tablet_bootstrap.cc:492] T 0186572e7bb9499a932a636ab280f544 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:18.749182 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 251455168e33430aafea8e0de42ca1b8 (table default.Table_ce2d44550fb5403a99372bc5d3d79265 [id=4f19275f416a425c9b9d100b2a83870a]) successfully deleted
I20260202 02:52:18.753798 521 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_61aa40074c044f6d9802711543c14e20" } new_table_name: "default.Table_18fec323d6b04e10a687de6361b643b2" modify_external_catalogs: true
I20260202 02:52:18.754817 1540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 2 replicates
I20260202 02:52:18.756068 1540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb: Bootstrap complete.
I20260202 02:52:18.757669 755 tablet_service.cc:1505] Processing CreateTablet for tablet 28a900bcaa8b41718a285e595886d805 (DEFAULT_TABLE table=default.Table_052de59e60de4473a2d13185440ab818 [id=cd07f9746ec94d9ebe956a7ca0d19b1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.758131 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28a900bcaa8b41718a285e595886d805. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.758253 1175 ts_tablet_manager.cc:1403] T 0186572e7bb9499a932a636ab280f544 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.267s user 0.007s sys 0.003s
I20260202 02:52:18.759020 898 tablet_service.cc:1505] Processing CreateTablet for tablet 2b8ba9dd450948ada395207ae1c5b325 (DEFAULT_TABLE table=default.Table_614c0ff5900b4d64b9c0af8c88a33ecb [id=29f81d15b06d4f8abe2aba83493cfa6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.759733 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b8ba9dd450948ada395207ae1c5b325. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.760805 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 13cd1383348b458894fdecc22bf24b74 (DEFAULT_TABLE table=default.Table_f92d6341db3143e490f6d64418ef6ff2 [id=ebc82aae55584b8ba2ba5c4223d9e258]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.770504 1540 raft_consensus.cc:359] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [term 2 FOLLOWER]: Replica starting. Triggering 2 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } }
W20260202 02:52:18.774051 526 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1f54c1dfe0a942b3bafd85522953a032 consensus state has no leader
I20260202 02:52:18.774498 526 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_bdfbf5f796b949e196b1c1609fbb3655" } modify_external_catalogs: true
I20260202 02:52:18.774693 526 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_bdfbf5f796b949e196b1c1609fbb3655" } modify_external_catalogs: true
I20260202 02:52:18.775290 525 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 0186572e7bb9499a932a636ab280f544
I20260202 02:52:18.775804 525 catalog_manager.cc:6724] ReplaceTablet: tablet 0186572e7bb9499a932a636ab280f544 deleted and replaced by tablet 79c73f4a92044bafb5b8384229cb3a4d
I20260202 02:52:18.776190 525 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_4720e764353243a495245b92e5b913fb" } new_table_name: "default.Table_ad048ed673a5401b8ffa739c0f03b545" modify_external_catalogs: true
I20260202 02:52:18.777019 759 tablet_service.cc:1505] Processing CreateTablet for tablet cb634d4bae554ef3b39eeaa9958adc6c (DEFAULT_TABLE table=default.Table_983bf3e0c5424c2588a0e754af1a5687 [id=c97e407e519d4a538860f66931f6aefd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.777503 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb634d4bae554ef3b39eeaa9958adc6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.778088 523 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 13cd1383348b458894fdecc22bf24b74
I20260202 02:52:18.778949 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_b13428495dde400e83de7d1032559c35"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:18.779601 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b13428495dde400e83de7d1032559c35 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:18.780519 520 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 8774c6bc0fe9414ea2f17ed859d9392a
I20260202 02:52:18.780828 1173 tablet_bootstrap.cc:654] T 61a9203035f2495f8f99924303ebf120 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:18.781275 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_6da292362a6c43738dbeb895750b918a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:18.781817 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6da292362a6c43738dbeb895750b918a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:18.781786 1540 raft_consensus.cc:740] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c3e3612bb94fb7b94c308041acfaeb, State: Initialized, Role: FOLLOWER
I20260202 02:52:18.782889 1540 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } }
I20260202 02:52:18.783674 528 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 2f0e8c5e2fd44370ae150a206f4e6cbc
I20260202 02:52:18.784473 528 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_2a4b8e3cee324b6da94baaeaba0c5f26" } new_table_name: "default.Table_82c14c20a7bf41b487efaff52ebc15a9" modify_external_catalogs: true
I20260202 02:52:18.786883 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13cd1383348b458894fdecc22bf24b74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.790421 1540 sys_catalog.cc:565] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [sys.catalog]: configured and running, proceeding with master startup.
I20260202 02:52:18.791922 1042 tablet_service.cc:1552] Processing DeleteTablet for tablet 13cd1383348b458894fdecc22bf24b74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:18.792135 1570 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } } }
I20260202 02:52:18.792594 1570 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:52:18.799089 1042 tablet_service.cc:1552] Processing DeleteTablet for tablet 8774c6bc0fe9414ea2f17ed859d9392a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:18.799151 896 tablet_service.cc:1505] Processing CreateTablet for tablet 20ea39bc5d3d46f5a4ba427277d6c6a3 (DEFAULT_TABLE table=default.Table_3b335a0806154f98a907f551596ba353 [id=0da4db61b3fe4cf4846edd5530b9327f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.805675 895 tablet_service.cc:1505] Processing CreateTablet for tablet 91ec01d25f8f441cbe3783cacbee51db (DEFAULT_TABLE table=default.Table_b18a63e4731b418abcb84c36082553c1 [id=255f4d94812f45d0b4423150098460ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.806206 1042 tablet_service.cc:1552] Processing DeleteTablet for tablet 2f0e8c5e2fd44370ae150a206f4e6cbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:18.811882 1175 raft_consensus.cc:359] T 0186572e7bb9499a932a636ab280f544 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.812153 1175 raft_consensus.cc:385] T 0186572e7bb9499a932a636ab280f544 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:18.812532 1043 tablet_service.cc:1552] Processing DeleteTablet for tablet 0186572e7bb9499a932a636ab280f544 with delete_type TABLET_DATA_DELETED (replaced by tablet 79c73f4a92044bafb5b8384229cb3a4d) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:18.815816 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20ea39bc5d3d46f5a4ba427277d6c6a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.816609 1175 raft_consensus.cc:740] T 0186572e7bb9499a932a636ab280f544 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:18.816929 1175 consensus_queue.cc:260] T 0186572e7bb9499a932a636ab280f544 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.818282 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91ec01d25f8f441cbe3783cacbee51db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.823100 1175 ts_tablet_manager.cc:1434] T 0186572e7bb9499a932a636ab280f544 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.060s user 0.002s sys 0.000s
I20260202 02:52:18.823343 900 tablet_service.cc:1505] Processing CreateTablet for tablet 79f694b740954d429145fba9157338ca (DEFAULT_TABLE table=default.Table_662111850b514df5ad7c69aa21783397 [id=d595db36d702401ab81d5401ad3a76e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.823469 1175 tablet_bootstrap.cc:492] T 80d443bdb66d4cc68ae283e08c410fe6 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:18.824515 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79f694b740954d429145fba9157338ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.832548 903 tablet_service.cc:1552] Processing DeleteTablet for tablet 267bd4752c5f4b99a3fe778ba8f38a2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:18.837128 903 tablet_service.cc:1552] Processing DeleteTablet for tablet 0b411e8fab074aa88f8380e3f5694340 with delete_type TABLET_DATA_DELETED (replaced by tablet 28a900bcaa8b41718a285e595886d805) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:18.841542 1183 ts_tablet_manager.cc:1929] T ffd8aa2bc6e344b78dce518411997fb5 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:18.841800 1183 log.cc:1199] T ffd8aa2bc6e344b78dce518411997fb5 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/ffd8aa2bc6e344b78dce518411997fb5
I20260202 02:52:18.842207 1183 ts_tablet_manager.cc:1950] T ffd8aa2bc6e344b78dce518411997fb5 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:18.842842 1183 tablet_replica.cc:333] T 97bd7223d41d40a6bcdc60b1f63abc67 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:18.843056 1183 raft_consensus.cc:2243] T 97bd7223d41d40a6bcdc60b1f63abc67 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:18.843247 1183 raft_consensus.cc:2272] T 97bd7223d41d40a6bcdc60b1f63abc67 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:18.844137 1183 ts_tablet_manager.cc:1916] T 97bd7223d41d40a6bcdc60b1f63abc67 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:18.846693 897 tablet_service.cc:1552] Processing DeleteTablet for tablet bbf31e5619ed4f60b4d7c8b19ec8a549 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:16 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:18.846937 902 tablet_service.cc:1552] Processing DeleteTablet for tablet ea040c03420c4b1b98897f39160b4130 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:18.847038 899 tablet_service.cc:1552] Processing DeleteTablet for tablet fae7e3941c124cc2871390a0b6471f27 with delete_type TABLET_DATA_DELETED (replaced by tablet a5622ef925bc46cca76f481ddfd12afd) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:18.847307 894 tablet_service.cc:1552] Processing DeleteTablet for tablet c064ea0dade840b4840a5715c2d83f69 with delete_type TABLET_DATA_DELETED (replaced by tablet 622a113af04a41c6852c29fc8a8acd78) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:18.847399 902 tablet_service.cc:1552] Processing DeleteTablet for tablet bf5ca99708044482970cb854d2eced18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:18.847580 893 tablet_service.cc:1552] Processing DeleteTablet for tablet 251455168e33430aafea8e0de42ca1b8 with delete_type TABLET_DATA_DELETED (replaced by tablet fe53a8bdbd13417f9ce195c54a52a93c) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:18.847671 902 tablet_service.cc:1552] Processing DeleteTablet for tablet ffd8aa2bc6e344b78dce518411997fb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:18.848160 902 tablet_service.cc:1505] Processing CreateTablet for tablet 68823eff1bc04108ba83487605aec0ea (DEFAULT_TABLE table=default.Table_d4ad0ff5676e4af2a187497eda81d7a2 [id=0769ae34ed3d479a9f7010b34c35a110]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.848616 891 tablet_service.cc:1505] Processing CreateTablet for tablet af5aa68e47bb420eac300754d274fa73 (DEFAULT_TABLE table=default.Table_91411bac2a2a45a395b81a4dab2617f7 [id=feb8b9f926434cec8b119843413f8c9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.848670 892 tablet_service.cc:1505] Processing CreateTablet for tablet 042d1ebf357b4abe8814a407891ce265 (DEFAULT_TABLE table=default.Table_fe06d3b0b66447e29403a54ab14c476c [id=cab83305ba6648e5b307441e71671a82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.849634 899 tablet_service.cc:1552] Processing DeleteTablet for tablet 415d14533fd54f20adb00455a7bcad5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:16 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:18.848161 893 tablet_service.cc:1505] Processing CreateTablet for tablet 8774c6bc0fe9414ea2f17ed859d9392a (DEFAULT_TABLE table=default.Table_0eb450e3edbb4a4ba0906567f831830c [id=8918ec9e2cdf4f2995b2b4baf21297d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.850394 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet ffd8aa2bc6e344b78dce518411997fb5 (table default.Table_ab8d74f30cd14c61b85950a82d0f3787 [id=3ccb047c30974c0f9cbe778971e61644]) successfully deleted
I20260202 02:52:18.850642 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af5aa68e47bb420eac300754d274fa73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.853230 1533 tablet.cc:1722] T c0937b6910534ddc8a2217f0cbd539c5 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:18.849339 889 tablet_service.cc:1505] Processing CreateTablet for tablet cb634d4bae554ef3b39eeaa9958adc6c (DEFAULT_TABLE table=default.Table_983bf3e0c5424c2588a0e754af1a5687 [id=c97e407e519d4a538860f66931f6aefd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.854349 886 tablet_service.cc:1505] Processing CreateTablet for tablet 38301e13e17c41fcae912c4b38c26a45 (DEFAULT_TABLE table=default.Table_80912cc76f934f139da7f866546ae97a [id=2f3bf7e0cf4c41ac8a2554360b57214a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.854990 894 tablet_service.cc:1505] Processing CreateTablet for tablet c25e329023c04f609d8a743b4f7a95a4 (DEFAULT_TABLE table=default.Table_52fba283eb604f81acbbc5ea3762d1f9 [id=81e199e091b149a0a103eb3b8a232983]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.857311 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 042d1ebf357b4abe8814a407891ce265. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.857282 1183 ts_tablet_manager.cc:1929] T 97bd7223d41d40a6bcdc60b1f63abc67 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:18.857445 1183 log.cc:1199] T 97bd7223d41d40a6bcdc60b1f63abc67 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/97bd7223d41d40a6bcdc60b1f63abc67
I20260202 02:52:18.858244 1183 ts_tablet_manager.cc:1950] T 97bd7223d41d40a6bcdc60b1f63abc67 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:18.858947 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 97bd7223d41d40a6bcdc60b1f63abc67 (table default.Table_53eaadb859904d6b95ef0e536019ad79 [id=0ca5aac09bb54feba75b2d3784a69140]) successfully deleted
I20260202 02:52:18.862613 1173 tablet_bootstrap.cc:492] T 61a9203035f2495f8f99924303ebf120 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:18.864250 1173 ts_tablet_manager.cc:1403] T 61a9203035f2495f8f99924303ebf120 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.192s user 0.005s sys 0.009s
W20260202 02:52:18.865046 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 13cd1383348b458894fdecc22bf24b74 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 13cd1383348b458894fdecc22bf24b74 already in progress: creating tablet
I20260202 02:52:18.865581 1173 raft_consensus.cc:359] T 61a9203035f2495f8f99924303ebf120 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:18.866122 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 8774c6bc0fe9414ea2f17ed859d9392a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8774c6bc0fe9414ea2f17ed859d9392a already in progress: creating tablet
W20260202 02:52:18.867007 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 2f0e8c5e2fd44370ae150a206f4e6cbc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2f0e8c5e2fd44370ae150a206f4e6cbc already in progress: creating tablet
I20260202 02:52:18.867261 1173 raft_consensus.cc:385] T 61a9203035f2495f8f99924303ebf120 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260202 02:52:18.867791 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 0186572e7bb9499a932a636ab280f544 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0186572e7bb9499a932a636ab280f544 already in progress: creating tablet
I20260202 02:52:18.867980 1173 raft_consensus.cc:740] T 61a9203035f2495f8f99924303ebf120 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:18.868536 1173 consensus_queue.cc:260] T 61a9203035f2495f8f99924303ebf120 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.869673 903 tablet_service.cc:1552] Processing DeleteTablet for tablet eaa47f3388144e8b852804c167f232ce with delete_type TABLET_DATA_DELETED (replaced by tablet 38fdbc6df7ec40b9afe4e0dfe0119e3e) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:18.870633 903 tablet_service.cc:1552] Processing DeleteTablet for tablet 13cd1383348b458894fdecc22bf24b74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:18.870882 903 tablet_service.cc:1552] Processing DeleteTablet for tablet 8774c6bc0fe9414ea2f17ed859d9392a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:18.873366 1173 ts_tablet_manager.cc:1434] T 61a9203035f2495f8f99924303ebf120 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.009s user 0.000s sys 0.002s
I20260202 02:52:18.871863 887 tablet_service.cc:1505] Processing CreateTablet for tablet 08d8d781bb4e4acf915f11afb4fb2e22 (DEFAULT_TABLE table=default.Table_2acd399bc33146fe9f1b6e2a15e0b101 [id=487a204c7845401ab7391ae8464fdf1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.875672 1173 tablet_bootstrap.cc:492] T c1fc01252d95407e954cfcbf7e715912 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:18.881652 903 tablet_service.cc:1552] Processing DeleteTablet for tablet 2f0e8c5e2fd44370ae150a206f4e6cbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:18.883107 896 tablet_service.cc:1552] Processing DeleteTablet for tablet 0186572e7bb9499a932a636ab280f544 with delete_type TABLET_DATA_DELETED (replaced by tablet 79c73f4a92044bafb5b8384229cb3a4d) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:18.884032 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8774c6bc0fe9414ea2f17ed859d9392a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.884528 897 tablet_service.cc:1552] Processing DeleteTablet for tablet 23c38ef4e3c840808ec36f859cf6f6de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:18.885449 884 tablet_service.cc:1505] Processing CreateTablet for tablet 13cd1383348b458894fdecc22bf24b74 (DEFAULT_TABLE table=default.Table_f92d6341db3143e490f6d64418ef6ff2 [id=ebc82aae55584b8ba2ba5c4223d9e258]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.886070 888 tablet_service.cc:1505] Processing CreateTablet for tablet a1f39e5b02f747819e636cddb4292eb6 (DEFAULT_TABLE table=default.Table_92b7f492c09948f784c64ccdcaadbb86 [id=abceac85d3524c96834546f67f41f933]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.886519 885 tablet_service.cc:1505] Processing CreateTablet for tablet 38fdbc6df7ec40b9afe4e0dfe0119e3e (DEFAULT_TABLE table=default.Table_87277c5d80184738b96b773f026b32a5 [id=52687dbd0e554cfb81d88e8fcdd4dd5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.889016 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08d8d781bb4e4acf915f11afb4fb2e22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.889199 899 tablet_service.cc:1505] Processing CreateTablet for tablet 9c1271ce0f194021b588b82ac9d3db79 (DEFAULT_TABLE table=default.Table_733044f900d44218a06c976c8701f122 [id=9f912bc02bec49a8b79e4e7e921595b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.890201 1173 tablet_bootstrap.cc:654] T c1fc01252d95407e954cfcbf7e715912 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:18.892952 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c1271ce0f194021b588b82ac9d3db79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.895246 1173 tablet_bootstrap.cc:492] T c1fc01252d95407e954cfcbf7e715912 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:18.895516 1173 ts_tablet_manager.cc:1403] T c1fc01252d95407e954cfcbf7e715912 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.001s
I20260202 02:52:18.896476 1173 raft_consensus.cc:359] T c1fc01252d95407e954cfcbf7e715912 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.896878 1173 raft_consensus.cc:385] T c1fc01252d95407e954cfcbf7e715912 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:18.897011 1173 raft_consensus.cc:740] T c1fc01252d95407e954cfcbf7e715912 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:18.897348 1173 consensus_queue.cc:260] T c1fc01252d95407e954cfcbf7e715912 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.898938 1109 heartbeater.cc:344] Connected to a master server at 127.0.104.188:39077
I20260202 02:52:18.900527 1173 ts_tablet_manager.cc:1434] T c1fc01252d95407e954cfcbf7e715912 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260202 02:52:18.903133 1173 tablet_bootstrap.cc:492] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:18.905081 1173 tablet_bootstrap.cc:654] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:18.921180 1173 tablet_bootstrap.cc:492] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:18.921507 1173 ts_tablet_manager.cc:1403] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.001s
I20260202 02:52:18.921590 1178 raft_consensus.cc:493] T 61a9203035f2495f8f99924303ebf120 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:18.921798 1178 raft_consensus.cc:515] T 61a9203035f2495f8f99924303ebf120 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.922551 1173 raft_consensus.cc:359] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.922845 1173 raft_consensus.cc:385] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:18.922919 1173 raft_consensus.cc:740] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:18.922569 1178 leader_election.cc:290] T 61a9203035f2495f8f99924303ebf120 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:18.923770 1173 consensus_queue.cc:260] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.890645 890 tablet_service.cc:1505] Processing CreateTablet for tablet 28a900bcaa8b41718a285e595886d805 (DEFAULT_TABLE table=default.Table_052de59e60de4473a2d13185440ab818 [id=cd07f9746ec94d9ebe956a7ca0d19b1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:18.924738 1173 ts_tablet_manager.cc:1434] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260202 02:52:18.926910 1173 tablet_bootstrap.cc:492] T 098c5cdfcd0f4c52b6ae4f986f07705a P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:18.928972 1173 tablet_bootstrap.cc:654] T 098c5cdfcd0f4c52b6ae4f986f07705a P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:18.944512 1173 tablet_bootstrap.cc:492] T 098c5cdfcd0f4c52b6ae4f986f07705a P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:18.944825 1173 ts_tablet_manager.cc:1403] T 098c5cdfcd0f4c52b6ae4f986f07705a P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.004s
I20260202 02:52:18.945935 1173 raft_consensus.cc:359] T 098c5cdfcd0f4c52b6ae4f986f07705a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.946208 1173 raft_consensus.cc:385] T 098c5cdfcd0f4c52b6ae4f986f07705a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:18.946347 1173 raft_consensus.cc:740] T 098c5cdfcd0f4c52b6ae4f986f07705a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:18.946681 1173 consensus_queue.cc:260] T 098c5cdfcd0f4c52b6ae4f986f07705a P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.947371 1173 ts_tablet_manager.cc:1434] T 098c5cdfcd0f4c52b6ae4f986f07705a P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260202 02:52:18.947785 1173 tablet_bootstrap.cc:492] T 4f2a9404f7424dc78b5bd062ebd8cb53 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:18.949699 1173 tablet_bootstrap.cc:654] T 4f2a9404f7424dc78b5bd062ebd8cb53 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:18.967343 1173 tablet_bootstrap.cc:492] T 4f2a9404f7424dc78b5bd062ebd8cb53 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:18.967650 1173 ts_tablet_manager.cc:1403] T 4f2a9404f7424dc78b5bd062ebd8cb53 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.000s
I20260202 02:52:18.969246 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38fdbc6df7ec40b9afe4e0dfe0119e3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.968534 1173 raft_consensus.cc:359] T 4f2a9404f7424dc78b5bd062ebd8cb53 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:18.969482 1173 raft_consensus.cc:385] T 4f2a9404f7424dc78b5bd062ebd8cb53 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:18.969558 1173 raft_consensus.cc:740] T 4f2a9404f7424dc78b5bd062ebd8cb53 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:18.969861 1173 consensus_queue.cc:260] T 4f2a9404f7424dc78b5bd062ebd8cb53 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:18.970374 1173 ts_tablet_manager.cc:1434] T 4f2a9404f7424dc78b5bd062ebd8cb53 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260202 02:52:18.970765 1173 tablet_bootstrap.cc:492] T 0d297b6243c348ee82c146a8339cf6c4 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:18.972555 1173 tablet_bootstrap.cc:654] T 0d297b6243c348ee82c146a8339cf6c4 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:18.974227 1175 tablet_bootstrap.cc:654] T 80d443bdb66d4cc68ae283e08c410fe6 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:18.978335 1173 tablet_bootstrap.cc:492] T 0d297b6243c348ee82c146a8339cf6c4 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:18.978515 1173 ts_tablet_manager.cc:1403] T 0d297b6243c348ee82c146a8339cf6c4 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260202 02:52:18.979483 1173 raft_consensus.cc:359] T 0d297b6243c348ee82c146a8339cf6c4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:18.979701 1173 raft_consensus.cc:385] T 0d297b6243c348ee82c146a8339cf6c4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:18.979969 1173 raft_consensus.cc:740] T 0d297b6243c348ee82c146a8339cf6c4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:18.980363 1173 consensus_queue.cc:260] T 0d297b6243c348ee82c146a8339cf6c4 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:18.981014 1173 ts_tablet_manager.cc:1434] T 0d297b6243c348ee82c146a8339cf6c4 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260202 02:52:18.981387 1173 tablet_bootstrap.cc:492] T 5986676f5484498da927b4c6da3f47ff P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:18.982339 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1f39e5b02f747819e636cddb4292eb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.982751 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28a900bcaa8b41718a285e595886d805. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:18.983673 1173 tablet_bootstrap.cc:654] T 5986676f5484498da927b4c6da3f47ff P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:18.993515 1173 tablet_bootstrap.cc:492] T 5986676f5484498da927b4c6da3f47ff P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:18.993841 1173 ts_tablet_manager.cc:1403] T 5986676f5484498da927b4c6da3f47ff P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.008s
I20260202 02:52:18.995337 1173 raft_consensus.cc:359] T 5986676f5484498da927b4c6da3f47ff P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.995666 1173 raft_consensus.cc:385] T 5986676f5484498da927b4c6da3f47ff P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:18.995790 1173 raft_consensus.cc:740] T 5986676f5484498da927b4c6da3f47ff P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:18.996178 1173 consensus_queue.cc:260] T 5986676f5484498da927b4c6da3f47ff P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:18.996658 1059 raft_consensus.cc:3060] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:18.996938 1173 ts_tablet_manager.cc:1434] T 5986676f5484498da927b4c6da3f47ff P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260202 02:52:18.997284 1173 tablet_bootstrap.cc:492] T 952d760159844ad08269654264e6381b P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:18.999318 1173 tablet_bootstrap.cc:654] T 952d760159844ad08269654264e6381b P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:19.008925 1173 tablet_bootstrap.cc:492] T 952d760159844ad08269654264e6381b P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:19.010778 1173 ts_tablet_manager.cc:1403] T 952d760159844ad08269654264e6381b P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.005s
I20260202 02:52:19.011747 1173 raft_consensus.cc:359] T 952d760159844ad08269654264e6381b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:19.012043 1173 raft_consensus.cc:385] T 952d760159844ad08269654264e6381b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:19.012184 1173 raft_consensus.cc:740] T 952d760159844ad08269654264e6381b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:19.012557 1173 consensus_queue.cc:260] T 952d760159844ad08269654264e6381b P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:19.013526 1173 ts_tablet_manager.cc:1434] T 952d760159844ad08269654264e6381b P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260202 02:52:19.013927 1173 tablet_bootstrap.cc:492] T 23c38ef4e3c840808ec36f859cf6f6de P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:19.015765 1173 tablet_bootstrap.cc:654] T 23c38ef4e3c840808ec36f859cf6f6de P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:19.018159 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13cd1383348b458894fdecc22bf24b74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.024209 1173 tablet_bootstrap.cc:492] T 23c38ef4e3c840808ec36f859cf6f6de P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:19.024467 1173 ts_tablet_manager.cc:1403] T 23c38ef4e3c840808ec36f859cf6f6de P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20260202 02:52:19.026069 1173 raft_consensus.cc:359] T 23c38ef4e3c840808ec36f859cf6f6de P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:19.026907 1173 raft_consensus.cc:385] T 23c38ef4e3c840808ec36f859cf6f6de P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:19.027032 1173 raft_consensus.cc:740] T 23c38ef4e3c840808ec36f859cf6f6de P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:19.027385 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68823eff1bc04108ba83487605aec0ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.027559 1173 consensus_queue.cc:260] T 23c38ef4e3c840808ec36f859cf6f6de P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:19.028277 1173 ts_tablet_manager.cc:1434] T 23c38ef4e3c840808ec36f859cf6f6de P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260202 02:52:19.028754 1173 tablet_bootstrap.cc:492] T aa526a69cf914dd08142074636c956b7 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:19.030578 1173 tablet_bootstrap.cc:654] T aa526a69cf914dd08142074636c956b7 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:19.031044 1182 ts_tablet_manager.cc:1929] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:19.032593 1182 log.cc:1199] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/f93e8d3c4faf48eaa98c3cbf88d585e2
I20260202 02:52:19.039033 1173 tablet_bootstrap.cc:492] T aa526a69cf914dd08142074636c956b7 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:19.039225 1173 ts_tablet_manager.cc:1403] T aa526a69cf914dd08142074636c956b7 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20260202 02:52:19.039433 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c25e329023c04f609d8a743b4f7a95a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.040128 1173 raft_consensus.cc:359] T aa526a69cf914dd08142074636c956b7 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:19.040365 1173 raft_consensus.cc:385] T aa526a69cf914dd08142074636c956b7 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:19.040630 1173 raft_consensus.cc:740] T aa526a69cf914dd08142074636c956b7 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:19.040978 1173 consensus_queue.cc:260] T aa526a69cf914dd08142074636c956b7 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:19.041517 1173 ts_tablet_manager.cc:1434] T aa526a69cf914dd08142074636c956b7 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:19.041853 1173 tablet_bootstrap.cc:492] T ea040c03420c4b1b98897f39160b4130 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:19.043784 1173 tablet_bootstrap.cc:654] T ea040c03420c4b1b98897f39160b4130 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:19.046183 1173 tablet_bootstrap.cc:492] T ea040c03420c4b1b98897f39160b4130 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:19.046370 1173 ts_tablet_manager.cc:1403] T ea040c03420c4b1b98897f39160b4130 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260202 02:52:19.048274 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb634d4bae554ef3b39eeaa9958adc6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.048434 1173 raft_consensus.cc:359] T ea040c03420c4b1b98897f39160b4130 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:19.048682 1173 raft_consensus.cc:385] T ea040c03420c4b1b98897f39160b4130 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:19.048758 1173 raft_consensus.cc:740] T ea040c03420c4b1b98897f39160b4130 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:19.048997 1173 consensus_queue.cc:260] T ea040c03420c4b1b98897f39160b4130 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:19.049515 1173 ts_tablet_manager.cc:1434] T ea040c03420c4b1b98897f39160b4130 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260202 02:52:19.049837 1173 tablet_bootstrap.cc:492] T bd7b0a0fcb7645fdb9edfeccb3d18268 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:19.051438 1173 tablet_bootstrap.cc:654] T bd7b0a0fcb7645fdb9edfeccb3d18268 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:19.059684 1178 raft_consensus.cc:493] T c050a2882f764dda8df2a7df6547f149 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:19.059942 1178 raft_consensus.cc:515] T c050a2882f764dda8df2a7df6547f149 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:19.060746 1178 leader_election.cc:290] T c050a2882f764dda8df2a7df6547f149 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:19.076367 1182 ts_tablet_manager.cc:1950] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:19.080861 1182 tablet_replica.cc:333] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:19.096504 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38301e13e17c41fcae912c4b38c26a45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.131449 1182 raft_consensus.cc:2243] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:19.131871 1182 raft_consensus.cc:2272] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:19.132625 1182 ts_tablet_manager.cc:1916] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:19.132992 1178 raft_consensus.cc:493] T ea040c03420c4b1b98897f39160b4130 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:19.133200 1178 raft_consensus.cc:515] T ea040c03420c4b1b98897f39160b4130 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:19.133921 1178 leader_election.cc:290] T ea040c03420c4b1b98897f39160b4130 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:19.148367 1453 raft_consensus.cc:493] T b428356fba0c4fb5a6033c5d39740cc5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:19.148638 1109 heartbeater.cc:344] Connected to a master server at 127.0.104.188:39077
I20260202 02:52:19.148562 1453 raft_consensus.cc:515] T b428356fba0c4fb5a6033c5d39740cc5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:19.149209 1453 leader_election.cc:290] T b428356fba0c4fb5a6033c5d39740cc5 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:19.150377 918 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ffd8aa2bc6e344b78dce518411997fb5" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:19.150431 920 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "61a9203035f2495f8f99924303ebf120" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
W20260202 02:52:19.150671 857 leader_election.cc:343] T bc1539e5c8aa4d6e98f687728b891530 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:19.152851 1063 raft_consensus.cc:3060] T ea040c03420c4b1b98897f39160b4130 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:19.159106 752 tablet_service.cc:1505] Processing CreateTablet for tablet a1f39e5b02f747819e636cddb4292eb6 (DEFAULT_TABLE table=default.Table_92b7f492c09948f784c64ccdcaadbb86 [id=abceac85d3524c96834546f67f41f933]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.160391 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 0186572e7bb9499a932a636ab280f544 with delete_type TABLET_DATA_DELETED (replaced by tablet 79c73f4a92044bafb5b8384229cb3a4d) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:19.164775 524 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet c0937b6910534ddc8a2217f0cbd539c5 (table default.Table_e0149b31be0c49d19600de4849d6577e [id=46b5d47377474ff9bac2acdb26b08c65]). Expected version 1 got 0
I20260202 02:52:19.165237 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 13cd1383348b458894fdecc22bf24b74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:19.165558 527 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_e0149b31be0c49d19600de4849d6577e [id=46b5d47377474ff9bac2acdb26b08c65] alter complete (version 1)
W20260202 02:52:19.166241 527 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 78f122342dd144b48de89902faa0f94f (table default.Table_5eefa56618e84f8d92f8d87132ccd7d4 [id=b4ab2ba86e1a49338a98d855ded99f5d], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 78f122342dd144b48de89902faa0f94f consensus state has no leader
I20260202 02:52:19.167485 1534 raft_consensus.cc:493] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:19.167694 1534 raft_consensus.cc:515] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:19.169044 1534 leader_election.cc:290] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:19.169421 754 tablet_service.cc:1505] Processing CreateTablet for tablet 13cd1383348b458894fdecc22bf24b74 (DEFAULT_TABLE table=default.Table_f92d6341db3143e490f6d64418ef6ff2 [id=ebc82aae55584b8ba2ba5c4223d9e258]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:19.172566 714 leader_election.cc:343] T bf74f96bea2e4ab7adec42c3c5207e79 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: bf74f96bea2e4ab7adec42c3c5207e79
I20260202 02:52:19.172925 1174 tablet_bootstrap.cc:492] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:19.173111 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_0c2fc83ca5324ab3b8b56c2aaf42864c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:19.173767 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0c2fc83ca5324ab3b8b56c2aaf42864c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:19.175230 522 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ae49654353a64141b6d7f0e6afe13d9d" } modify_external_catalogs: true
I20260202 02:52:19.175457 522 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ae49654353a64141b6d7f0e6afe13d9d" } modify_external_catalogs: true
I20260202 02:52:19.176913 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_38e8a46dd2dd498baa45935fa4670b09"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:19.177500 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_38e8a46dd2dd498baa45935fa4670b09 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:19.182197 528 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 26eeef6eebbf4f2cb44f69255fb37594 has no consensus state
I20260202 02:52:19.186031 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_5ed512e7fc01454392b327c54420ec55"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:19.186825 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5ed512e7fc01454392b327c54420ec55 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:19.189276 520 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_b6e7efe67b0e4efbb7e7c51ee5f1caa7" } new_table_name: "default.Table_0bf3a06700844105a02b98edd499e75a" modify_external_catalogs: true
I20260202 02:52:19.192760 528 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 61a9203035f2495f8f99924303ebf120 (table default.Table_b3001bd4a86843ada9ee50b8e4915a9a [id=36a229921ecc4185adbfdfba372a38ff]). Expected version 1 got 0
I20260202 02:52:19.193441 528 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet d2d27ceea1bb4122b18f0fcf4d2d1bf4 (table default.Table_10aa0dafec0547069eb92ecf12b3c780 [id=1f9c48c8e2484856a66a0719268f22ae]). Expected version 1 got 0
I20260202 02:52:19.193967 528 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 5986676f5484498da927b4c6da3f47ff (table default.Table_8cb3cbab1a654f88a07d6d480fc72e52 [id=d8485f23f63b493781bea557dbe8a0c1]). Expected version 1 got 0
I20260202 02:52:19.198529 1037 tablet_service.cc:1552] Processing DeleteTablet for tablet 4f2a9404f7424dc78b5bd062ebd8cb53 with delete_type TABLET_DATA_DELETED (replaced by tablet 526f2c7b95184a0987e7aa94efc8475a) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:19.198923 1183 tablet_replica.cc:333] T 4f2a9404f7424dc78b5bd062ebd8cb53 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:19.199149 1183 raft_consensus.cc:2243] T 4f2a9404f7424dc78b5bd062ebd8cb53 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:19.199368 1183 raft_consensus.cc:2272] T 4f2a9404f7424dc78b5bd062ebd8cb53 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:19.199429 1037 tablet_service.cc:1552] Processing DeleteTablet for tablet 098c5cdfcd0f4c52b6ae4f986f07705a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:16 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:19.201010 1039 tablet_service.cc:1552] Processing DeleteTablet for tablet ea040c03420c4b1b98897f39160b4130 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:19.201239 1034 tablet_service.cc:1552] Processing DeleteTablet for tablet c1fc01252d95407e954cfcbf7e715912 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:19.201440 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet aa526a69cf914dd08142074636c956b7 with delete_type TABLET_DATA_DELETED (replaced by tablet c184d44d7f434b39a997a7387e5d3de0) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:19.201632 1043 tablet_service.cc:1552] Processing DeleteTablet for tablet 23c38ef4e3c840808ec36f859cf6f6de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:19.201812 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 0d297b6243c348ee82c146a8339cf6c4 with delete_type TABLET_DATA_DELETED (replaced by tablet d64f42d187bc4c9f9381c931449ec0cc) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:19.202232 1037 tablet_service.cc:1552] Processing DeleteTablet for tablet 952d760159844ad08269654264e6381b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:19.202311 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_458f570486154241ad8c7f10a8135ace"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:19.202904 1183 ts_tablet_manager.cc:1916] T 4f2a9404f7424dc78b5bd062ebd8cb53 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:19.203577 1324 raft_consensus.cc:493] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:19.203970 750 tablet_service.cc:1505] Processing CreateTablet for tablet 38301e13e17c41fcae912c4b38c26a45 (DEFAULT_TABLE table=default.Table_80912cc76f934f139da7f866546ae97a [id=2f3bf7e0cf4c41ac8a2554360b57214a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.204438 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13cd1383348b458894fdecc22bf24b74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.208724 753 tablet_service.cc:1505] Processing CreateTablet for tablet 38fdbc6df7ec40b9afe4e0dfe0119e3e (DEFAULT_TABLE table=default.Table_87277c5d80184738b96b773f026b32a5 [id=52687dbd0e554cfb81d88e8fcdd4dd5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.209245 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 8774c6bc0fe9414ea2f17ed859d9392a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:19.209908 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d06b77ff1a6f4f78b73082b00145fcf6" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:19.210424 749 tablet_service.cc:1505] Processing CreateTablet for tablet 9c1271ce0f194021b588b82ac9d3db79 (DEFAULT_TABLE table=default.Table_733044f900d44218a06c976c8701f122 [id=9f912bc02bec49a8b79e4e7e921595b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.210865 1183 ts_tablet_manager.cc:1929] T 4f2a9404f7424dc78b5bd062ebd8cb53 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:19.212994 779 raft_consensus.cc:3060] T d06b77ff1a6f4f78b73082b00145fcf6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:19.213155 1174 ts_tablet_manager.cc:1403] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.470s user 0.021s sys 0.089s
I20260202 02:52:19.213629 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "61a9203035f2495f8f99924303ebf120" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:19.214105 1324 raft_consensus.cc:515] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:19.214315 1174 raft_consensus.cc:359] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:19.214648 1174 raft_consensus.cc:385] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:19.214751 1174 raft_consensus.cc:740] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:19.214846 1324 leader_election.cc:290] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:19.215032 1174 consensus_queue.cc:260] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:19.215529 1174 ts_tablet_manager.cc:1434] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:19.212915 1183 log.cc:1199] T 4f2a9404f7424dc78b5bd062ebd8cb53 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/4f2a9404f7424dc78b5bd062ebd8cb53
I20260202 02:52:19.229223 1183 ts_tablet_manager.cc:1950] T 4f2a9404f7424dc78b5bd062ebd8cb53 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:19.235517 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bbf31e5619ed4f60b4d7c8b19ec8a549" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:19.235533 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "526f2c7b95184a0987e7aa94efc8475a" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:19.235998 758 tablet_service.cc:1505] Processing CreateTablet for tablet 08d8d781bb4e4acf915f11afb4fb2e22 (DEFAULT_TABLE table=default.Table_2acd399bc33146fe9f1b6e2a15e0b101 [id=487a204c7845401ab7391ae8464fdf1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.239077 1174 tablet_bootstrap.cc:492] T 70306ed9dec54286885d965f2c012da4 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:19.239217 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38fdbc6df7ec40b9afe4e0dfe0119e3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.212555 1277 consensus_queue.cc:1048] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.082s
I20260202 02:52:19.243131 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08d8d781bb4e4acf915f11afb4fb2e22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.243273 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9884c90cf1d245b2a7eaaf00d71b1c3e" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:19.243191 776 raft_consensus.cc:2468] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
I20260202 02:52:19.246562 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 4f2a9404f7424dc78b5bd062ebd8cb53 (table default.Table_0842fd6a82e145e7a98b3fed2c0c6993 [id=4e076b2023be4c888ac1c27babde1607]) successfully deleted
I20260202 02:52:19.247397 1178 raft_consensus.cc:493] T 0d297b6243c348ee82c146a8339cf6c4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:19.247661 1178 raft_consensus.cc:515] T 0d297b6243c348ee82c146a8339cf6c4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:19.248633 1178 leader_election.cc:290] T 0d297b6243c348ee82c146a8339cf6c4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:19.249145 1180 ts_tablet_manager.cc:1929] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:19.249359 1180 log.cc:1199] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/ffd8aa2bc6e344b78dce518411997fb5
I20260202 02:52:19.213124 1597 raft_consensus.cc:493] T c184d44d7f434b39a997a7387e5d3de0 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:19.249884 1180 ts_tablet_manager.cc:1950] T ffd8aa2bc6e344b78dce518411997fb5 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:19.249787 1597 raft_consensus.cc:515] T c184d44d7f434b39a997a7387e5d3de0 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:19.251049 1180 tablet_replica.cc:333] T bf5ca99708044482970cb854d2eced18 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:19.251299 1180 raft_consensus.cc:2243] T bf5ca99708044482970cb854d2eced18 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:19.251478 1180 raft_consensus.cc:2272] T bf5ca99708044482970cb854d2eced18 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:19.251533 1597 leader_election.cc:290] T c184d44d7f434b39a997a7387e5d3de0 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:19.252133 1180 ts_tablet_manager.cc:1916] T bf5ca99708044482970cb854d2eced18 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:19.268123 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c1271ce0f194021b588b82ac9d3db79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.268078 714 leader_election.cc:304] T bf74f96bea2e4ab7adec42c3c5207e79 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
W20260202 02:52:19.268571 714 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 96016.6us
I20260202 02:52:19.269232 714 leader_election.cc:304] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
W20260202 02:52:19.269742 714 leader_election.cc:343] T bf5ca99708044482970cb854d2eced18 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: bf5ca99708044482970cb854d2eced18
I20260202 02:52:19.269903 714 leader_election.cc:304] T bf5ca99708044482970cb854d2eced18 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
W20260202 02:52:19.270206 714 raft_consensus.cc:2686] T bf5ca99708044482970cb854d2eced18 P 1f442ef85029461091de847118c34c0d: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260202 02:52:19.270673 714 consensus_peers.cc:597] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23c38ef4e3c840808ec36f859cf6f6de. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:19.270854 714 leader_election.cc:304] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:19.273306 1183 tablet_replica.cc:333] T 098c5cdfcd0f4c52b6ae4f986f07705a P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:19.275507 1183 raft_consensus.cc:2243] T 098c5cdfcd0f4c52b6ae4f986f07705a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:19.281028 1324 raft_consensus.cc:2749] T bf74f96bea2e4ab7adec42c3c5207e79 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:19.280807 1183 raft_consensus.cc:2272] T 098c5cdfcd0f4c52b6ae4f986f07705a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:19.284422 1534 consensus_queue.cc:1048] T c050a2882f764dda8df2a7df6547f149 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260202 02:52:19.284739 1324 consensus_queue.cc:1048] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:19.285629 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38301e13e17c41fcae912c4b38c26a45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.283202 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b428356fba0c4fb5a6033c5d39740cc5" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:19.288311 779 raft_consensus.cc:2468] T d06b77ff1a6f4f78b73082b00145fcf6 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
I20260202 02:52:19.287936 1183 ts_tablet_manager.cc:1916] T 098c5cdfcd0f4c52b6ae4f986f07705a P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:19.291244 1534 consensus_queue.cc:1048] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260202 02:52:19.291873 1324 raft_consensus.cc:2804] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:19.291915 1601 raft_consensus.cc:2804] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:19.292057 1324 raft_consensus.cc:697] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:19.292115 1601 raft_consensus.cc:697] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:19.292418 1324 consensus_queue.cc:237] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:19.294234 1173 tablet_bootstrap.cc:492] T bd7b0a0fcb7645fdb9edfeccb3d18268 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:19.294402 1173 ts_tablet_manager.cc:1403] T bd7b0a0fcb7645fdb9edfeccb3d18268 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.245s user 0.042s sys 0.081s
I20260202 02:52:19.295248 1173 raft_consensus.cc:359] T bd7b0a0fcb7645fdb9edfeccb3d18268 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:19.295441 1173 raft_consensus.cc:385] T bd7b0a0fcb7645fdb9edfeccb3d18268 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:19.296203 1570 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92a8191cf1e14e4d99f1c378c2d3c85b. Latest consensus state: current_term: 2 leader_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } } }
I20260202 02:52:19.296473 1570 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:52:19.299469 979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } } }
I20260202 02:52:19.299880 979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [sys.catalog]: This master's current role is: LEADER
I20260202 02:52:19.295529 1173 raft_consensus.cc:740] T bd7b0a0fcb7645fdb9edfeccb3d18268 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:19.300597 1173 consensus_queue.cc:260] T bd7b0a0fcb7645fdb9edfeccb3d18268 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:19.301843 1173 ts_tablet_manager.cc:1434] T bd7b0a0fcb7645fdb9edfeccb3d18268 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260202 02:52:19.302215 1173 tablet_bootstrap.cc:492] T bf5ca99708044482970cb854d2eced18 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:19.304148 1173 tablet_bootstrap.cc:654] T bf5ca99708044482970cb854d2eced18 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:19.307328 1183 ts_tablet_manager.cc:1929] T 098c5cdfcd0f4c52b6ae4f986f07705a P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:19.307549 1183 log.cc:1199] T 098c5cdfcd0f4c52b6ae4f986f07705a P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/098c5cdfcd0f4c52b6ae4f986f07705a
I20260202 02:52:19.307967 1183 ts_tablet_manager.cc:1950] T 098c5cdfcd0f4c52b6ae4f986f07705a P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:19.310765 1183 tablet_replica.cc:333] T ea040c03420c4b1b98897f39160b4130 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:19.311096 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 098c5cdfcd0f4c52b6ae4f986f07705a (table default.Table_601ac359670d4a44a24253267b53a875 [id=cae0773031e040a2972ccb42c18f2267]) successfully deleted
I20260202 02:52:19.312847 1183 raft_consensus.cc:2243] T ea040c03420c4b1b98897f39160b4130 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:19.313059 1183 raft_consensus.cc:2272] T ea040c03420c4b1b98897f39160b4130 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:19.315239 1183 ts_tablet_manager.cc:1916] T ea040c03420c4b1b98897f39160b4130 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:19.150611 919 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c050a2882f764dda8df2a7df6547f149" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:19.159304 1453 raft_consensus.cc:493] T eaa47f3388144e8b852804c167f232ce P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260202 02:52:19.319717 859 leader_election.cc:343] T b428356fba0c4fb5a6033c5d39740cc5 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:19.334939 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_458f570486154241ad8c7f10a8135ace in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:19.336205 1183 ts_tablet_manager.cc:1929] T ea040c03420c4b1b98897f39160b4130 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:19.339529 1183 log.cc:1199] T ea040c03420c4b1b98897f39160b4130 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/ea040c03420c4b1b98897f39160b4130
W20260202 02:52:19.339665 525 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ff6f5710de4b4c48948aacdf5be29e8b consensus state has no leader
I20260202 02:52:19.340507 1183 ts_tablet_manager.cc:1950] T ea040c03420c4b1b98897f39160b4130 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:19.341008 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1f39e5b02f747819e636cddb4292eb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.341189 714 leader_election.cc:304] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:19.342157 1183 tablet_replica.cc:333] T c1fc01252d95407e954cfcbf7e715912 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:19.343020 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet ea040c03420c4b1b98897f39160b4130 (table default.Table_9bac792e43464105846e5482edbccacc [id=0a3c23955dde47d9982c4659d9a73e30]) successfully deleted
I20260202 02:52:19.343842 1534 raft_consensus.cc:2804] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:19.344008 1183 raft_consensus.cc:2243] T c1fc01252d95407e954cfcbf7e715912 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:19.345094 1183 raft_consensus.cc:2272] T c1fc01252d95407e954cfcbf7e715912 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:19.345353 714 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 60858.3us
I20260202 02:52:19.345727 1534 raft_consensus.cc:493] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:19.345886 1534 raft_consensus.cc:3060] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:19.346473 1183 ts_tablet_manager.cc:1916] T c1fc01252d95407e954cfcbf7e715912 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:19.292444 1601 consensus_queue.cc:237] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:19.350853 1453 raft_consensus.cc:515] T eaa47f3388144e8b852804c167f232ce P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:19.351286 919 raft_consensus.cc:2393] T c050a2882f764dda8df2a7df6547f149 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1f05820b6034228bdad884cc348716d in current term 1: Already voted for candidate 1f442ef85029461091de847118c34c0d in this term.
I20260202 02:52:19.352830 1175 tablet_bootstrap.cc:492] T 80d443bdb66d4cc68ae283e08c410fe6 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:19.353555 1453 leader_election.cc:290] T eaa47f3388144e8b852804c167f232ce P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:19.354496 859 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 45822.9us
I20260202 02:52:19.355736 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "eaa47f3388144e8b852804c167f232ce" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:19.360890 859 leader_election.cc:343] T eaa47f3388144e8b852804c167f232ce P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:19.367420 1183 ts_tablet_manager.cc:1929] T c1fc01252d95407e954cfcbf7e715912 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:19.373091 1183 log.cc:1199] T c1fc01252d95407e954cfcbf7e715912 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/c1fc01252d95407e954cfcbf7e715912
I20260202 02:52:19.374220 1183 ts_tablet_manager.cc:1950] T c1fc01252d95407e954cfcbf7e715912 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:19.374204 1175 ts_tablet_manager.cc:1403] T 80d443bdb66d4cc68ae283e08c410fe6 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.530s user 0.009s sys 0.004s
I20260202 02:52:19.374441 1174 tablet_bootstrap.cc:654] T 70306ed9dec54286885d965f2c012da4 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:19.374933 755 tablet_service.cc:1552] Processing DeleteTablet for tablet 2f0e8c5e2fd44370ae150a206f4e6cbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:19.376694 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet c1fc01252d95407e954cfcbf7e715912 (table default.Table_f23c311557a14f4a8c66eaf4af7a8a92 [id=10880639cfc743fcafecc538fa2ecd85]) successfully deleted
I20260202 02:52:19.378291 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet ffd8aa2bc6e344b78dce518411997fb5 (table default.Table_ab8d74f30cd14c61b85950a82d0f3787 [id=3ccb047c30974c0f9cbe778971e61644]) successfully deleted
I20260202 02:52:19.379175 1183 tablet_replica.cc:333] T aa526a69cf914dd08142074636c956b7 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:19.380163 1183 raft_consensus.cc:2243] T aa526a69cf914dd08142074636c956b7 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:19.380981 1183 raft_consensus.cc:2272] T aa526a69cf914dd08142074636c956b7 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:19.382874 1183 ts_tablet_manager.cc:1916] T aa526a69cf914dd08142074636c956b7 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:19.390674 1175 raft_consensus.cc:359] T 80d443bdb66d4cc68ae283e08c410fe6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:19.391273 1175 raft_consensus.cc:385] T 80d443bdb66d4cc68ae283e08c410fe6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:19.391369 1175 raft_consensus.cc:740] T 80d443bdb66d4cc68ae283e08c410fe6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:19.392454 857 leader_election.cc:304] T bc1539e5c8aa4d6e98f687728b891530 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:19.411087 1183 ts_tablet_manager.cc:1929] T aa526a69cf914dd08142074636c956b7 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260202 02:52:19.412362 857 raft_consensus.cc:2686] T bc1539e5c8aa4d6e98f687728b891530 P 1521a6d262b7451bbef8debe970dd6b5: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260202 02:52:19.412576 1183 log.cc:1199] T aa526a69cf914dd08142074636c956b7 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/aa526a69cf914dd08142074636c956b7
I20260202 02:52:19.413328 1183 ts_tablet_manager.cc:1950] T aa526a69cf914dd08142074636c956b7 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
W20260202 02:52:19.413561 857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 262902us
I20260202 02:52:19.413966 1182 ts_tablet_manager.cc:1929] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:19.414155 1182 log.cc:1199] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/267bd4752c5f4b99a3fe778ba8f38a2f
I20260202 02:52:19.415087 1175 consensus_queue.cc:260] T 80d443bdb66d4cc68ae283e08c410fe6 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:19.415539 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet aa526a69cf914dd08142074636c956b7 (table default.Table_ab29f752f1e140da9038d9e11ee61f39 [id=9ef504467a544f299b48499e49adf27f]) successfully deleted
I20260202 02:52:19.418291 1183 tablet_replica.cc:333] T 23c38ef4e3c840808ec36f859cf6f6de P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:19.419108 1183 raft_consensus.cc:2243] T 23c38ef4e3c840808ec36f859cf6f6de P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:19.419934 1183 raft_consensus.cc:2272] T 23c38ef4e3c840808ec36f859cf6f6de P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:19.420403 1182 ts_tablet_manager.cc:1950] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:19.423635 1182 tablet_replica.cc:333] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:19.424028 1182 raft_consensus.cc:2243] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:19.424271 1182 pending_rounds.cc:70] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5: Trying to abort 1 pending ops.
I20260202 02:52:19.424868 1182 pending_rounds.cc:77] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923023360086016 op_type: NO_OP noop_request { }
W20260202 02:52:19.427982 857 leader_election.cc:343] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:19.428149 857 leader_election.cc:304] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:19.428455 1453 raft_consensus.cc:2749] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:19.429016 857 leader_election.cc:304] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:19.429312 1453 raft_consensus.cc:2804] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:19.429466 1453 raft_consensus.cc:697] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:19.429762 1453 consensus_queue.cc:237] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:19.436133 1175 ts_tablet_manager.cc:1434] T 80d443bdb66d4cc68ae283e08c410fe6 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.050s user 0.002s sys 0.000s
I20260202 02:52:19.437469 1182 raft_consensus.cc:2889] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:19.439709 1183 ts_tablet_manager.cc:1916] T 23c38ef4e3c840808ec36f859cf6f6de P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:19.455678 1182 raft_consensus.cc:2272] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:19.457137 1182 ts_tablet_manager.cc:1916] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:19.459028 1175 tablet_bootstrap.cc:492] T 75f7824681344d8d88df9402694fff1b P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
W20260202 02:52:19.465641 715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 270196us
I20260202 02:52:19.466683 1173 tablet_bootstrap.cc:492] T bf5ca99708044482970cb854d2eced18 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:19.466859 1173 ts_tablet_manager.cc:1403] T bf5ca99708044482970cb854d2eced18 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.165s user 0.011s sys 0.026s
I20260202 02:52:19.467131 1063 raft_consensus.cc:3060] T bd7b0a0fcb7645fdb9edfeccb3d18268 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:19.467664 1173 raft_consensus.cc:359] T bf5ca99708044482970cb854d2eced18 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:19.468204 1059 raft_consensus.cc:3060] T c050a2882f764dda8df2a7df6547f149 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:19.472497 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0d3633f5f9f54115bc78b9c1b78aba67" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:19.473579 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c63a670d5c5e489897bc6dbb142b844d" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:19.474598 1173 raft_consensus.cc:385] T bf5ca99708044482970cb854d2eced18 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:19.474709 1173 raft_consensus.cc:740] T bf5ca99708044482970cb854d2eced18 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:19.474895 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c184d44d7f434b39a997a7387e5d3de0" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:19.474970 1173 consensus_queue.cc:260] T bf5ca99708044482970cb854d2eced18 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:19.475519 1173 ts_tablet_manager.cc:1434] T bf5ca99708044482970cb854d2eced18 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260202 02:52:19.475832 1173 tablet_bootstrap.cc:492] T 724d47241f0342768fb1321ef37c3a7b P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
W20260202 02:52:19.478338 857 leader_election.cc:343] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:19.478637 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c050a2882f764dda8df2a7df6547f149" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:19.486779 1277 consensus_queue.cc:1048] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260202 02:52:19.489171 1173 tablet_bootstrap.cc:654] T 724d47241f0342768fb1321ef37c3a7b P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:19.488107 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ea040c03420c4b1b98897f39160b4130" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:19.490242 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0d297b6243c348ee82c146a8339cf6c4" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:19.491897 1617 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260202 02:52:19.492558 857 leader_election.cc:304] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:19.493019 1175 tablet_bootstrap.cc:654] T 75f7824681344d8d88df9402694fff1b P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:19.494403 857 raft_consensus.cc:2686] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1521a6d262b7451bbef8debe970dd6b5: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260202 02:52:19.500443 857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 60173.3us
W20260202 02:52:19.500991 857 leader_election.cc:343] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:19.501304 857 leader_election.cc:304] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
W20260202 02:52:19.501577 857 raft_consensus.cc:2686] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1521a6d262b7451bbef8debe970dd6b5: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260202 02:52:19.507614 1277 raft_consensus.cc:493] T bc1539e5c8aa4d6e98f687728b891530 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:19.509734 779 raft_consensus.cc:2468] T 0d297b6243c348ee82c146a8339cf6c4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:19.510218 1599 raft_consensus.cc:493] T c064ea0dade840b4840a5715c2d83f69 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:19.515971 1175 tablet_bootstrap.cc:492] T 75f7824681344d8d88df9402694fff1b P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:19.516156 1175 ts_tablet_manager.cc:1403] T 75f7824681344d8d88df9402694fff1b P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.078s user 0.004s sys 0.000s
I20260202 02:52:19.517068 1534 raft_consensus.cc:515] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:19.517824 1534 leader_election.cc:290] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:19.518119 1277 raft_consensus.cc:515] T bc1539e5c8aa4d6e98f687728b891530 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:19.518658 1277 leader_election.cc:290] T bc1539e5c8aa4d6e98f687728b891530 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
W20260202 02:52:19.522141 857 leader_election.cc:343] T 526f2c7b95184a0987e7aa94efc8475a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:19.522300 857 leader_election.cc:304] T 526f2c7b95184a0987e7aa94efc8475a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:19.522733 1173 tablet_bootstrap.cc:492] T 724d47241f0342768fb1321ef37c3a7b P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:19.523342 1564 raft_consensus.cc:2749] T 526f2c7b95184a0987e7aa94efc8475a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:19.523919 1175 raft_consensus.cc:359] T 75f7824681344d8d88df9402694fff1b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:19.522912 1173 ts_tablet_manager.cc:1403] T 724d47241f0342768fb1321ef37c3a7b P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.047s user 0.004s sys 0.000s
W20260202 02:52:19.525190 513 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.188:39077, user_credentials={real_user=slave}} blocked reactor thread for 80602.1us
I20260202 02:52:19.525599 1175 raft_consensus.cc:385] T 75f7824681344d8d88df9402694fff1b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260202 02:52:19.524464 857 leader_election.cc:343] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:19.526319 1173 raft_consensus.cc:359] T 724d47241f0342768fb1321ef37c3a7b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:19.526499 1173 raft_consensus.cc:385] T 724d47241f0342768fb1321ef37c3a7b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:19.526588 1173 raft_consensus.cc:740] T 724d47241f0342768fb1321ef37c3a7b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
W20260202 02:52:19.520056 1607 log.cc:927] Time spent T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d: Append to log took a long time: real 0.068s user 0.000s sys 0.000s
W20260202 02:52:19.509707 715 leader_election.cc:343] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:19.510608 1599 raft_consensus.cc:515] T c064ea0dade840b4840a5715c2d83f69 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:19.528751 715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 47451.4us
I20260202 02:52:19.526542 857 leader_election.cc:304] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:19.529486 857 leader_election.cc:304] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:19.531853 998 leader_election.cc:304] T 0d297b6243c348ee82c146a8339cf6c4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:19.532239 1178 raft_consensus.cc:2804] T 0d297b6243c348ee82c146a8339cf6c4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:19.532369 1178 raft_consensus.cc:493] T 0d297b6243c348ee82c146a8339cf6c4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:19.532466 1178 raft_consensus.cc:3060] T 0d297b6243c348ee82c146a8339cf6c4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:19.535104 1178 raft_consensus.cc:515] T 0d297b6243c348ee82c146a8339cf6c4 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:19.538666 525 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet bd7b0a0fcb7645fdb9edfeccb3d18268 (table default.Table_18fec323d6b04e10a687de6361b643b2 [id=1ffcc11561524fe8a7b09a7ddc92d5b2]). Expected version 2 got 0
I20260202 02:52:19.538489 1173 consensus_queue.cc:260] T 724d47241f0342768fb1321ef37c3a7b P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:19.541906 1178 leader_election.cc:290] T 0d297b6243c348ee82c146a8339cf6c4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:19.542773 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0d297b6243c348ee82c146a8339cf6c4" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:19.529733 1453 raft_consensus.cc:2749] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:19.526238 1175 raft_consensus.cc:740] T 75f7824681344d8d88df9402694fff1b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:19.543576 1175 consensus_queue.cc:260] T 75f7824681344d8d88df9402694fff1b P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:19.544996 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 06b80a1c9251420cbc16fc95ae98aca9 (DEFAULT_TABLE table=default.Table_84d6693620cc468e82d5cb1ec6cc69ea [id=f6625b21b6064828b11acdfdb694cd71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.545457 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06b80a1c9251420cbc16fc95ae98aca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.547222 1039 tablet_service.cc:1505] Processing CreateTablet for tablet fe53a8bdbd13417f9ce195c54a52a93c (DEFAULT_TABLE table=default.Table_ce2d44550fb5403a99372bc5d3d79265 [id=4f19275f416a425c9b9d100b2a83870a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.547644 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe53a8bdbd13417f9ce195c54a52a93c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.548612 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 3d1a7578e83447669e3c3972a426ad44 (DEFAULT_TABLE table=default.Table_7b5f2de229cb4b2f8bd174ccfd6b9553 [id=97a0437f66864a7291b966c9b72c0eb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.549038 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d1a7578e83447669e3c3972a426ad44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.551744 754 tablet_service.cc:1505] Processing CreateTablet for tablet 06b80a1c9251420cbc16fc95ae98aca9 (DEFAULT_TABLE table=default.Table_84d6693620cc468e82d5cb1ec6cc69ea [id=f6625b21b6064828b11acdfdb694cd71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.552666 1043 tablet_service.cc:1505] Processing CreateTablet for tablet d30557fbe9084cd3860820f69dda8752 (DEFAULT_TABLE table=default.Table_b6e7efe67b0e4efbb7e7c51ee5f1caa7 [id=966fa513d6e947cdb895edc05d26ff65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.553114 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d30557fbe9084cd3860820f69dda8752. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.553114 755 tablet_service.cc:1505] Processing CreateTablet for tablet fe53a8bdbd13417f9ce195c54a52a93c (DEFAULT_TABLE table=default.Table_ce2d44550fb5403a99372bc5d3d79265 [id=4f19275f416a425c9b9d100b2a83870a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.553642 758 tablet_service.cc:1505] Processing CreateTablet for tablet fe9ea4c0502f4976b440465992f2ea2c (DEFAULT_TABLE table=default.Table_9181c8b857b540ff846c397f9f1a5484 [id=d2fb3868962547c48e29f6353e5747a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.554142 749 tablet_service.cc:1505] Processing CreateTablet for tablet 16b47a2831f54d809ed0b16deaec7e3d (DEFAULT_TABLE table=default.Table_9dfc2d7b50e94228b7aad2a3f575f625 [id=1e291b35a6764638acfb781bb4e6adb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.554637 751 tablet_service.cc:1505] Processing CreateTablet for tablet 26eeef6eebbf4f2cb44f69255fb37594 (DEFAULT_TABLE table=default.Table_82c14c20a7bf41b487efaff52ebc15a9 [id=7fbb33c105f149ef98505dcf622930c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.556668 1038 tablet_service.cc:1505] Processing CreateTablet for tablet 26eeef6eebbf4f2cb44f69255fb37594 (DEFAULT_TABLE table=default.Table_82c14c20a7bf41b487efaff52ebc15a9 [id=7fbb33c105f149ef98505dcf622930c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.557070 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26eeef6eebbf4f2cb44f69255fb37594. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.560835 1034 tablet_service.cc:1505] Processing CreateTablet for tablet 16b47a2831f54d809ed0b16deaec7e3d (DEFAULT_TABLE table=default.Table_9dfc2d7b50e94228b7aad2a3f575f625 [id=1e291b35a6764638acfb781bb4e6adb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.561290 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16b47a2831f54d809ed0b16deaec7e3d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:19.562408 715 leader_election.cc:343] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:19.563823 779 raft_consensus.cc:3060] T 0d297b6243c348ee82c146a8339cf6c4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:19.563786 715 leader_election.cc:343] T c184d44d7f434b39a997a7387e5d3de0 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:19.564699 1042 tablet_service.cc:1505] Processing CreateTablet for tablet fe9ea4c0502f4976b440465992f2ea2c (DEFAULT_TABLE table=default.Table_9181c8b857b540ff846c397f9f1a5484 [id=d2fb3868962547c48e29f6353e5747a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.565106 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe9ea4c0502f4976b440465992f2ea2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.568703 1040 tablet_service.cc:1505] Processing CreateTablet for tablet 02b434fa058b45529ff3938b053263c6 (DEFAULT_TABLE table=default.Table_6133c617ff8d40eca5cbde96533acd82 [id=5a502f86d98b42c6a3e274de5bf08544]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.569152 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02b434fa058b45529ff3938b053263c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.569953 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_2107cef8ed314cad9f2c7c115e2baedc" } new_table_name: "default.Table_11d7a6e6268042a7a26c8ecc00bc9f6e" modify_external_catalogs: true
W20260202 02:52:19.571426 520 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d30557fbe9084cd3860820f69dda8752 consensus state has no leader
I20260202 02:52:19.571822 520 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_26642341d3d54c0fa9af19fef3a5b06d" } modify_external_catalogs: true
I20260202 02:52:19.572002 520 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_26642341d3d54c0fa9af19fef3a5b06d" } modify_external_catalogs: true
I20260202 02:52:19.572906 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_22c52df5d2c0467ead479c202dfd2805"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:19.573514 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_22c52df5d2c0467ead479c202dfd2805 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:19.573637 522 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 75f7824681344d8d88df9402694fff1b
I20260202 02:52:19.574539 522 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_38c5bca682e343c1be1ebdc253c9e848" } modify_external_catalogs: true
I20260202 02:52:19.574730 522 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_38c5bca682e343c1be1ebdc253c9e848" } modify_external_catalogs: true
I20260202 02:52:19.574869 1039 tablet_service.cc:1552] Processing DeleteTablet for tablet 75f7824681344d8d88df9402694fff1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:19.575484 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_524aa17b0b964c9e985e643ee9f12e94"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:19.576097 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_524aa17b0b964c9e985e643ee9f12e94 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:19.576382 523 catalog_manager.cc:5654] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130). New cstate: current_term: 1 leader_uuid: "1521a6d262b7451bbef8debe970dd6b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: HEALTHY } } }
I20260202 02:52:19.576822 523 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 583e547d0a1943f09f4efe82070fc777 (table default.Table_2669ca9662b942c08bcbfd7641cfe2e3 [id=fb813e4c5b0849f8a488c7a22d1defae]). Expected version 1 got 0
I20260202 02:52:19.577194 523 catalog_manager.cc:5654] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130). New cstate: current_term: 1 leader_uuid: "1521a6d262b7451bbef8debe970dd6b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: HEALTHY } } }
I20260202 02:52:19.577479 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_242f4cdb6f514121a1a113910a2aaec8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:19.578047 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_242f4cdb6f514121a1a113910a2aaec8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:19.578855 526 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 723921dfe54c4e66af5cab76a25fee0a
I20260202 02:52:19.579532 526 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_086abcdd67e94b38ac7f5d98777f5c6b" } modify_external_catalogs: true
I20260202 02:52:19.579684 526 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_086abcdd67e94b38ac7f5d98777f5c6b" } modify_external_catalogs: true
I20260202 02:52:19.580082 1039 tablet_service.cc:1552] Processing DeleteTablet for tablet 723921dfe54c4e66af5cab76a25fee0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:19.583683 519 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.104.129:48435 (request call id 32) took 924 ms (client timeout 1000 ms). Trace:
I20260202 02:52:19.587121 528 master_service.cc:954] ReplaceTablet: received request to replace tablet d1e0e74f32a24729b250c5751677cfb0 from {username='slave'} at 127.0.0.1:47430
W20260202 02:52:19.589471 527 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 874ca4ba4ebc4721bb4b60cba8e01841 has no consensus state
W20260202 02:52:19.583961 519 rpcz_store.cc:269] 0202 02:52:18.658910 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:18.662812 (+ 3902us) service_pool.cc:225] Handling call
0202 02:52:19.583614 (+920802us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:18.769534 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:18.769626 (+ 92us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:18.769631 (+ 5us) write_op.cc:435] Acquired schema lock
0202 02:52:18.769633 (+ 2us) tablet.cc:660] Decoding operations
0202 02:52:18.769779 (+ 146us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:18.769788 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:18.769793 (+ 5us) tablet.cc:683] Acquiring locks for 3 operations
0202 02:52:18.769867 (+ 74us) tablet.cc:699] Row locks acquired
0202 02:52:18.769870 (+ 3us) write_op.cc:260] PREPARE: finished
0202 02:52:18.769902 (+ 32us) write_op.cc:270] Start()
0202 02:52:18.769929 (+ 27us) write_op.cc:276] Timestamp: P: 1770000738769847 usec, L: 0
0202 02:52:18.769932 (+ 3us) op_driver.cc:348] REPLICATION: starting
0202 02:52:18.770088 (+ 156us) log.cc:844] Serialized 1293 byte log entry
0202 02:52:19.160276 (+390188us) op_driver.cc:464] REPLICATION: finished
0202 02:52:19.161883 (+ 1607us) write_op.cc:301] APPLY: starting
0202 02:52:19.161894 (+ 11us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:19.161929 (+ 35us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:19.161960 (+ 31us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:19.162031 (+ 71us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:19.162057 (+ 26us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0202 02:52:19.162065 (+ 8us) write_op.cc:312] APPLY: finished
0202 02:52:19.162118 (+ 53us) log.cc:844] Serialized 35 byte log entry
0202 02:52:19.162154 (+ 36us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:19.162168 (+ 14us) write_op.cc:454] Released schema lock
0202 02:52:19.162174 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0202 02:52:19.162231 (+ 57us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:19.162235 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":18,"child_traces":[["op",{"apply.queue_time_us":1579,"apply.run_cpu_time_us":340,"apply.run_wall_time_us":340,"num_ops":3,"prepare.queue_time_us":83569,"prepare.run_cpu_time_us":644,"prepare.run_wall_time_us":643,"replication_time_us":390331}]]}
I20260202 02:52:19.589802 527 master_service.cc:954] ReplaceTablet: received request to replace tablet 61a9203035f2495f8f99924303ebf120 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:19.592126 519 master_service.cc:954] ReplaceTablet: received request to replace tablet c12ff20ca6b44ddba0c936a28cfb2eb9 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:19.599907 1180 ts_tablet_manager.cc:1929] T bf5ca99708044482970cb854d2eced18 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:19.606050 753 tablet_service.cc:1505] Processing CreateTablet for tablet 3d1a7578e83447669e3c3972a426ad44 (DEFAULT_TABLE table=default.Table_7b5f2de229cb4b2f8bd174ccfd6b9553 [id=97a0437f66864a7291b966c9b72c0eb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.606215 759 tablet_service.cc:1505] Processing CreateTablet for tablet d30557fbe9084cd3860820f69dda8752 (DEFAULT_TABLE table=default.Table_b6e7efe67b0e4efbb7e7c51ee5f1caa7 [id=966fa513d6e947cdb895edc05d26ff65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.606531 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06b80a1c9251420cbc16fc95ae98aca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.606614 748 tablet_service.cc:1552] Processing DeleteTablet for tablet 75f7824681344d8d88df9402694fff1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:19.606943 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe53a8bdbd13417f9ce195c54a52a93c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.607295 1180 log.cc:1199] T bf5ca99708044482970cb854d2eced18 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/bf5ca99708044482970cb854d2eced18
I20260202 02:52:19.607756 1180 ts_tablet_manager.cc:1950] T bf5ca99708044482970cb854d2eced18 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:19.608615 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 220) took 1017 ms. Trace:
I20260202 02:52:19.608893 1564 raft_consensus.cc:493] T 0186572e7bb9499a932a636ab280f544 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:19.609087 1564 raft_consensus.cc:515] T 0186572e7bb9499a932a636ab280f544 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:19.608741 1180 rpcz_store.cc:276] 0202 02:52:18.591022 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:18.591273 (+ 251us) service_pool.cc:225] Handling call
0202 02:52:19.250946 (+659673us) ts_tablet_manager.cc:1151] Deleting tablet bf5ca99708044482970cb854d2eced18
0202 02:52:19.250957 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:19.492972 (+242015us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:19.599740 (+106768us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:19.608549 (+ 8809us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":659449}
I20260202 02:52:19.609731 1564 leader_election.cc:290] T 0186572e7bb9499a932a636ab280f544 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:19.609820 1180 tablet_replica.cc:333] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:19.610030 1180 raft_consensus.cc:2243] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:19.610636 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0186572e7bb9499a932a636ab280f544" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:19.611207 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d30557fbe9084cd3860820f69dda8752. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.611778 1173 ts_tablet_manager.cc:1434] T 724d47241f0342768fb1321ef37c3a7b P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.087s user 0.001s sys 0.004s
I20260202 02:52:19.612110 1173 tablet_bootstrap.cc:492] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:19.612114 1175 ts_tablet_manager.cc:1434] T 75f7824681344d8d88df9402694fff1b P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.096s user 0.002s sys 0.001s
I20260202 02:52:19.612421 1175 tablet_bootstrap.cc:492] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:19.613018 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16b47a2831f54d809ed0b16deaec7e3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.614368 1173 tablet_bootstrap.cc:654] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:19.614365 1175 tablet_bootstrap.cc:654] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:19.544220 857 leader_election.cc:304] T d06b77ff1a6f4f78b73082b00145fcf6 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:19.544422 1453 raft_consensus.cc:2804] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:19.616011 1453 raft_consensus.cc:493] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:19.616127 1453 raft_consensus.cc:3060] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:19.616161 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d1a7578e83447669e3c3972a426ad44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.617992 1624 raft_consensus.cc:493] T bd7b0a0fcb7645fdb9edfeccb3d18268 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:19.618726 1453 raft_consensus.cc:515] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:19.618590 1624 raft_consensus.cc:515] T bd7b0a0fcb7645fdb9edfeccb3d18268 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:19.619453 1624 leader_election.cc:290] T bd7b0a0fcb7645fdb9edfeccb3d18268 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:19.620486 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe9ea4c0502f4976b440465992f2ea2c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:19.621107 859 leader_election.cc:343] T 0186572e7bb9499a932a636ab280f544 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:19.621191 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26eeef6eebbf4f2cb44f69255fb37594. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.628800 1173 tablet_bootstrap.cc:492] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:19.629132 1173 ts_tablet_manager.cc:1403] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.001s
W20260202 02:52:19.629798 715 consensus_peers.cc:597] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ea040c03420c4b1b98897f39160b4130. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:19.629926 756 tablet_service.cc:1505] Processing CreateTablet for tablet 02b434fa058b45529ff3938b053263c6 (DEFAULT_TABLE table=default.Table_6133c617ff8d40eca5cbde96533acd82 [id=5a502f86d98b42c6a3e274de5bf08544]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.630873 1173 raft_consensus.cc:359] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:19.631673 1173 raft_consensus.cc:385] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:19.631862 1173 raft_consensus.cc:740] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:19.631882 1564 raft_consensus.cc:2804] T d06b77ff1a6f4f78b73082b00145fcf6 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election won for term 1
W20260202 02:52:19.631938 857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 87715.2us
I20260202 02:52:19.632056 1564 raft_consensus.cc:697] T d06b77ff1a6f4f78b73082b00145fcf6 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:19.632486 1564 consensus_queue.cc:237] T d06b77ff1a6f4f78b73082b00145fcf6 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:19.634203 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet d67e91a022f1401ea724dff898c3b81b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d67e91a022f1401ea724dff898c3b81b already in progress: creating tablet
I20260202 02:52:19.632308 1173 consensus_queue.cc:260] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:19.635442 999 leader_election.cc:304] T 61a9203035f2495f8f99924303ebf120 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:19.636121 999 leader_election.cc:304] T ffd8aa2bc6e344b78dce518411997fb5 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
W20260202 02:52:19.636405 999 raft_consensus.cc:2686] T ffd8aa2bc6e344b78dce518411997fb5 P b1f05820b6034228bdad884cc348716d: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260202 02:52:19.636523 1624 raft_consensus.cc:2749] T 61a9203035f2495f8f99924303ebf120 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:19.637202 999 leader_election.cc:343] T ea040c03420c4b1b98897f39160b4130 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: ea040c03420c4b1b98897f39160b4130
I20260202 02:52:19.637668 999 leader_election.cc:304] T ea040c03420c4b1b98897f39160b4130 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
W20260202 02:52:19.637867 999 raft_consensus.cc:2686] T ea040c03420c4b1b98897f39160b4130 P b1f05820b6034228bdad884cc348716d: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260202 02:52:19.631697 1183 ts_tablet_manager.cc:1929] T 23c38ef4e3c840808ec36f859cf6f6de P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:19.643410 1183 log.cc:1199] T 23c38ef4e3c840808ec36f859cf6f6de P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/23c38ef4e3c840808ec36f859cf6f6de
I20260202 02:52:19.643802 1183 ts_tablet_manager.cc:1950] T 23c38ef4e3c840808ec36f859cf6f6de P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:19.646761 524 master_service.cc:954] ReplaceTablet: received request to replace tablet 267bd4752c5f4b99a3fe778ba8f38a2f from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:19.647302 1183 tablet_replica.cc:333] T 0d297b6243c348ee82c146a8339cf6c4 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:19.647532 1183 raft_consensus.cc:2243] T 0d297b6243c348ee82c146a8339cf6c4 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:19.647739 1183 raft_consensus.cc:2272] T 0d297b6243c348ee82c146a8339cf6c4 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:19.648352 1183 ts_tablet_manager.cc:1916] T 0d297b6243c348ee82c146a8339cf6c4 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:19.649482 521 master_service.cc:954] ReplaceTablet: received request to replace tablet d2fa4c3d6ec44047abf5986e502c1590 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:19.650266 524 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_addc8f2e088548c99dbfaf1ae14135d0" } new_table_name: "default.Table_53cc5731c3e24222a97702bdde84a954" modify_external_catalogs: true
I20260202 02:52:19.650343 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_7a314585590647ca8fd0f51e125b0622"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:19.650959 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7a314585590647ca8fd0f51e125b0622 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:19.652626 1182 ts_tablet_manager.cc:1929] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:19.652801 1182 log.cc:1199] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/0b411e8fab074aa88f8380e3f5694340
I20260202 02:52:19.653373 1182 ts_tablet_manager.cc:1950] T 0b411e8fab074aa88f8380e3f5694340 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
W20260202 02:52:19.653889 715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 65876.7us
I20260202 02:52:19.654349 1182 tablet_replica.cc:333] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:19.654552 1182 raft_consensus.cc:2243] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:19.654708 1182 raft_consensus.cc:2272] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:19.655500 1453 leader_election.cc:290] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:19.655536 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bc1539e5c8aa4d6e98f687728b891530" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:19.655646 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bbf31e5619ed4f60b4d7c8b19ec8a549" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:19.655944 1057 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bbf31e5619ed4f60b4d7c8b19ec8a549" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
W20260202 02:52:19.656694 859 leader_election.cc:343] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:19.657519 999 leader_election.cc:304] T c050a2882f764dda8df2a7df6547f149 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:19.657830 1624 raft_consensus.cc:2749] T c050a2882f764dda8df2a7df6547f149 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:19.658123 526 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 94bada52db6a47179415d302b541d01f
I20260202 02:52:19.658030 1183 ts_tablet_manager.cc:1929] T 0d297b6243c348ee82c146a8339cf6c4 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:19.658854 1057 raft_consensus.cc:3060] T 61a9203035f2495f8f99924303ebf120 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:19.658936 1183 log.cc:1199] T 0d297b6243c348ee82c146a8339cf6c4 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/0d297b6243c348ee82c146a8339cf6c4
I20260202 02:52:19.658885 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_12db441a8c934f9780599755191c58f5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:19.659328 1183 ts_tablet_manager.cc:1950] T 0d297b6243c348ee82c146a8339cf6c4 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:19.660074 1183 tablet_replica.cc:333] T 952d760159844ad08269654264e6381b P b1f05820b6034228bdad884cc348716d: stopping tablet replica
W20260202 02:52:19.660193 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_12db441a8c934f9780599755191c58f5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:19.660287 1183 raft_consensus.cc:2243] T 952d760159844ad08269654264e6381b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:19.660486 1183 raft_consensus.cc:2272] T 952d760159844ad08269654264e6381b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:19.661059 1183 ts_tablet_manager.cc:1916] T 952d760159844ad08269654264e6381b P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:19.661904 520 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 59990791123c4d4eb620090a1e8e31e1
I20260202 02:52:19.662355 522 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet ffd80badce1048f3ac95366cad05ae53
W20260202 02:52:19.662727 859 consensus_peers.cc:597] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5 -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:19.663121 859 consensus_peers.cc:597] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5 -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:19.663055 522 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_99b75b61234e40289fc183ba92b187e8" } new_table_name: "default.Table_c37ce6d5cf70417c83a68c31d02881ef" modify_external_catalogs: true
I20260202 02:52:19.663851 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 23c38ef4e3c840808ec36f859cf6f6de (table default.Table_9fd715491a6a4bbc8685138d0194362d [id=98ddcbfb5ad14ae480a54845ec7c45a4]) successfully deleted
I20260202 02:52:19.664525 919 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0d3633f5f9f54115bc78b9c1b78aba67" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:19.664726 919 raft_consensus.cc:2468] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:19.665462 1037 tablet_service.cc:1552] Processing DeleteTablet for tablet bf5ca99708044482970cb854d2eced18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:15 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:19.666173 919 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c63a670d5c5e489897bc6dbb142b844d" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:19.666219 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet ffd80badce1048f3ac95366cad05ae53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:19.666356 919 raft_consensus.cc:2468] T c63a670d5c5e489897bc6dbb142b844d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:19.667411 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 94bada52db6a47179415d302b541d01f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:19.670622 1183 ts_tablet_manager.cc:1929] T 952d760159844ad08269654264e6381b P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:19.670872 1183 log.cc:1199] T 952d760159844ad08269654264e6381b P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/952d760159844ad08269654264e6381b
I20260202 02:52:19.671298 1183 ts_tablet_manager.cc:1950] T 952d760159844ad08269654264e6381b P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:19.672410 1414 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "b1f05820b6034228bdad884cc348716d" instance_seqno: 1770000733441128) as {username='slave'} at 127.0.104.131:39717; Asking this server to re-register.
I20260202 02:52:19.673218 1183 tablet_replica.cc:333] T bf5ca99708044482970cb854d2eced18 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:19.673178 1422 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "1f442ef85029461091de847118c34c0d" instance_seqno: 1770000732390773) as {username='slave'} at 127.0.104.129:33337; Asking this server to re-register.
I20260202 02:52:19.673430 1183 raft_consensus.cc:2243] T bf5ca99708044482970cb854d2eced18 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:19.673594 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 0d297b6243c348ee82c146a8339cf6c4 (table default.Table_0c83932cab13463f9cca28129f18b576 [id=7c76848b98614fbf8b2950e5d55cc609]) successfully deleted
I20260202 02:52:19.673645 1183 raft_consensus.cc:2272] T bf5ca99708044482970cb854d2eced18 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:19.674046 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 267bd4752c5f4b99a3fe778ba8f38a2f (table default.Table_dda472e63fac4873ab3ccefcac3635cd [id=40105de1d8074d3e983ff62557d14f48]) successfully deleted
I20260202 02:52:19.674198 1109 heartbeater.cc:461] Registering TS with master...
I20260202 02:52:19.674256 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 952d760159844ad08269654264e6381b (table default.Table_b6c1fb1e7282492d920385869810b0e0 [id=2cc4567c7bd44745a5851cd8b95b22c4]) successfully deleted
I20260202 02:52:19.674688 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 59990791123c4d4eb620090a1e8e31e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:19.674774 1173 ts_tablet_manager.cc:1434] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.043s user 0.001s sys 0.002s
I20260202 02:52:19.675194 1183 ts_tablet_manager.cc:1916] T bf5ca99708044482970cb854d2eced18 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:19.677804 1173 tablet_bootstrap.cc:492] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:19.679452 1063 raft_consensus.cc:3060] T 5986676f5484498da927b4c6da3f47ff P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:19.681279 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c064ea0dade840b4840a5715c2d83f69" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:19.681442 1182 ts_tablet_manager.cc:1916] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:19.682858 1175 tablet_bootstrap.cc:492] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:19.683091 1175 ts_tablet_manager.cc:1403] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.071s user 0.003s sys 0.000s
I20260202 02:52:19.683411 752 tablet_service.cc:1552] Processing DeleteTablet for tablet fe7b6004e5094c0dbe5847144a2489b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:19.684504 755 tablet_service.cc:1552] Processing DeleteTablet for tablet 723921dfe54c4e66af5cab76a25fee0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:42950
W20260202 02:52:19.685514 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 75f7824681344d8d88df9402694fff1b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 75f7824681344d8d88df9402694fff1b already in progress: creating tablet
W20260202 02:52:19.685812 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 723921dfe54c4e66af5cab76a25fee0a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 723921dfe54c4e66af5cab76a25fee0a already in progress: creating tablet
I20260202 02:52:19.686070 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02b434fa058b45529ff3938b053263c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.686630 1599 leader_election.cc:290] T c064ea0dade840b4840a5715c2d83f69 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:19.687604 523 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_c09bacb95cba44a3ad80c21c0a77b6e0" } new_table_name: "default.Table_63e36ff851f849849172c3c8e473214e" modify_external_catalogs: true
I20260202 02:52:19.690716 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bd7b0a0fcb7645fdb9edfeccb3d18268" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:19.692059 528 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet d1e0e74f32a24729b250c5751677cfb0
I20260202 02:52:19.692067 1175 raft_consensus.cc:359] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:19.692382 1175 raft_consensus.cc:385] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:19.692463 1175 raft_consensus.cc:740] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:19.692667 528 catalog_manager.cc:6724] ReplaceTablet: tablet d1e0e74f32a24729b250c5751677cfb0 deleted and replaced by tablet 2c31b45dc8004eae993e972e9faa3ff8
I20260202 02:52:19.692724 1175 consensus_queue.cc:260] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:19.693272 1175 ts_tablet_manager.cc:1434] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20260202 02:52:19.693645 1175 tablet_bootstrap.cc:492] T 757492288d7c44fb91b25b7e6e21e01b P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:19.694605 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_2d42dcf81d2d4b2c90fe09860b9b2318"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:19.696105 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet d1e0e74f32a24729b250c5751677cfb0 with delete_type TABLET_DATA_DELETED (replaced by tablet 2c31b45dc8004eae993e972e9faa3ff8) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:19.696436 527 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 61a9203035f2495f8f99924303ebf120
I20260202 02:52:19.698465 519 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet c12ff20ca6b44ddba0c936a28cfb2eb9
I20260202 02:52:19.700209 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet c12ff20ca6b44ddba0c936a28cfb2eb9 with delete_type TABLET_DATA_DELETED (replaced by tablet 072bb7f0e4ab4935be49461bc8a3c163) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:19.700641 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet 61a9203035f2495f8f99924303ebf120 with delete_type TABLET_DATA_DELETED (replaced by tablet 4c63192b273b462a9c1a0e530e938024) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:19.700758 1180 pending_rounds.cc:70] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d: Trying to abort 1 pending ops.
W20260202 02:52:19.700867 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2d42dcf81d2d4b2c90fe09860b9b2318 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:19.701025 527 catalog_manager.cc:6724] ReplaceTablet: tablet 61a9203035f2495f8f99924303ebf120 deleted and replaced by tablet 4c63192b273b462a9c1a0e530e938024
I20260202 02:52:19.705780 1183 ts_tablet_manager.cc:1929] T bf5ca99708044482970cb854d2eced18 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:19.706040 1183 log.cc:1199] T bf5ca99708044482970cb854d2eced18 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/bf5ca99708044482970cb854d2eced18
W20260202 02:52:19.706305 513 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet ff6f5710de4b4c48948aacdf5be29e8b (table default.Table_ad048ed673a5401b8ffa739c0f03b545 [id=f2276e1c2ca04a70804053da61dd3447], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet ff6f5710de4b4c48948aacdf5be29e8b consensus state has no leader
I20260202 02:52:19.706667 1183 ts_tablet_manager.cc:1950] T bf5ca99708044482970cb854d2eced18 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:19.708470 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 724d47241f0342768fb1321ef37c3a7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:16 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:19.708513 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet bf5ca99708044482970cb854d2eced18 (table default.Table_34c95b3937dd4587a9bd81a7c805f562 [id=8227929c83a441af86babfb3cff579bd]) successfully deleted
W20260202 02:52:19.708772 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet ffd80badce1048f3ac95366cad05ae53 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ffd80badce1048f3ac95366cad05ae53 already in progress: creating tablet
I20260202 02:52:19.709137 519 catalog_manager.cc:6724] ReplaceTablet: tablet c12ff20ca6b44ddba0c936a28cfb2eb9 deleted and replaced by tablet 072bb7f0e4ab4935be49461bc8a3c163
W20260202 02:52:19.709286 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 94bada52db6a47179415d302b541d01f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 94bada52db6a47179415d302b541d01f already in progress: creating tablet
W20260202 02:52:19.710265 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 59990791123c4d4eb620090a1e8e31e1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 59990791123c4d4eb620090a1e8e31e1 already in progress: creating tablet
I20260202 02:52:19.715636 824 heartbeater.cc:461] Registering TS with master...
W20260202 02:52:19.715669 715 leader_election.cc:343] T bc1539e5c8aa4d6e98f687728b891530 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:19.715967 524 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d0671586e45e45ffb7fd9e1be6e2bbf8 consensus state has no leader
W20260202 02:52:19.716408 522 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b428356fba0c4fb5a6033c5d39740cc5 consensus state has no leader
I20260202 02:52:19.717260 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bbf31e5619ed4f60b4d7c8b19ec8a549" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:19.717398 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 9884c90cf1d245b2a7eaaf00d71b1c3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:19.718034 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 526f2c7b95184a0987e7aa94efc8475a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:19.719151 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet d1e0e74f32a24729b250c5751677cfb0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d1e0e74f32a24729b250c5751677cfb0 already in progress: creating tablet
I20260202 02:52:19.719499 886 tablet_service.cc:1505] Processing CreateTablet for tablet 06b80a1c9251420cbc16fc95ae98aca9 (DEFAULT_TABLE table=default.Table_84d6693620cc468e82d5cb1ec6cc69ea [id=f6625b21b6064828b11acdfdb694cd71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.719622 1183 tablet_replica.cc:333] T 61a9203035f2495f8f99924303ebf120 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:19.719889 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06b80a1c9251420cbc16fc95ae98aca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.725703 521 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet d2fa4c3d6ec44047abf5986e502c1590
I20260202 02:52:19.726243 521 catalog_manager.cc:6724] ReplaceTablet: tablet d2fa4c3d6ec44047abf5986e502c1590 deleted and replaced by tablet a0b5d82e5fb94b60b4aa324b978a5378
I20260202 02:52:19.725881 1180 pending_rounds.cc:77] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923021633286144 op_type: NO_OP noop_request { }
I20260202 02:52:19.726537 755 tablet_service.cc:1552] Processing DeleteTablet for tablet c064ea0dade840b4840a5715c2d83f69 with delete_type TABLET_DATA_DELETED (replaced by tablet 622a113af04a41c6852c29fc8a8acd78) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:19.727604 752 tablet_service.cc:1552] Processing DeleteTablet for tablet f58a7b25815c4b5bac568aa11c19610c with delete_type TABLET_DATA_DELETED (replaced by tablet cb634d4bae554ef3b39eeaa9958adc6c) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:19.727556 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "eaa47f3388144e8b852804c167f232ce" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:19.690023 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b428356fba0c4fb5a6033c5d39740cc5" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:19.726014 715 leader_election.cc:343] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:19.729142 1174 tablet_bootstrap.cc:492] T 70306ed9dec54286885d965f2c012da4 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:19.729202 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet d2fa4c3d6ec44047abf5986e502c1590 with delete_type TABLET_DATA_DELETED (replaced by tablet a0b5d82e5fb94b60b4aa324b978a5378) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:19.730058 752 tablet_service.cc:1552] Processing DeleteTablet for tablet 94bada52db6a47179415d302b541d01f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:19.733105 1182 ts_tablet_manager.cc:1929] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:19.733265 1182 log.cc:1199] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/bbf31e5619ed4f60b4d7c8b19ec8a549
I20260202 02:52:19.733592 1182 ts_tablet_manager.cc:1950] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:19.735797 754 tablet_service.cc:1552] Processing DeleteTablet for tablet c184d44d7f434b39a997a7387e5d3de0 with delete_type TABLET_DATA_DELETED (replaced by tablet 3711910e050c4515bed4294d1271d5a9) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:19.736239 1174 ts_tablet_manager.cc:1403] T 70306ed9dec54286885d965f2c012da4 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.514s user 0.018s sys 0.006s
I20260202 02:52:19.737092 886 tablet_service.cc:1505] Processing CreateTablet for tablet 3d1a7578e83447669e3c3972a426ad44 (DEFAULT_TABLE table=default.Table_7b5f2de229cb4b2f8bd174ccfd6b9553 [id=97a0437f66864a7291b966c9b72c0eb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.737576 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d1a7578e83447669e3c3972a426ad44. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:19.738071 715 consensus_peers.cc:597] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:19.738296 715 leader_election.cc:343] T c064ea0dade840b4840a5715c2d83f69 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:19.738591 527 catalog_manager.cc:5654] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130). New cstate: current_term: 1 leader_uuid: "1521a6d262b7451bbef8debe970dd6b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: HEALTHY } } }
I20260202 02:52:19.739056 527 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet b76d1837813c46e8a0925b49939d8884 (table default.Table_3919d090261e4b90b0672f2b863588b2 [id=fe7b88b74c864c2f98365c0956117bf1]). Expected version 1 got 0
I20260202 02:52:19.739312 527 catalog_manager.cc:5654] T d06b77ff1a6f4f78b73082b00145fcf6 P 1521a6d262b7451bbef8debe970dd6b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130). New cstate: current_term: 1 leader_uuid: "1521a6d262b7451bbef8debe970dd6b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: HEALTHY } } }
I20260202 02:52:19.739609 527 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet d06b77ff1a6f4f78b73082b00145fcf6 (table default.Table_c09bacb95cba44a3ad80c21c0a77b6e0 [id=915b211258ad498286448a5b2598a289]). Expected version 2 got 0
I20260202 02:52:19.739928 527 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet d2d27ceea1bb4122b18f0fcf4d2d1bf4 (table default.Table_10aa0dafec0547069eb92ecf12b3c780 [id=1f9c48c8e2484856a66a0719268f22ae]). Expected version 1 got 0
I20260202 02:52:19.742717 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_0a56eb8a4a2944a495e4b076b63c0e19"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:19.744870 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0a56eb8a4a2944a495e4b076b63c0e19 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:19.744902 523 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d06b77ff1a6f4f78b73082b00145fcf6 consensus state has no leader
I20260202 02:52:19.745293 894 tablet_service.cc:1505] Processing CreateTablet for tablet d30557fbe9084cd3860820f69dda8752 (DEFAULT_TABLE table=default.Table_b6e7efe67b0e4efbb7e7c51ee5f1caa7 [id=966fa513d6e947cdb895edc05d26ff65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.745723 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d30557fbe9084cd3860820f69dda8752. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.748494 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_d1815e87de114d408122224276266f69"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:19.749150 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d1815e87de114d408122224276266f69 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:19.750319 1180 raft_consensus.cc:2889] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:19.750463 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet bf5ca99708044482970cb854d2eced18 (table default.Table_34c95b3937dd4587a9bd81a7c805f562 [id=8227929c83a441af86babfb3cff579bd]) successfully deleted
W20260202 02:52:19.752519 715 consensus_peers.cc:597] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:19.752774 715 consensus_peers.cc:597] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:19.753854 889 tablet_service.cc:1505] Processing CreateTablet for tablet 16b47a2831f54d809ed0b16deaec7e3d (DEFAULT_TABLE table=default.Table_9dfc2d7b50e94228b7aad2a3f575f625 [id=1e291b35a6764638acfb781bb4e6adb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.754281 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16b47a2831f54d809ed0b16deaec7e3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.754295 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 26b890ed87c3471b9fc0b172c90e0a11 (DEFAULT_TABLE table=default.Table_0c2fc83ca5324ab3b8b56c2aaf42864c [id=d2e4220a0cbf443e8380433da4f064f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.754791 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26b890ed87c3471b9fc0b172c90e0a11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.755923 1634 mvcc.cc:204] Tried to move back new op lower bound from 7249923028573380608 to 7249923022097240064. Current Snapshot: MvccSnapshot[applied={T|T < 7249923028573380608}]
I20260202 02:52:19.757496 754 tablet_service.cc:1552] Processing DeleteTablet for tablet ffd80badce1048f3ac95366cad05ae53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:19.757705 752 tablet_service.cc:1552] Processing DeleteTablet for tablet 59990791123c4d4eb620090a1e8e31e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:19.761682 894 tablet_service.cc:1505] Processing CreateTablet for tablet 26eeef6eebbf4f2cb44f69255fb37594 (DEFAULT_TABLE table=default.Table_82c14c20a7bf41b487efaff52ebc15a9 [id=7fbb33c105f149ef98505dcf622930c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.762004 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0186572e7bb9499a932a636ab280f544" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:19.762547 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26eeef6eebbf4f2cb44f69255fb37594. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.766937 521 catalog_manager.cc:5654] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f442ef85029461091de847118c34c0d (127.0.104.129). New cstate: current_term: 1 leader_uuid: "1f442ef85029461091de847118c34c0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } }
I20260202 02:52:19.767562 521 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet ff6f5710de4b4c48948aacdf5be29e8b (table default.Table_ad048ed673a5401b8ffa739c0f03b545 [id=f2276e1c2ca04a70804053da61dd3447]). Expected version 2 got 0
I20260202 02:52:19.767814 521 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet d2d27ceea1bb4122b18f0fcf4d2d1bf4 (table default.Table_10aa0dafec0547069eb92ecf12b3c780 [id=1f9c48c8e2484856a66a0719268f22ae]). Expected version 1 got 0
I20260202 02:52:19.768095 521 catalog_manager.cc:5654] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f442ef85029461091de847118c34c0d (127.0.104.129). New cstate: current_term: 1 leader_uuid: "1f442ef85029461091de847118c34c0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } }
I20260202 02:52:19.776253 528 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_e70b044a267b465d94fede069f2f0df6" } new_table_name: "default.Table_f97ead9efdf047a2a4fc35c017853fb5" modify_external_catalogs: true
I20260202 02:52:19.779774 757 tablet_service.cc:1552] Processing DeleteTablet for tablet c12ff20ca6b44ddba0c936a28cfb2eb9 with delete_type TABLET_DATA_DELETED (replaced by tablet 072bb7f0e4ab4935be49461bc8a3c163) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:19.781853 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_5ecb81038c9f48438e74fe874fc83e74"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:19.787632 522 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_b3001bd4a86843ada9ee50b8e4915a9a" } modify_external_catalogs: true
I20260202 02:52:19.787916 522 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_b3001bd4a86843ada9ee50b8e4915a9a" } modify_external_catalogs: true
I20260202 02:52:19.788929 1038 tablet_service.cc:1505] Processing CreateTablet for tablet be993c1512b04df4b92cab3195241d84 (DEFAULT_TABLE table=default.Table_b13428495dde400e83de7d1032559c35 [id=22f1ff6e13a84a72820c887dd3107f49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.789295 524 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_6133c617ff8d40eca5cbde96533acd82" } modify_external_catalogs: true
I20260202 02:52:19.789498 524 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_6133c617ff8d40eca5cbde96533acd82" } modify_external_catalogs: true
I20260202 02:52:19.789841 1173 tablet_bootstrap.cc:654] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:19.790838 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_4532505bd70548f7b20362dc9629dbee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:19.791445 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4532505bd70548f7b20362dc9629dbee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:19.795850 750 tablet_service.cc:1552] Processing DeleteTablet for tablet 61a9203035f2495f8f99924303ebf120 with delete_type TABLET_DATA_DELETED (replaced by tablet 4c63192b273b462a9c1a0e530e938024) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:19.796083 753 tablet_service.cc:1552] Processing DeleteTablet for tablet bbf31e5619ed4f60b4d7c8b19ec8a549 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:16 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:19.796620 1180 raft_consensus.cc:2272] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:19.797413 753 tablet_service.cc:1505] Processing CreateTablet for tablet 26b890ed87c3471b9fc0b172c90e0a11 (DEFAULT_TABLE table=default.Table_0c2fc83ca5324ab3b8b56c2aaf42864c [id=d2e4220a0cbf443e8380433da4f064f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.797510 748 tablet_service.cc:1552] Processing DeleteTablet for tablet d2fa4c3d6ec44047abf5986e502c1590 with delete_type TABLET_DATA_DELETED (replaced by tablet a0b5d82e5fb94b60b4aa324b978a5378) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:19.797746 755 tablet_service.cc:1552] Processing DeleteTablet for tablet d1e0e74f32a24729b250c5751677cfb0 with delete_type TABLET_DATA_DELETED (replaced by tablet 2c31b45dc8004eae993e972e9faa3ff8) from {username='slave'} at 127.0.0.1:42950
W20260202 02:52:19.798154 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5ecb81038c9f48438e74fe874fc83e74 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:19.798264 1643 tablet.cc:1722] T bd7b0a0fcb7645fdb9edfeccb3d18268 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:19.804844 1422 ts_manager.cc:194] Registered new tserver with Master: 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:19.809417 1173 tablet_bootstrap.cc:492] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:19.809590 1173 ts_tablet_manager.cc:1403] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.133s user 0.000s sys 0.004s
I20260202 02:52:19.808784 1644 tablet.cc:1722] T 5986676f5484498da927b4c6da3f47ff P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:19.810516 1173 raft_consensus.cc:359] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:19.810719 1173 raft_consensus.cc:385] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:19.810804 1173 raft_consensus.cc:740] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:19.814287 519 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_052de59e60de4473a2d13185440ab818" } new_table_name: "default.Table_911bb209b4554ee8b9bc09edaf488d63" modify_external_catalogs: true
I20260202 02:52:19.818953 524 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 02b434fa058b45529ff3938b053263c6
I20260202 02:52:19.819927 522 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet 4c63192b273b462a9c1a0e530e938024
I20260202 02:52:19.820174 1037 tablet_service.cc:1552] Processing DeleteTablet for tablet 02b434fa058b45529ff3938b053263c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:19.822871 1174 raft_consensus.cc:359] T 70306ed9dec54286885d965f2c012da4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:19.825121 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 75569511179b41038e521fa3346a92ae (DEFAULT_TABLE table=default.Table_6da292362a6c43738dbeb895750b918a [id=6d50800a3c8642c0992dd1a7f38cfd30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.825644 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be993c1512b04df4b92cab3195241d84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.826784 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26b890ed87c3471b9fc0b172c90e0a11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.828655 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75569511179b41038e521fa3346a92ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.829648 1173 consensus_queue.cc:260] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:19.831571 1174 raft_consensus.cc:385] T 70306ed9dec54286885d965f2c012da4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:19.833611 748 tablet_service.cc:1552] Processing DeleteTablet for tablet 02b434fa058b45529ff3938b053263c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:19.834225 755 tablet_service.cc:1552] Processing DeleteTablet for tablet bc1539e5c8aa4d6e98f687728b891530 with delete_type TABLET_DATA_DELETED (replaced by tablet 84689b18ad274ba0a1522441980b62ce) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:19.835094 1173 ts_tablet_manager.cc:1434] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.025s user 0.000s sys 0.003s
I20260202 02:52:19.835441 1173 tablet_bootstrap.cc:492] T 761f5a51335a4de5977f5f7fef0d4260 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:19.838050 750 tablet_service.cc:1505] Processing CreateTablet for tablet be993c1512b04df4b92cab3195241d84 (DEFAULT_TABLE table=default.Table_b13428495dde400e83de7d1032559c35 [id=22f1ff6e13a84a72820c887dd3107f49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.838629 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be993c1512b04df4b92cab3195241d84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.832490 779 raft_consensus.cc:2468] T 0d297b6243c348ee82c146a8339cf6c4 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:19.841717 998 leader_election.cc:304] T 0d297b6243c348ee82c146a8339cf6c4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
W20260202 02:52:19.841974 998 raft_consensus.cc:2686] T 0d297b6243c348ee82c146a8339cf6c4 P b1f05820b6034228bdad884cc348716d: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260202 02:52:19.842360 1180 ts_tablet_manager.cc:1916] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:19.853665 519 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 28a900bcaa8b41718a285e595886d805 consensus state has no leader
I20260202 02:52:19.854887 755 tablet_service.cc:1505] Processing CreateTablet for tablet 75569511179b41038e521fa3346a92ae (DEFAULT_TABLE table=default.Table_6da292362a6c43738dbeb895750b918a [id=6d50800a3c8642c0992dd1a7f38cfd30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.857069 1174 raft_consensus.cc:740] T 70306ed9dec54286885d965f2c012da4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:19.858147 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 026e6ee0b3ae4d938fdf1550044ed1fa (DEFAULT_TABLE table=default.Table_f93d759396c04d25ac602a609ecc4962 [id=7482d135c01241b4b3752b88fb1a0b2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.858145 1174 consensus_queue.cc:260] T 70306ed9dec54286885d965f2c012da4 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:19.859057 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 026e6ee0b3ae4d938fdf1550044ed1fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.859243 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 874ca4ba4ebc4721bb4b60cba8e01841 (DEFAULT_TABLE table=default.Table_11d7a6e6268042a7a26c8ecc00bc9f6e [id=6777a441613b40cbb21d431744fca23f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.859926 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 874ca4ba4ebc4721bb4b60cba8e01841. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.860939 748 tablet_service.cc:1505] Processing CreateTablet for tablet 026e6ee0b3ae4d938fdf1550044ed1fa (DEFAULT_TABLE table=default.Table_f93d759396c04d25ac602a609ecc4962 [id=7482d135c01241b4b3752b88fb1a0b2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.862804 1174 ts_tablet_manager.cc:1434] T 70306ed9dec54286885d965f2c012da4 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.123s user 0.002s sys 0.001s
I20260202 02:52:19.867421 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_0968e46468aa4819be9dd259b080f33f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:19.868341 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0968e46468aa4819be9dd259b080f33f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:19.869839 525 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_733044f900d44218a06c976c8701f122" } modify_external_catalogs: true
I20260202 02:52:19.869740 759 tablet_service.cc:1552] Processing DeleteTablet for tablet 267bd4752c5f4b99a3fe778ba8f38a2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:19.870157 525 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_733044f900d44218a06c976c8701f122" } modify_external_catalogs: true
I20260202 02:52:19.870824 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_a0f1f2d30d064244bec97181c11c3a06"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:19.871434 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a0f1f2d30d064244bec97181c11c3a06 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:19.872915 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_d040708acb92480496bac8b012890006"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:19.873431 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d040708acb92480496bac8b012890006 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:19.874548 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75569511179b41038e521fa3346a92ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.875217 527 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_2669ca9662b942c08bcbfd7641cfe2e3" } modify_external_catalogs: true
I20260202 02:52:19.875432 527 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_2669ca9662b942c08bcbfd7641cfe2e3" } modify_external_catalogs: true
I20260202 02:52:19.877013 1173 tablet_bootstrap.cc:654] T 761f5a51335a4de5977f5f7fef0d4260 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:19.881238 889 tablet_service.cc:1505] Processing CreateTablet for tablet fe53a8bdbd13417f9ce195c54a52a93c (DEFAULT_TABLE table=default.Table_ce2d44550fb5403a99372bc5d3d79265 [id=4f19275f416a425c9b9d100b2a83870a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.882685 1174 tablet_bootstrap.cc:492] T 80d443bdb66d4cc68ae283e08c410fe6 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:19.884516 1174 tablet_bootstrap.cc:654] T 80d443bdb66d4cc68ae283e08c410fe6 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:19.885293 751 tablet_service.cc:1505] Processing CreateTablet for tablet 874ca4ba4ebc4721bb4b60cba8e01841 (DEFAULT_TABLE table=default.Table_11d7a6e6268042a7a26c8ecc00bc9f6e [id=6777a441613b40cbb21d431744fca23f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.885752 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 874ca4ba4ebc4721bb4b60cba8e01841. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.886647 884 tablet_service.cc:1505] Processing CreateTablet for tablet fe9ea4c0502f4976b440465992f2ea2c (DEFAULT_TABLE table=default.Table_9181c8b857b540ff846c397f9f1a5484 [id=d2fb3868962547c48e29f6353e5747a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.887135 888 tablet_service.cc:1505] Processing CreateTablet for tablet 02b434fa058b45529ff3938b053263c6 (DEFAULT_TABLE table=default.Table_6133c617ff8d40eca5cbde96533acd82 [id=5a502f86d98b42c6a3e274de5bf08544]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:19.888943 859 consensus_peers.cc:597] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5 -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:19.890242 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe53a8bdbd13417f9ce195c54a52a93c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.891175 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 026e6ee0b3ae4d938fdf1550044ed1fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.900491 893 tablet_service.cc:1552] Processing DeleteTablet for tablet 723921dfe54c4e66af5cab76a25fee0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:19.901383 1178 raft_consensus.cc:493] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:19.901577 1178 raft_consensus.cc:515] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:19.902318 1178 leader_election.cc:290] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:19.904407 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "560465c063d94ffaab33f9f79a58a357" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:19.904633 779 raft_consensus.cc:2468] T 560465c063d94ffaab33f9f79a58a357 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:19.905303 998 leader_election.cc:304] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:19.905705 1178 raft_consensus.cc:2804] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:19.905804 1178 raft_consensus.cc:493] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:19.905913 1178 raft_consensus.cc:3060] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:19.906405 522 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_c37ce6d5cf70417c83a68c31d02881ef" } modify_external_catalogs: true
I20260202 02:52:19.906616 522 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_c37ce6d5cf70417c83a68c31d02881ef" } modify_external_catalogs: true
I20260202 02:52:19.908471 893 tablet_service.cc:1552] Processing DeleteTablet for tablet 94bada52db6a47179415d302b541d01f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:19.909379 1178 raft_consensus.cc:515] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:19.910784 525 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 9c1271ce0f194021b588b82ac9d3db79
I20260202 02:52:19.911674 1178 leader_election.cc:290] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:19.912559 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 9c1271ce0f194021b588b82ac9d3db79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:19.912571 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "560465c063d94ffaab33f9f79a58a357" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:19.912792 779 raft_consensus.cc:3060] T 560465c063d94ffaab33f9f79a58a357 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:19.913643 1173 tablet_bootstrap.cc:492] T 761f5a51335a4de5977f5f7fef0d4260 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:19.915053 1173 ts_tablet_manager.cc:1403] T 761f5a51335a4de5977f5f7fef0d4260 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.079s user 0.000s sys 0.006s
I20260202 02:52:19.917214 779 raft_consensus.cc:2468] T 560465c063d94ffaab33f9f79a58a357 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:19.917869 998 leader_election.cc:304] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:19.918220 1178 raft_consensus.cc:2804] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:19.918376 1178 raft_consensus.cc:697] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:19.918650 1178 consensus_queue.cc:237] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:19.918772 893 tablet_service.cc:1552] Processing DeleteTablet for tablet 59990791123c4d4eb620090a1e8e31e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:19.919257 895 tablet_service.cc:1552] Processing DeleteTablet for tablet ffd80badce1048f3ac95366cad05ae53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:19.919456 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe9ea4c0502f4976b440465992f2ea2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.920356 748 tablet_service.cc:1552] Processing DeleteTablet for tablet 9c1271ce0f194021b588b82ac9d3db79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:19.920591 524 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_0a56eb8a4a2944a495e4b076b63c0e19" } new_table_name: "default.Table_499eb7d77ae34d88ad956bf4a5c55648" modify_external_catalogs: true
I20260202 02:52:19.923537 525 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_0c2fc83ca5324ab3b8b56c2aaf42864c" } new_table_name: "default.Table_044ef6a2b1af4e3cab3c8cf00505c6a6" modify_external_catalogs: true
I20260202 02:52:19.922745 1173 raft_consensus.cc:359] T 761f5a51335a4de5977f5f7fef0d4260 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:19.924795 1173 raft_consensus.cc:385] T 761f5a51335a4de5977f5f7fef0d4260 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:19.925789 1173 raft_consensus.cc:740] T 761f5a51335a4de5977f5f7fef0d4260 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:19.928435 1173 consensus_queue.cc:260] T 761f5a51335a4de5977f5f7fef0d4260 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:19.930590 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_1cc0209a40ad471fb1c41012a476df79"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:19.931355 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1cc0209a40ad471fb1c41012a476df79 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:19.931979 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_2be7fcc0f92846059283a7691fcda407"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:19.933204 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2be7fcc0f92846059283a7691fcda407 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:19.933465 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_8e6ef9e6965c47f2ac2eb775c1e60dc8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:19.934132 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8e6ef9e6965c47f2ac2eb775c1e60dc8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:19.935497 520 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet bd7b0a0fcb7645fdb9edfeccb3d18268 (table default.Table_18fec323d6b04e10a687de6361b643b2 [id=1ffcc11561524fe8a7b09a7ddc92d5b2]). Expected version 2 got 0
I20260202 02:52:19.935676 520 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 5986676f5484498da927b4c6da3f47ff (table default.Table_8cb3cbab1a654f88a07d6d480fc72e52 [id=d8485f23f63b493781bea557dbe8a0c1]). Expected version 1 got 0
I20260202 02:52:19.937139 520 master_service.cc:954] ReplaceTablet: received request to replace tablet 38301e13e17c41fcae912c4b38c26a45 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:19.937989 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02b434fa058b45529ff3938b053263c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.938575 1642 tablet.cc:1722] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:19.942947 1646 tablet.cc:1722] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:19.945768 1643 tablet.cc:1722] T 61a9203035f2495f8f99924303ebf120 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:19.948427 1183 raft_consensus.cc:2243] T 61a9203035f2495f8f99924303ebf120 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:19.949409 1183 raft_consensus.cc:2272] T 61a9203035f2495f8f99924303ebf120 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:19.949808 1173 ts_tablet_manager.cc:1434] T 761f5a51335a4de5977f5f7fef0d4260 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.034s user 0.000s sys 0.003s
I20260202 02:52:19.953848 1422 ts_manager.cc:194] Registered new tserver with Master: b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:19.954736 1173 tablet_bootstrap.cc:492] T 415d14533fd54f20adb00455a7bcad5f P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:19.956632 1183 ts_tablet_manager.cc:1916] T 61a9203035f2495f8f99924303ebf120 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:19.958302 1564 raft_consensus.cc:493] T 75f7824681344d8d88df9402694fff1b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:19.964259 902 tablet_service.cc:1552] Processing DeleteTablet for tablet d2fa4c3d6ec44047abf5986e502c1590 with delete_type TABLET_DATA_DELETED (replaced by tablet a0b5d82e5fb94b60b4aa324b978a5378) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:19.966086 1180 ts_tablet_manager.cc:1929] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:19.967741 1175 tablet_bootstrap.cc:654] T 757492288d7c44fb91b25b7e6e21e01b P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:19.968245 1180 log.cc:1199] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/ea040c03420c4b1b98897f39160b4130
I20260202 02:52:19.970536 1173 tablet_bootstrap.cc:654] T 415d14533fd54f20adb00455a7bcad5f P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:19.972554 1183 ts_tablet_manager.cc:1929] T 61a9203035f2495f8f99924303ebf120 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:19.972972 1183 log.cc:1199] T 61a9203035f2495f8f99924303ebf120 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/61a9203035f2495f8f99924303ebf120
W20260202 02:52:19.975538 859 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260202 02:52:19.979912 1183 ts_tablet_manager.cc:1950] T 61a9203035f2495f8f99924303ebf120 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
W20260202 02:52:19.981505 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet c12ff20ca6b44ddba0c936a28cfb2eb9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c12ff20ca6b44ddba0c936a28cfb2eb9 already in progress: creating tablet
I20260202 02:52:19.981897 1183 tablet_replica.cc:333] T 724d47241f0342768fb1321ef37c3a7b P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:19.982112 1183 raft_consensus.cc:2243] T 724d47241f0342768fb1321ef37c3a7b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:19.982597 1183 raft_consensus.cc:2272] T 724d47241f0342768fb1321ef37c3a7b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:19.985528 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 33150e1456144b8e9c64cc2a2ac6dcca (DEFAULT_TABLE table=default.Table_38e8a46dd2dd498baa45935fa4670b09 [id=97791994f92a47519b28e44ec02df5ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:19.986009 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33150e1456144b8e9c64cc2a2ac6dcca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:19.988941 1183 ts_tablet_manager.cc:1916] T 724d47241f0342768fb1321ef37c3a7b P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:19.998858 1180 ts_tablet_manager.cc:1950] T ea040c03420c4b1b98897f39160b4130 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:19.999748 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 222) took 1408 ms. Trace:
I20260202 02:52:19.999869 1180 rpcz_store.cc:276] 0202 02:52:18.591232 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:18.591529 (+ 297us) service_pool.cc:225] Handling call
0202 02:52:19.609725 (+1018196us) ts_tablet_manager.cc:1151] Deleting tablet ea040c03420c4b1b98897f39160b4130
0202 02:52:19.609736 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:19.678398 (+ 68662us) spinlock_profiling.cc:243] Waited 20.8 ms on lock 0x6170000825f0. stack: 00007f1e37b5aa86 00007f1e37b5a840 00007f1e36b353ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f1e4392782f 00007f1e439d801c 00007f1e43a482c3 00007f1e44d960b3 00007f1e47956757 00007f1e47989744 00007f1e4796a240 00007f1e47969fcd 0000000000a8970c 00007f1e37ba0daf
0202 02:52:19.873809 (+195411us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:19.964461 (+ 90652us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:19.999678 (+ 35217us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1077,"spinlock_wait_cycles":47886592,"tablet-delete.queue_time_us":1018068}
I20260202 02:52:20.001940 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet ea040c03420c4b1b98897f39160b4130 (table default.Table_9bac792e43464105846e5482edbccacc [id=0a3c23955dde47d9982c4659d9a73e30]) successfully deleted
I20260202 02:52:20.009536 1609 mvcc.cc:204] Tried to move back new op lower bound from 7249923028573380608 to 7249923022097240064. Current Snapshot: MvccSnapshot[applied={T|T < 7249923028573380608}]
I20260202 02:52:20.010195 898 tablet_service.cc:1552] Processing DeleteTablet for tablet 02b434fa058b45529ff3938b053263c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:20.013157 1643 tablet.cc:1717] T bd7b0a0fcb7645fdb9edfeccb3d18268 P b1f05820b6034228bdad884cc348716d: Skipping requested alter to schema version 2, tablet already version 2
I20260202 02:52:20.011771 1183 ts_tablet_manager.cc:1929] T 724d47241f0342768fb1321ef37c3a7b P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:20.014775 1183 log.cc:1199] T 724d47241f0342768fb1321ef37c3a7b P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/724d47241f0342768fb1321ef37c3a7b
I20260202 02:52:20.015981 1183 ts_tablet_manager.cc:1950] T 724d47241f0342768fb1321ef37c3a7b P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:20.015775 1642 tablet.cc:1717] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d: Skipping requested alter to schema version 2, tablet already version 2
I20260202 02:52:20.019867 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 222) took 1069 ms. Trace:
I20260202 02:52:20.023605 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 724d47241f0342768fb1321ef37c3a7b (table default.Table_5c9c0f09396a4a0e8b3d676fcb2dbb2c [id=6a47f828bd9c4d7aa7d385e4a1dda9ea]) successfully deleted
W20260202 02:52:20.024015 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet d2fa4c3d6ec44047abf5986e502c1590 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d2fa4c3d6ec44047abf5986e502c1590 already in progress: creating tablet
W20260202 02:52:20.024559 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 02b434fa058b45529ff3938b053263c6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 02b434fa058b45529ff3938b053263c6 already in progress: creating tablet
W20260202 02:52:20.024847 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 9c1271ce0f194021b588b82ac9d3db79 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9c1271ce0f194021b588b82ac9d3db79 already in progress: creating tablet
I20260202 02:52:20.025063 895 tablet_service.cc:1552] Processing DeleteTablet for tablet d1e0e74f32a24729b250c5751677cfb0 with delete_type TABLET_DATA_DELETED (replaced by tablet 2c31b45dc8004eae993e972e9faa3ff8) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:20.025261 1180 tablet_replica.cc:333] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:20.025543 1180 raft_consensus.cc:2243] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:20.025846 1180 raft_consensus.cc:2272] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:20.026525 893 tablet_service.cc:1552] Processing DeleteTablet for tablet 61a9203035f2495f8f99924303ebf120 with delete_type TABLET_DATA_DELETED (replaced by tablet 4c63192b273b462a9c1a0e530e938024) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:20.026782 895 tablet_service.cc:1505] Processing CreateTablet for tablet 026e6ee0b3ae4d938fdf1550044ed1fa (DEFAULT_TABLE table=default.Table_f93d759396c04d25ac602a609ecc4962 [id=7482d135c01241b4b3752b88fb1a0b2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.027683 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 026e6ee0b3ae4d938fdf1550044ed1fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.028605 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 75f7824681344d8d88df9402694fff1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:20.028941 1180 ts_tablet_manager.cc:1916] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:20.032727 1564 raft_consensus.cc:515] T 75f7824681344d8d88df9402694fff1b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.033506 1564 leader_election.cc:290] T 75f7824681344d8d88df9402694fff1b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:20.035477 1057 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "75f7824681344d8d88df9402694fff1b" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:20.035743 902 tablet_service.cc:1552] Processing DeleteTablet for tablet 80d443bdb66d4cc68ae283e08c410fe6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:20.036585 901 tablet_service.cc:1505] Processing CreateTablet for tablet 26b890ed87c3471b9fc0b172c90e0a11 (DEFAULT_TABLE table=default.Table_0c2fc83ca5324ab3b8b56c2aaf42864c [id=d2e4220a0cbf443e8380433da4f064f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.037811 885 tablet_service.cc:1505] Processing CreateTablet for tablet be993c1512b04df4b92cab3195241d84 (DEFAULT_TABLE table=default.Table_b13428495dde400e83de7d1032559c35 [id=22f1ff6e13a84a72820c887dd3107f49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.039497 898 tablet_service.cc:1505] Processing CreateTablet for tablet 75569511179b41038e521fa3346a92ae (DEFAULT_TABLE table=default.Table_6da292362a6c43738dbeb895750b918a [id=6d50800a3c8642c0992dd1a7f38cfd30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.040464 899 tablet_service.cc:1552] Processing DeleteTablet for tablet c12ff20ca6b44ddba0c936a28cfb2eb9 with delete_type TABLET_DATA_DELETED (replaced by tablet 072bb7f0e4ab4935be49461bc8a3c163) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:20.040753 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26b890ed87c3471b9fc0b172c90e0a11. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:20.043044 859 leader_election.cc:343] T 75f7824681344d8d88df9402694fff1b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:20.043363 899 tablet_service.cc:1552] Processing DeleteTablet for tablet 9c1271ce0f194021b588b82ac9d3db79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:20.043565 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be993c1512b04df4b92cab3195241d84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.044094 1180 ts_tablet_manager.cc:1929] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:20.044247 1180 log.cc:1199] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/4f2a9404f7424dc78b5bd062ebd8cb53
I20260202 02:52:20.044670 1180 ts_tablet_manager.cc:1950] T 4f2a9404f7424dc78b5bd062ebd8cb53 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:20.044910 902 tablet_service.cc:1505] Processing CreateTablet for tablet 874ca4ba4ebc4721bb4b60cba8e01841 (DEFAULT_TABLE table=default.Table_11d7a6e6268042a7a26c8ecc00bc9f6e [id=6777a441613b40cbb21d431744fca23f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.045625 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 221) took 1454 ms. Trace:
I20260202 02:52:20.045840 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 874ca4ba4ebc4721bb4b60cba8e01841. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.048113 1564 raft_consensus.cc:493] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:20.048285 1564 raft_consensus.cc:515] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.049533 899 tablet_service.cc:1552] Processing DeleteTablet for tablet eb0ac55fbb534b29a86d6b3e8aa72ebc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:20.049759 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75569511179b41038e521fa3346a92ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.053666 1180 rpcz_store.cc:276] 0202 02:52:18.591137 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:18.591674 (+ 537us) service_pool.cc:225] Handling call
0202 02:52:20.006319 (+1414645us) ts_tablet_manager.cc:1151] Deleting tablet 4f2a9404f7424dc78b5bd062ebd8cb53
0202 02:52:20.025160 (+ 18841us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:20.033112 (+ 7952us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:20.036222 (+ 3110us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:20.045578 (+ 9356us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1414313}
I20260202 02:52:20.054320 1180 tablet_replica.cc:333] T 560465c063d94ffaab33f9f79a58a357 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:20.054481 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 4f2a9404f7424dc78b5bd062ebd8cb53 (table default.Table_0842fd6a82e145e7a98b3fed2c0c6993 [id=4e076b2023be4c888ac1c27babde1607]) successfully deleted
I20260202 02:52:20.054615 1180 raft_consensus.cc:2243] T 560465c063d94ffaab33f9f79a58a357 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:20.054829 1180 raft_consensus.cc:2272] T 560465c063d94ffaab33f9f79a58a357 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:20.055521 1180 ts_tablet_manager.cc:1916] T 560465c063d94ffaab33f9f79a58a357 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:20.059264 1057 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "eb0ac55fbb534b29a86d6b3e8aa72ebc" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:20.062016 1564 leader_election.cc:290] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:20.063036 1180 ts_tablet_manager.cc:1929] T 560465c063d94ffaab33f9f79a58a357 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:20.063211 1180 log.cc:1199] T 560465c063d94ffaab33f9f79a58a357 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/560465c063d94ffaab33f9f79a58a357
I20260202 02:52:20.063594 1180 ts_tablet_manager.cc:1950] T 560465c063d94ffaab33f9f79a58a357 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:20.064580 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 225) took 1471 ms. Trace:
W20260202 02:52:20.066197 859 leader_election.cc:343] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:20.071357 918 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bd7b0a0fcb7645fdb9edfeccb3d18268" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:20.074605 1180 rpcz_store.cc:276] 0202 02:52:18.592875 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:18.592982 (+ 107us) service_pool.cc:225] Handling call
0202 02:52:20.054240 (+1461258us) ts_tablet_manager.cc:1151] Deleting tablet 560465c063d94ffaab33f9f79a58a357
0202 02:52:20.054254 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:20.059616 (+ 5362us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:20.062893 (+ 3277us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:20.064519 (+ 1626us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1461021}
I20260202 02:52:20.075340 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 560465c063d94ffaab33f9f79a58a357 (table default.Table_3b335a0806154f98a907f551596ba353 [id=0da4db61b3fe4cf4846edd5530b9327f]) successfully deleted
I20260202 02:52:20.075423 1180 tablet_replica.cc:333] T cb0dcacdf0f1419da0b25f20cac1d62d P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:20.075639 1180 raft_consensus.cc:2243] T cb0dcacdf0f1419da0b25f20cac1d62d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:20.075892 1180 raft_consensus.cc:2272] T cb0dcacdf0f1419da0b25f20cac1d62d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:20.076485 1180 ts_tablet_manager.cc:1916] T cb0dcacdf0f1419da0b25f20cac1d62d P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:20.081311 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 0b411e8fab074aa88f8380e3f5694340 (table default.Table_911bb209b4554ee8b9bc09edaf488d63 [id=cd07f9746ec94d9ebe956a7ca0d19b1c]) successfully deleted
I20260202 02:52:20.081626 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bbf31e5619ed4f60b4d7c8b19ec8a549" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:20.081872 778 raft_consensus.cc:2393] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1521a6d262b7451bbef8debe970dd6b5 in current term 1: Already voted for candidate 1f442ef85029461091de847118c34c0d in this term.
I20260202 02:52:20.082222 714 leader_election.cc:304] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:20.082535 1534 raft_consensus.cc:2804] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:20.082623 1534 raft_consensus.cc:493] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:20.082710 1534 raft_consensus.cc:3060] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:20.082852 714 leader_election.cc:304] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:20.083331 1277 raft_consensus.cc:2804] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:20.083423 1277 raft_consensus.cc:493] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:20.083521 1277 raft_consensus.cc:3060] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:20.086047 1277 raft_consensus.cc:515] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.086712 1277 leader_election.cc:290] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:20.087404 1277 consensus_queue.cc:1048] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260202 02:52:20.088194 714 leader_election.cc:343] T c184d44d7f434b39a997a7387e5d3de0 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: c184d44d7f434b39a997a7387e5d3de0
I20260202 02:52:20.088366 714 leader_election.cc:304] T c184d44d7f434b39a997a7387e5d3de0 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:20.088742 1277 raft_consensus.cc:2749] T c184d44d7f434b39a997a7387e5d3de0 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:20.089182 1277 consensus_queue.cc:1048] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:20.090003 1057 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c63a670d5c5e489897bc6dbb142b844d" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
W20260202 02:52:20.091089 715 leader_election.cc:343] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:20.091336 1534 raft_consensus.cc:515] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.092412 1057 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0d3633f5f9f54115bc78b9c1b78aba67" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
W20260202 02:52:20.092999 715 leader_election.cc:343] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:20.093428 1534 leader_election.cc:290] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:20.094439 1180 ts_tablet_manager.cc:1929] T cb0dcacdf0f1419da0b25f20cac1d62d P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:20.094623 1180 log.cc:1199] T cb0dcacdf0f1419da0b25f20cac1d62d P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/cb0dcacdf0f1419da0b25f20cac1d62d
I20260202 02:52:20.095142 1180 ts_tablet_manager.cc:1950] T cb0dcacdf0f1419da0b25f20cac1d62d P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:20.096101 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 229) took 1502 ms. Trace:
I20260202 02:52:20.096297 1180 rpcz_store.cc:276] 0202 02:52:18.593288 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:18.593335 (+ 47us) service_pool.cc:225] Handling call
0202 02:52:20.075325 (+1481990us) ts_tablet_manager.cc:1151] Deleting tablet cb0dcacdf0f1419da0b25f20cac1d62d
0202 02:52:20.075338 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:20.079979 (+ 4641us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:20.094262 (+ 14283us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:20.096062 (+ 1800us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1481766}
I20260202 02:52:20.096973 1180 tablet_replica.cc:333] T aa526a69cf914dd08142074636c956b7 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:20.096952 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet cb0dcacdf0f1419da0b25f20cac1d62d (table default.Table_c245b83c1987454fabbe8646d14252d6 [id=08498a654b6a4a9ca3f968d70daad9b6]) successfully deleted
I20260202 02:52:20.097281 1180 raft_consensus.cc:2243] T aa526a69cf914dd08142074636c956b7 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:20.097550 1180 raft_consensus.cc:2272] T aa526a69cf914dd08142074636c956b7 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:20.098196 1180 ts_tablet_manager.cc:1916] T aa526a69cf914dd08142074636c956b7 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:20.034225 1182 rpcz_store.cc:276] 0202 02:52:18.846762 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:18.846825 (+ 63us) service_pool.cc:225] Handling call
0202 02:52:19.823913 (+977088us) ts_tablet_manager.cc:1151] Deleting tablet ea040c03420c4b1b98897f39160b4130
0202 02:52:19.823928 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:19.916382 (+ 92454us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":976828}
I20260202 02:52:20.101593 1534 consensus_queue.cc:1048] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260202 02:52:20.101775 714 leader_election.cc:343] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Illegal state: Tablet not RUNNING: STOPPED
I20260202 02:52:20.101907 714 leader_election.cc:304] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:20.102165 1534 raft_consensus.cc:2749] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260202 02:52:20.102625 714 leader_election.cc:343] T bc1539e5c8aa4d6e98f687728b891530 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: bc1539e5c8aa4d6e98f687728b891530
I20260202 02:52:20.102773 714 leader_election.cc:304] T bc1539e5c8aa4d6e98f687728b891530 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:20.103071 1534 raft_consensus.cc:2749] T bc1539e5c8aa4d6e98f687728b891530 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:20.103281 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 221) took 1256 ms. Trace:
I20260202 02:52:20.103395 1182 rpcz_store.cc:276] 0202 02:52:18.846628 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:18.846926 (+ 298us) service_pool.cc:225] Handling call
0202 02:52:20.100275 (+1253349us) ts_tablet_manager.cc:1151] Deleting tablet fae7e3941c124cc2871390a0b6471f27
0202 02:52:20.100287 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:20.103218 (+ 2931us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1253128}
I20260202 02:52:20.103770 1182 tablet_replica.cc:333] T c064ea0dade840b4840a5715c2d83f69 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:20.107761 1180 ts_tablet_manager.cc:1929] T aa526a69cf914dd08142074636c956b7 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:20.108899 1180 log.cc:1199] T aa526a69cf914dd08142074636c956b7 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/aa526a69cf914dd08142074636c956b7
I20260202 02:52:20.112417 1180 ts_tablet_manager.cc:1950] T aa526a69cf914dd08142074636c956b7 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:20.116048 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet bbf31e5619ed4f60b4d7c8b19ec8a549 (table default.Table_ef11a1ba065d483c9f7862345ab1aedb [id=0e54d0abe11c46fa8dbf0befb46219cb]) successfully deleted
I20260202 02:52:20.116135 748 tablet_service.cc:1505] Processing CreateTablet for tablet 33150e1456144b8e9c64cc2a2ac6dcca (DEFAULT_TABLE table=default.Table_38e8a46dd2dd498baa45935fa4670b09 [id=97791994f92a47519b28e44ec02df5ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.116576 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33150e1456144b8e9c64cc2a2ac6dcca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.117624 1044 tablet_service.cc:1505] Processing CreateTablet for tablet dada209d982d48f3b74a49905f19c5b6 (DEFAULT_TABLE table=default.Table_1a25d92824e340579a0b522dab94fce6 [id=116247212fca4c0a95ad1bdbbac4787f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.117940 756 tablet_service.cc:1505] Processing CreateTablet for tablet dada209d982d48f3b74a49905f19c5b6 (DEFAULT_TABLE table=default.Table_1a25d92824e340579a0b522dab94fce6 [id=116247212fca4c0a95ad1bdbbac4787f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.118079 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dada209d982d48f3b74a49905f19c5b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.118417 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dada209d982d48f3b74a49905f19c5b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.128293 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 232) took 1534 ms. Trace:
I20260202 02:52:20.132869 1180 rpcz_store.cc:276] 0202 02:52:18.593599 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:18.593636 (+ 37us) service_pool.cc:225] Handling call
0202 02:52:20.096905 (+1503269us) ts_tablet_manager.cc:1151] Deleting tablet aa526a69cf914dd08142074636c956b7
0202 02:52:20.096915 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:20.101971 (+ 5056us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:20.106803 (+ 4832us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:20.127958 (+ 21155us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1503008}
I20260202 02:52:20.133874 1180 tablet_replica.cc:333] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:20.134796 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet aa526a69cf914dd08142074636c956b7 (table default.Table_ab29f752f1e140da9038d9e11ee61f39 [id=9ef504467a544f299b48499e49adf27f]) successfully deleted
I20260202 02:52:20.135730 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "75f7824681344d8d88df9402694fff1b" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:20.138188 1180 raft_consensus.cc:2243] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:20.139142 1180 pending_rounds.cc:70] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d: Trying to abort 1 pending ops.
I20260202 02:52:20.139230 1180 pending_rounds.cc:77] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923021566361600 op_type: NO_OP noop_request { }
I20260202 02:52:20.139372 1180 raft_consensus.cc:2889] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:20.139482 1180 raft_consensus.cc:2272] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:20.140115 1180 ts_tablet_manager.cc:1916] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:20.157179 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "eb0ac55fbb534b29a86d6b3e8aa72ebc" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:20.157928 999 leader_election.cc:343] T 0d297b6243c348ee82c146a8339cf6c4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 0d297b6243c348ee82c146a8339cf6c4
W20260202 02:52:20.158336 999 leader_election.cc:343] T 0d297b6243c348ee82c146a8339cf6c4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 0d297b6243c348ee82c146a8339cf6c4
I20260202 02:52:20.158483 1182 raft_consensus.cc:2243] T c064ea0dade840b4840a5715c2d83f69 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:20.159446 528 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet 5986676f5484498da927b4c6da3f47ff (table default.Table_8cb3cbab1a654f88a07d6d480fc72e52 [id=d8485f23f63b493781bea557dbe8a0c1]). Expected version 1 got 0
I20260202 02:52:20.159472 999 leader_election.cc:304] T bd7b0a0fcb7645fdb9edfeccb3d18268 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
W20260202 02:52:20.160081 999 leader_election.cc:343] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 560465c063d94ffaab33f9f79a58a357
I20260202 02:52:20.161007 1178 raft_consensus.cc:2749] T bd7b0a0fcb7645fdb9edfeccb3d18268 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260202 02:52:20.162102 528 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet bd7b0a0fcb7645fdb9edfeccb3d18268 (table default.Table_18fec323d6b04e10a687de6361b643b2 [id=1ffcc11561524fe8a7b09a7ddc92d5b2]). Expected version 2 got 0
I20260202 02:52:20.162299 528 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 5986676f5484498da927b4c6da3f47ff (table default.Table_8cb3cbab1a654f88a07d6d480fc72e52 [id=d8485f23f63b493781bea557dbe8a0c1]). Expected version 1 got 0
I20260202 02:52:20.163230 1174 tablet_bootstrap.cc:492] T 80d443bdb66d4cc68ae283e08c410fe6 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:20.164278 528 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_0968e46468aa4819be9dd259b080f33f" } new_table_name: "default.Table_22445ccfd7b64298b92a0f70733f3625" modify_external_catalogs: true
I20260202 02:52:20.164093 1174 ts_tablet_manager.cc:1403] T 80d443bdb66d4cc68ae283e08c410fe6 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.281s user 0.001s sys 0.037s
I20260202 02:52:20.166922 1042 tablet_service.cc:1505] Processing CreateTablet for tablet 79c73f4a92044bafb5b8384229cb3a4d (DEFAULT_TABLE table=default.Table_8cae34d2c6f1443ba07e3f2f8abd1989 [id=dc77705cb5704fd58af1eed3edd45675]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.167366 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79c73f4a92044bafb5b8384229cb3a4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.167605 1174 raft_consensus.cc:359] T 80d443bdb66d4cc68ae283e08c410fe6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.167824 1174 raft_consensus.cc:385] T 80d443bdb66d4cc68ae283e08c410fe6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:20.167912 1174 raft_consensus.cc:740] T 80d443bdb66d4cc68ae283e08c410fe6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:20.168180 1174 consensus_queue.cc:260] T 80d443bdb66d4cc68ae283e08c410fe6 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:20.169610 715 consensus_peers.cc:597] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:20.172886 1453 consensus_queue.cc:1048] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260202 02:52:20.173615 748 tablet_service.cc:1505] Processing CreateTablet for tablet 79c73f4a92044bafb5b8384229cb3a4d (DEFAULT_TABLE table=default.Table_8cae34d2c6f1443ba07e3f2f8abd1989 [id=dc77705cb5704fd58af1eed3edd45675]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.175197 1174 ts_tablet_manager.cc:1434] T 80d443bdb66d4cc68ae283e08c410fe6 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.008s user 0.000s sys 0.002s
W20260202 02:52:20.179567 857 leader_election.cc:343] T eaa47f3388144e8b852804c167f232ce P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:20.180039 1182 raft_consensus.cc:2272] T c064ea0dade840b4840a5715c2d83f69 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:20.180568 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet 560465c063d94ffaab33f9f79a58a357 with delete_type TABLET_DATA_DELETED (replaced by tablet 20ea39bc5d3d46f5a4ba427277d6c6a3) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:20.181205 1183 tablet_replica.cc:333] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:20.181584 1183 raft_consensus.cc:2243] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:20.181990 1183 pending_rounds.cc:70] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d: Trying to abort 1 pending ops.
I20260202 02:52:20.182091 1183 pending_rounds.cc:77] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923030709112832 op_type: NO_OP noop_request { }
I20260202 02:52:20.182237 1183 raft_consensus.cc:2889] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:20.182369 1183 raft_consensus.cc:2272] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:20.184562 1183 ts_tablet_manager.cc:1916] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:20.184881 522 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet b428356fba0c4fb5a6033c5d39740cc5
I20260202 02:52:20.187400 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet b428356fba0c4fb5a6033c5d39740cc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:20.188607 527 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 583e547d0a1943f09f4efe82070fc777
I20260202 02:52:20.189743 527 master_service.cc:954] ReplaceTablet: received request to replace tablet 94bada52db6a47179415d302b541d01f from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:20.195640 1178 raft_consensus.cc:493] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:20.195907 1178 raft_consensus.cc:515] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:20.176229 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79c73f4a92044bafb5b8384229cb3a4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.187165 753 tablet_service.cc:1552] Processing DeleteTablet for tablet b428356fba0c4fb5a6033c5d39740cc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:20.196646 1174 tablet_bootstrap.cc:492] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:20.196669 1178 leader_election.cc:290] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:20.201088 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_0601f3c5b49446b3a7ec44443f89a38c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:20.201969 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf74f96bea2e4ab7adec42c3c5207e79" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:20.202229 779 raft_consensus.cc:2468] T bf74f96bea2e4ab7adec42c3c5207e79 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:20.203704 753 tablet_service.cc:1552] Processing DeleteTablet for tablet 583e547d0a1943f09f4efe82070fc777 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:42950
W20260202 02:52:20.205370 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0601f3c5b49446b3a7ec44443f89a38c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:20.205902 998 leader_election.cc:304] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:20.207113 1182 ts_tablet_manager.cc:1916] T c064ea0dade840b4840a5715c2d83f69 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:20.207594 1178 raft_consensus.cc:2804] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:20.207726 1178 raft_consensus.cc:493] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:20.207827 1178 raft_consensus.cc:3060] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:20.209702 1183 ts_tablet_manager.cc:1929] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:20.210729 1178 raft_consensus.cc:515] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:20.210772 1183 log.cc:1199] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/560465c063d94ffaab33f9f79a58a357
I20260202 02:52:20.212386 1178 leader_election.cc:290] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:20.212363 857 leader_election.cc:304] T eaa47f3388144e8b852804c167f232ce P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:20.213096 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf74f96bea2e4ab7adec42c3c5207e79" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:20.213325 779 raft_consensus.cc:3060] T bf74f96bea2e4ab7adec42c3c5207e79 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:20.213900 525 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 26b890ed87c3471b9fc0b172c90e0a11 consensus state has no leader
W20260202 02:52:20.214197 524 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fbcb4a522df648ba9542250450c08f1b has no consensus state
I20260202 02:52:20.215221 1183 ts_tablet_manager.cc:1950] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:20.171123 1180 ts_tablet_manager.cc:1929] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:20.215616 1180 log.cc:1199] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/23c38ef4e3c840808ec36f859cf6f6de
I20260202 02:52:20.216034 779 raft_consensus.cc:2468] T bf74f96bea2e4ab7adec42c3c5207e79 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:20.216635 998 leader_election.cc:304] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
W20260202 02:52:20.216954 859 consensus_peers.cc:597] T d06b77ff1a6f4f78b73082b00145fcf6 P 1521a6d262b7451bbef8debe970dd6b5 -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:20.217334 1178 raft_consensus.cc:2804] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:20.217537 1178 raft_consensus.cc:697] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:20.217855 1178 consensus_queue.cc:237] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:20.224684 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 560465c063d94ffaab33f9f79a58a357 (table default.Table_3b335a0806154f98a907f551596ba353 [id=0da4db61b3fe4cf4846edd5530b9327f]) successfully deleted
W20260202 02:52:20.225159 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet b428356fba0c4fb5a6033c5d39740cc5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b428356fba0c4fb5a6033c5d39740cc5 already in progress: creating tablet
I20260202 02:52:20.230252 1037 tablet_service.cc:1552] Processing DeleteTablet for tablet 583e547d0a1943f09f4efe82070fc777 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:20.232293 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 583e547d0a1943f09f4efe82070fc777 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 583e547d0a1943f09f4efe82070fc777 already in progress: creating tablet
I20260202 02:52:20.233255 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 61a9203035f2495f8f99924303ebf120 with delete_type TABLET_DATA_DELETED (replaced by tablet 4c63192b273b462a9c1a0e530e938024) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:20.229652 1174 tablet_bootstrap.cc:654] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:20.234632 515 catalog_manager.cc:4952] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 61a9203035f2495f8f99924303ebf120 because the tablet was not found. No further retry: Not found: Tablet not found: 61a9203035f2495f8f99924303ebf120
I20260202 02:52:20.236492 1173 tablet_bootstrap.cc:492] T 415d14533fd54f20adb00455a7bcad5f P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:20.236676 1173 ts_tablet_manager.cc:1403] T 415d14533fd54f20adb00455a7bcad5f P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.285s user 0.024s sys 0.025s
I20260202 02:52:20.237632 1173 raft_consensus.cc:359] T 415d14533fd54f20adb00455a7bcad5f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.237859 1173 raft_consensus.cc:385] T 415d14533fd54f20adb00455a7bcad5f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:20.237983 1173 raft_consensus.cc:740] T 415d14533fd54f20adb00455a7bcad5f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:20.238248 1173 consensus_queue.cc:260] T 415d14533fd54f20adb00455a7bcad5f P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.240412 524 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_22c52df5d2c0467ead479c202dfd2805" } modify_external_catalogs: true
I20260202 02:52:20.240626 524 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_22c52df5d2c0467ead479c202dfd2805" } modify_external_catalogs: true
I20260202 02:52:20.241986 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_408b8f21ff294ce9880d0b9a4d681f93"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:20.242609 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_408b8f21ff294ce9880d0b9a4d681f93 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:20.243023 1182 ts_tablet_manager.cc:1929] T c064ea0dade840b4840a5715c2d83f69 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:20.243490 1182 log.cc:1199] T c064ea0dade840b4840a5715c2d83f69 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/c064ea0dade840b4840a5715c2d83f69
I20260202 02:52:20.243871 525 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_959d3e3da2814b3a986d29864e47246c" } modify_external_catalogs: true
I20260202 02:52:20.244050 525 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_959d3e3da2814b3a986d29864e47246c" } modify_external_catalogs: true
I20260202 02:52:20.244122 1182 ts_tablet_manager.cc:1950] T c064ea0dade840b4840a5715c2d83f69 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:20.245045 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 223) took 1398 ms. Trace:
I20260202 02:52:20.245473 526 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_b18a63e4731b418abcb84c36082553c1" } new_table_name: "default.Table_882b22c2c2b0446c83239be15da80b95" modify_external_catalogs: true
I20260202 02:52:20.250312 519 master_service.cc:954] ReplaceTablet: received request to replace tablet 06b80a1c9251420cbc16fc95ae98aca9 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:20.250646 520 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 38301e13e17c41fcae912c4b38c26a45
I20260202 02:52:20.251137 520 catalog_manager.cc:6724] ReplaceTablet: tablet 38301e13e17c41fcae912c4b38c26a45 deleted and replaced by tablet 79badde1001b432b95f58b5161ace15c
I20260202 02:52:20.251461 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_51648e2a42614c3d9c5820901d153932"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:20.251873 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 38301e13e17c41fcae912c4b38c26a45 with delete_type TABLET_DATA_DELETED (replaced by tablet 79badde1001b432b95f58b5161ace15c) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:20.252090 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_51648e2a42614c3d9c5820901d153932 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:20.256642 528 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f94f25f4059a43b8babedb6b45a08a19 has no consensus state
I20260202 02:52:20.257107 528 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_1a25d92824e340579a0b522dab94fce6" } new_table_name: "default.Table_a69dc5f8fd7e4c15b218f332ac4a74e2" modify_external_catalogs: true
I20260202 02:52:20.258366 523 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_8cb3cbab1a654f88a07d6d480fc72e52 [id=d8485f23f63b493781bea557dbe8a0c1] alter complete (version 1)
I20260202 02:52:20.260516 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_ff54266e8b5a4334ad1fbeda5270f0f2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:20.261216 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ff54266e8b5a4334ad1fbeda5270f0f2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:20.263346 521 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ab29f752f1e140da9038d9e11ee61f39" } modify_external_catalogs: true
I20260202 02:52:20.263587 521 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ab29f752f1e140da9038d9e11ee61f39" } modify_external_catalogs: true
W20260202 02:52:20.266279 526 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 91ec01d25f8f441cbe3783cacbee51db consensus state has no leader
I20260202 02:52:20.266754 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_881157f1bd2945a5be1340a99a941bef"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:20.266942 520 master_service.cc:954] ReplaceTablet: received request to replace tablet c0937b6910534ddc8a2217f0cbd539c5 from {username='slave'} at 127.0.0.1:47430
W20260202 02:52:20.267416 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_881157f1bd2945a5be1340a99a941bef in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:20.267727 524 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet 3bce7f5680b7445c87718a7e725f4855
I20260202 02:52:20.267933 524 master_service.cc:954] ReplaceTablet: received request to replace tablet 9884c90cf1d245b2a7eaaf00d71b1c3e from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:20.268798 1173 ts_tablet_manager.cc:1434] T 415d14533fd54f20adb00455a7bcad5f P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.032s user 0.002s sys 0.000s
I20260202 02:52:20.270534 1173 tablet_bootstrap.cc:492] T cb0dcacdf0f1419da0b25f20cac1d62d P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:20.274258 1173 tablet_bootstrap.cc:654] T cb0dcacdf0f1419da0b25f20cac1d62d P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:20.275138 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 38301e13e17c41fcae912c4b38c26a45 with delete_type TABLET_DATA_DELETED (replaced by tablet 79badde1001b432b95f58b5161ace15c) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:20.276302 1174 tablet_bootstrap.cc:492] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:20.276582 1174 ts_tablet_manager.cc:1403] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.080s user 0.003s sys 0.002s
I20260202 02:52:20.277454 1174 raft_consensus.cc:359] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.277727 1174 raft_consensus.cc:385] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:20.277884 1174 raft_consensus.cc:740] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:20.278224 1174 consensus_queue.cc:260] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.278366 1277 raft_consensus.cc:493] T 80d443bdb66d4cc68ae283e08c410fe6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:20.278836 1277 raft_consensus.cc:515] T 80d443bdb66d4cc68ae283e08c410fe6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.278957 1174 ts_tablet_manager.cc:1434] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260202 02:52:20.279822 1277 leader_election.cc:290] T 80d443bdb66d4cc68ae283e08c410fe6 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:20.279346 1174 tablet_bootstrap.cc:492] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:20.281606 1182 rpcz_store.cc:276] 0202 02:52:18.846872 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:18.847178 (+ 306us) service_pool.cc:225] Handling call
0202 02:52:20.103682 (+1256504us) ts_tablet_manager.cc:1151] Deleting tablet c064ea0dade840b4840a5715c2d83f69
0202 02:52:20.103695 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:20.222262 (+118567us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:20.242671 (+ 20409us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:20.244992 (+ 2321us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1256235}
W20260202 02:52:20.283556 528 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet dada209d982d48f3b74a49905f19c5b6 consensus state has no leader
I20260202 02:52:20.284051 523 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_18fec323d6b04e10a687de6361b643b2 [id=1ffcc11561524fe8a7b09a7ddc92d5b2] alter complete (version 2)
I20260202 02:52:20.285877 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 80d443bdb66d4cc68ae283e08c410fe6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:20.282130 1057 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "80d443bdb66d4cc68ae283e08c410fe6" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:20.290992 1175 tablet_bootstrap.cc:492] T 757492288d7c44fb91b25b7e6e21e01b P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
W20260202 02:52:20.292979 715 leader_election.cc:343] T 80d443bdb66d4cc68ae283e08c410fe6 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:20.293249 1175 ts_tablet_manager.cc:1403] T 757492288d7c44fb91b25b7e6e21e01b P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.600s user 0.010s sys 0.012s
I20260202 02:52:20.294095 1178 raft_consensus.cc:493] T 761f5a51335a4de5977f5f7fef0d4260 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:20.294241 1178 raft_consensus.cc:515] T 761f5a51335a4de5977f5f7fef0d4260 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.295341 1178 leader_election.cc:290] T 761f5a51335a4de5977f5f7fef0d4260 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:20.295831 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "761f5a51335a4de5977f5f7fef0d4260" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:20.290989 1037 tablet_service.cc:1552] Processing DeleteTablet for tablet bf74f96bea2e4ab7adec42c3c5207e79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:16 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:20.296108 1174 tablet_bootstrap.cc:654] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:20.291215 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 560465c063d94ffaab33f9f79a58a357 with delete_type TABLET_DATA_DELETED (replaced by tablet 20ea39bc5d3d46f5a4ba427277d6c6a3) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:20.297199 751 tablet_service.cc:1552] Processing DeleteTablet for tablet f93e8d3c4faf48eaa98c3cbf88d585e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:20.291442 1656 raft_consensus.cc:2749] T eaa47f3388144e8b852804c167f232ce P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:20.291864 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 226) took 1444 ms. Trace:
I20260202 02:52:20.298831 1182 rpcz_store.cc:276] 0202 02:52:18.847269 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:18.847318 (+ 49us) service_pool.cc:225] Handling call
0202 02:52:20.291699 (+1444381us) ts_tablet_manager.cc:1151] Deleting tablet bf5ca99708044482970cb854d2eced18
0202 02:52:20.291713 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:20.291828 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1444185}
W20260202 02:52:20.301329 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 38301e13e17c41fcae912c4b38c26a45 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 38301e13e17c41fcae912c4b38c26a45 already in progress: creating tablet
I20260202 02:52:20.307488 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 415d14533fd54f20adb00455a7bcad5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:16 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:20.310324 1183 tablet_replica.cc:333] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:20.310943 1182 tablet_replica.cc:333] T 251455168e33430aafea8e0de42ca1b8 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:20.312129 1174 tablet_bootstrap.cc:492] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:20.312314 520 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet c0937b6910534ddc8a2217f0cbd539c5
I20260202 02:52:20.312539 1174 ts_tablet_manager.cc:1403] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.033s user 0.004s sys 0.001s
I20260202 02:52:20.312870 520 catalog_manager.cc:6724] ReplaceTablet: tablet c0937b6910534ddc8a2217f0cbd539c5 deleted and replaced by tablet 44c8d5b74c8e4c499f7f68e8185efc17
I20260202 02:52:20.313855 1174 raft_consensus.cc:359] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:20.314438 1174 raft_consensus.cc:385] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:20.314703 1174 raft_consensus.cc:740] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:20.314817 751 tablet_service.cc:1552] Processing DeleteTablet for tablet c0937b6910534ddc8a2217f0cbd539c5 with delete_type TABLET_DATA_DELETED (replaced by tablet 44c8d5b74c8e4c499f7f68e8185efc17) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:20.216049 1180 ts_tablet_manager.cc:1950] T 23c38ef4e3c840808ec36f859cf6f6de P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:20.314568 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet c0937b6910534ddc8a2217f0cbd539c5 with delete_type TABLET_DATA_DELETED (replaced by tablet 44c8d5b74c8e4c499f7f68e8185efc17) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:20.315716 1182 raft_consensus.cc:2243] T 251455168e33430aafea8e0de42ca1b8 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:20.315899 1183 raft_consensus.cc:2243] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:20.316183 1183 pending_rounds.cc:70] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d: Trying to abort 1 pending ops.
I20260202 02:52:20.316262 1183 pending_rounds.cc:77] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923031935410176 op_type: NO_OP noop_request { }
I20260202 02:52:20.316408 1183 raft_consensus.cc:2889] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:20.316522 1183 raft_consensus.cc:2272] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:20.317160 1182 raft_consensus.cc:2272] T 251455168e33430aafea8e0de42ca1b8 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:20.317859 1182 ts_tablet_manager.cc:1916] T 251455168e33430aafea8e0de42ca1b8 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:20.320744 857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 150587us
I20260202 02:52:20.320868 520 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet d2d27ceea1bb4122b18f0fcf4d2d1bf4 (table default.Table_10aa0dafec0547069eb92ecf12b3c780 [id=1f9c48c8e2484856a66a0719268f22ae]). Expected version 1 got 0
W20260202 02:52:20.321063 857 leader_election.cc:343] T b428356fba0c4fb5a6033c5d39740cc5 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:20.321182 857 leader_election.cc:304] T b428356fba0c4fb5a6033c5d39740cc5 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
W20260202 02:52:20.321679 857 leader_election.cc:343] T 0186572e7bb9499a932a636ab280f544 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:20.321799 857 leader_election.cc:304] T 0186572e7bb9499a932a636ab280f544 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
W20260202 02:52:20.322008 859 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 68757.3us
I20260202 02:52:20.313570 1175 raft_consensus.cc:359] T 757492288d7c44fb91b25b7e6e21e01b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.322607 1175 raft_consensus.cc:385] T 757492288d7c44fb91b25b7e6e21e01b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:20.322777 1175 raft_consensus.cc:740] T 757492288d7c44fb91b25b7e6e21e01b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:20.324687 1183 ts_tablet_manager.cc:1916] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:20.324968 1175 consensus_queue.cc:260] T 757492288d7c44fb91b25b7e6e21e01b P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.326645 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 227) took 1733 ms. Trace:
I20260202 02:52:20.326857 1180 rpcz_store.cc:276] 0202 02:52:18.593074 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:18.593737 (+ 663us) service_pool.cc:225] Handling call
0202 02:52:20.133770 (+1540033us) ts_tablet_manager.cc:1151] Deleting tablet 23c38ef4e3c840808ec36f859cf6f6de
0202 02:52:20.133783 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:20.146580 (+ 12797us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:20.170969 (+ 24389us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:20.326566 (+155597us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1539763}
I20260202 02:52:20.327458 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 23c38ef4e3c840808ec36f859cf6f6de (table default.Table_9fd715491a6a4bbc8685138d0194362d [id=98ddcbfb5ad14ae480a54845ec7c45a4]) successfully deleted
I20260202 02:52:20.327956 1180 tablet_replica.cc:333] T 724d47241f0342768fb1321ef37c3a7b P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:20.328245 1180 raft_consensus.cc:2243] T 724d47241f0342768fb1321ef37c3a7b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:20.328569 1180 raft_consensus.cc:2272] T 724d47241f0342768fb1321ef37c3a7b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:20.329223 1180 ts_tablet_manager.cc:1916] T 724d47241f0342768fb1321ef37c3a7b P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:20.329665 1174 consensus_queue.cc:260] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:20.332293 1174 ts_tablet_manager.cc:1434] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.020s user 0.002s sys 0.000s
I20260202 02:52:20.333263 1174 tablet_bootstrap.cc:492] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:20.341161 1173 tablet_bootstrap.cc:492] T cb0dcacdf0f1419da0b25f20cac1d62d P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:20.341464 1173 ts_tablet_manager.cc:1403] T cb0dcacdf0f1419da0b25f20cac1d62d P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.071s user 0.006s sys 0.001s
I20260202 02:52:20.342463 1173 raft_consensus.cc:359] T cb0dcacdf0f1419da0b25f20cac1d62d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.342783 1173 raft_consensus.cc:385] T cb0dcacdf0f1419da0b25f20cac1d62d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:20.342906 1173 raft_consensus.cc:740] T cb0dcacdf0f1419da0b25f20cac1d62d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:20.343245 1173 consensus_queue.cc:260] T cb0dcacdf0f1419da0b25f20cac1d62d P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.343807 1173 ts_tablet_manager.cc:1434] T cb0dcacdf0f1419da0b25f20cac1d62d P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:20.346467 1182 ts_tablet_manager.cc:1929] T 251455168e33430aafea8e0de42ca1b8 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260202 02:52:20.349385 1182 log.cc:1199] T 251455168e33430aafea8e0de42ca1b8 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/251455168e33430aafea8e0de42ca1b8
I20260202 02:52:20.347723 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet cb0dcacdf0f1419da0b25f20cac1d62d with delete_type TABLET_DATA_DELETED (replaced by tablet d67e91a022f1401ea724dff898c3b81b) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:20.349960 1182 ts_tablet_manager.cc:1950] T 251455168e33430aafea8e0de42ca1b8 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:20.350306 1173 tablet_bootstrap.cc:492] T 0b411e8fab074aa88f8380e3f5694340 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:20.349179 1183 ts_tablet_manager.cc:1929] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:20.350548 1183 log.cc:1199] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/bf74f96bea2e4ab7adec42c3c5207e79
I20260202 02:52:20.351372 1183 ts_tablet_manager.cc:1950] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:20.352164 1173 tablet_bootstrap.cc:654] T 0b411e8fab074aa88f8380e3f5694340 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:20.352170 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet bf74f96bea2e4ab7adec42c3c5207e79 (table default.Table_d142ff2cb10e491997bf0e302d6a9fa5 [id=a0b8442c02974d6596db002c9292be0e]) successfully deleted
W20260202 02:52:20.352979 515 catalog_manager.cc:4952] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 560465c063d94ffaab33f9f79a58a357 because the tablet was not found. No further retry: Not found: Tablet not found: 560465c063d94ffaab33f9f79a58a357
I20260202 02:52:20.353363 1183 tablet_replica.cc:333] T 415d14533fd54f20adb00455a7bcad5f P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:20.353611 1183 raft_consensus.cc:2243] T 415d14533fd54f20adb00455a7bcad5f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:20.353888 1183 raft_consensus.cc:2272] T 415d14533fd54f20adb00455a7bcad5f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:20.354626 1183 ts_tablet_manager.cc:1916] T 415d14533fd54f20adb00455a7bcad5f P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:20.355050 1174 tablet_bootstrap.cc:654] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:20.358172 1059 raft_consensus.cc:3060] T 761f5a51335a4de5977f5f7fef0d4260 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:20.358412 1277 raft_consensus.cc:493] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:20.358733 1277 raft_consensus.cc:515] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.359424 1277 leader_election.cc:290] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:20.360175 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_7d0cce2fe6094a54a4b51acf6b1d42da"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:20.360522 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f93e8d3c4faf48eaa98c3cbf88d585e2" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:20.360817 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7d0cce2fe6094a54a4b51acf6b1d42da in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:20.361172 1173 tablet_bootstrap.cc:492] T 0b411e8fab074aa88f8380e3f5694340 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:20.361335 1173 ts_tablet_manager.cc:1403] T 0b411e8fab074aa88f8380e3f5694340 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.003s
W20260202 02:52:20.362622 715 leader_election.cc:343] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:20.363091 1277 consensus_queue.cc:1048] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:20.363467 1183 ts_tablet_manager.cc:1929] T 415d14533fd54f20adb00455a7bcad5f P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:20.363610 1183 log.cc:1199] T 415d14533fd54f20adb00455a7bcad5f P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/415d14533fd54f20adb00455a7bcad5f
I20260202 02:52:20.349869 1180 ts_tablet_manager.cc:1929] T 724d47241f0342768fb1321ef37c3a7b P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:20.369356 1180 log.cc:1199] T 724d47241f0342768fb1321ef37c3a7b P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/724d47241f0342768fb1321ef37c3a7b
I20260202 02:52:20.370293 1180 ts_tablet_manager.cc:1950] T 724d47241f0342768fb1321ef37c3a7b P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:20.370726 1173 raft_consensus.cc:359] T 0b411e8fab074aa88f8380e3f5694340 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.371937 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 230) took 1778 ms. Trace:
I20260202 02:52:20.372129 1173 raft_consensus.cc:385] T 0b411e8fab074aa88f8380e3f5694340 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:20.372423 1183 ts_tablet_manager.cc:1950] T 415d14533fd54f20adb00455a7bcad5f P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:20.373291 1173 raft_consensus.cc:740] T 0b411e8fab074aa88f8380e3f5694340 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:20.373605 1173 consensus_queue.cc:260] T 0b411e8fab074aa88f8380e3f5694340 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.374487 1180 rpcz_store.cc:276] 0202 02:52:18.593424 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:18.593910 (+ 486us) service_pool.cc:225] Handling call
0202 02:52:20.327847 (+1733937us) ts_tablet_manager.cc:1151] Deleting tablet 724d47241f0342768fb1321ef37c3a7b
0202 02:52:20.327861 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:20.346846 (+ 18985us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:20.349739 (+ 2893us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:20.371895 (+ 22156us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1733788}
I20260202 02:52:20.374938 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 415d14533fd54f20adb00455a7bcad5f (table default.Table_22489ecf56264d93993407ba7d0e6b55 [id=7c19fad4b9554c0682a01bbd3f73d198]) successfully deleted
I20260202 02:52:20.373126 1174 tablet_bootstrap.cc:492] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:20.375279 1174 ts_tablet_manager.cc:1403] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.042s user 0.002s sys 0.003s
I20260202 02:52:20.375903 1183 tablet_replica.cc:333] T c0937b6910534ddc8a2217f0cbd539c5 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:20.376435 1174 raft_consensus.cc:359] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.376637 1174 raft_consensus.cc:385] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:20.376729 1174 raft_consensus.cc:740] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:20.377104 1174 consensus_queue.cc:260] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.377660 1174 ts_tablet_manager.cc:1434] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:20.377959 1174 tablet_bootstrap.cc:492] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:20.378962 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 724d47241f0342768fb1321ef37c3a7b (table default.Table_5c9c0f09396a4a0e8b3d676fcb2dbb2c [id=6a47f828bd9c4d7aa7d385e4a1dda9ea]) successfully deleted
I20260202 02:52:20.381536 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 0b411e8fab074aa88f8380e3f5694340 with delete_type TABLET_DATA_DELETED (replaced by tablet 28a900bcaa8b41718a285e595886d805) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:20.381949 1173 ts_tablet_manager.cc:1434] T 0b411e8fab074aa88f8380e3f5694340 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.020s user 0.001s sys 0.001s
I20260202 02:52:20.382339 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 228) took 1789 ms. Trace:
I20260202 02:52:20.382195 1183 raft_consensus.cc:2243] T c0937b6910534ddc8a2217f0cbd539c5 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:20.384308 1173 tablet_bootstrap.cc:492] T ff6f5710de4b4c48948aacdf5be29e8b P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:20.384526 1183 raft_consensus.cc:2272] T c0937b6910534ddc8a2217f0cbd539c5 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:20.384914 1180 rpcz_store.cc:276] 0202 02:52:18.593195 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:18.594046 (+ 851us) service_pool.cc:225] Handling call
0202 02:52:20.382181 (+1788135us) ts_tablet_manager.cc:1151] Deleting tablet fae7e3941c124cc2871390a0b6471f27
0202 02:52:20.382193 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:20.382288 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1787952}
W20260202 02:52:20.385572 513 catalog_manager.cc:4952] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet fae7e3941c124cc2871390a0b6471f27 because the tablet was not found. No further retry: Not found: Tablet not found: fae7e3941c124cc2871390a0b6471f27
I20260202 02:52:20.385597 1183 ts_tablet_manager.cc:1916] T c0937b6910534ddc8a2217f0cbd539c5 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:20.386128 1180 tablet_replica.cc:333] T 952d760159844ad08269654264e6381b P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:20.386423 1180 raft_consensus.cc:2243] T 952d760159844ad08269654264e6381b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:20.388149 1180 raft_consensus.cc:2272] T 952d760159844ad08269654264e6381b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:20.383903 1174 tablet_bootstrap.cc:654] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:20.390465 526 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ad048ed673a5401b8ffa739c0f03b545" } modify_external_catalogs: true
I20260202 02:52:20.390760 526 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ad048ed673a5401b8ffa739c0f03b545" } modify_external_catalogs: true
I20260202 02:52:20.391366 1180 ts_tablet_manager.cc:1916] T 952d760159844ad08269654264e6381b P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:20.398728 526 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet ff6f5710de4b4c48948aacdf5be29e8b
I20260202 02:52:20.399500 522 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_53cc5731c3e24222a97702bdde84a954" } modify_external_catalogs: true
I20260202 02:52:20.399698 751 tablet_service.cc:1552] Processing DeleteTablet for tablet ff6f5710de4b4c48948aacdf5be29e8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:20.399747 522 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_53cc5731c3e24222a97702bdde84a954" } modify_external_catalogs: true
I20260202 02:52:20.399997 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_e71a7347855d447cabe6ea2cc64b293a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:20.400317 1173 tablet_bootstrap.cc:654] T ff6f5710de4b4c48948aacdf5be29e8b P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:20.400635 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e71a7347855d447cabe6ea2cc64b293a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:20.401158 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet ff6f5710de4b4c48948aacdf5be29e8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:20.413473 1173 tablet_bootstrap.cc:492] T ff6f5710de4b4c48948aacdf5be29e8b P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:20.413676 1173 ts_tablet_manager.cc:1403] T ff6f5710de4b4c48948aacdf5be29e8b P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.002s
I20260202 02:52:20.414577 1173 raft_consensus.cc:359] T ff6f5710de4b4c48948aacdf5be29e8b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:20.414819 1173 raft_consensus.cc:385] T ff6f5710de4b4c48948aacdf5be29e8b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:20.414896 1173 raft_consensus.cc:740] T ff6f5710de4b4c48948aacdf5be29e8b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:20.415159 1173 consensus_queue.cc:260] T ff6f5710de4b4c48948aacdf5be29e8b P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:20.415728 1173 ts_tablet_manager.cc:1434] T ff6f5710de4b4c48948aacdf5be29e8b P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260202 02:52:20.416103 1173 tablet_bootstrap.cc:492] T c63a670d5c5e489897bc6dbb142b844d P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:20.419103 526 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_e0149b31be0c49d19600de4849d6577e" } new_table_name: "default.Table_668e8ec9913f43fa8499eff32be8db0a" modify_external_catalogs: true
I20260202 02:52:20.419281 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_80e6b91d5978450dbfd9bb2b4b410d3c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:20.419868 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_80e6b91d5978450dbfd9bb2b4b410d3c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:20.421118 1174 tablet_bootstrap.cc:492] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:20.421279 1174 ts_tablet_manager.cc:1403] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.043s user 0.003s sys 0.002s
I20260202 02:52:20.421286 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_9d4fbf619567432a88aa9a0c1b19cb3e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:20.421808 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9d4fbf619567432a88aa9a0c1b19cb3e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:20.422443 1174 raft_consensus.cc:359] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.422626 1174 raft_consensus.cc:385] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:20.422698 1174 raft_consensus.cc:740] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:20.422991 1174 consensus_queue.cc:260] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.423494 1174 ts_tablet_manager.cc:1434] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:20.423835 1174 tablet_bootstrap.cc:492] T 75f7824681344d8d88df9402694fff1b P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:20.425339 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_92b7f492c09948f784c64ccdcaadbb86" } new_table_name: "default.Table_61c2ba441aa746edbb73f8b5de4204ba" modify_external_catalogs: true
W20260202 02:52:20.426638 526 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 44c8d5b74c8e4c499f7f68e8185efc17 has no consensus state
I20260202 02:52:20.428359 1180 ts_tablet_manager.cc:1929] T 952d760159844ad08269654264e6381b P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:20.428519 1180 log.cc:1199] T 952d760159844ad08269654264e6381b P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/952d760159844ad08269654264e6381b
I20260202 02:52:20.428925 1180 ts_tablet_manager.cc:1950] T 952d760159844ad08269654264e6381b P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:20.429735 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 231) took 1836 ms. Trace:
I20260202 02:52:20.429847 1180 rpcz_store.cc:276] 0202 02:52:18.593512 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:18.594253 (+ 741us) service_pool.cc:225] Handling call
0202 02:52:20.385967 (+1791714us) ts_tablet_manager.cc:1151] Deleting tablet 952d760159844ad08269654264e6381b
0202 02:52:20.385980 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:20.404586 (+ 18606us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:20.428183 (+ 23597us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:20.429675 (+ 1492us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1791534}
I20260202 02:52:20.430534 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 952d760159844ad08269654264e6381b (table default.Table_b6c1fb1e7282492d920385869810b0e0 [id=2cc4567c7bd44745a5851cd8b95b22c4]) successfully deleted
I20260202 02:52:20.431756 1173 tablet_bootstrap.cc:654] T c63a670d5c5e489897bc6dbb142b844d P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:20.433090 1174 tablet_bootstrap.cc:654] T 75f7824681344d8d88df9402694fff1b P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:20.437776 527 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a1f39e5b02f747819e636cddb4292eb6 consensus state has no leader
I20260202 02:52:20.438961 1180 tablet_replica.cc:333] T bf74f96bea2e4ab7adec42c3c5207e79 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:20.439555 1180 raft_consensus.cc:2243] T bf74f96bea2e4ab7adec42c3c5207e79 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:20.440565 1180 raft_consensus.cc:2272] T bf74f96bea2e4ab7adec42c3c5207e79 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:20.441484 1180 ts_tablet_manager.cc:1916] T bf74f96bea2e4ab7adec42c3c5207e79 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:20.448544 523 master_service.cc:954] ReplaceTablet: received request to replace tablet c63a670d5c5e489897bc6dbb142b844d from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:20.450212 1173 tablet_bootstrap.cc:492] T c63a670d5c5e489897bc6dbb142b844d P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:20.450407 1173 ts_tablet_manager.cc:1403] T c63a670d5c5e489897bc6dbb142b844d P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.034s user 0.002s sys 0.002s
I20260202 02:52:20.451627 1183 ts_tablet_manager.cc:1929] T c0937b6910534ddc8a2217f0cbd539c5 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:20.456991 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_444485558ccc4c6ba771ab686bc9e1c1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:20.457912 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_444485558ccc4c6ba771ab686bc9e1c1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:20.458333 1277 raft_consensus.cc:493] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:20.458565 1277 raft_consensus.cc:515] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.459335 519 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 06b80a1c9251420cbc16fc95ae98aca9
I20260202 02:52:20.459750 519 catalog_manager.cc:6724] ReplaceTablet: tablet 06b80a1c9251420cbc16fc95ae98aca9 deleted and replaced by tablet 2f5aae089fcc492f9ce55f84674a452c
I20260202 02:52:20.460561 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 06b80a1c9251420cbc16fc95ae98aca9 with delete_type TABLET_DATA_DELETED (replaced by tablet 2f5aae089fcc492f9ce55f84674a452c) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:20.461124 1277 leader_election.cc:290] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:20.463315 1180 ts_tablet_manager.cc:1929] T bf74f96bea2e4ab7adec42c3c5207e79 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:20.463917 1180 log.cc:1199] T bf74f96bea2e4ab7adec42c3c5207e79 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/bf74f96bea2e4ab7adec42c3c5207e79
I20260202 02:52:20.466020 1180 ts_tablet_manager.cc:1950] T bf74f96bea2e4ab7adec42c3c5207e79 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:20.466769 519 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_fe06d3b0b66447e29403a54ab14c476c" } new_table_name: "default.Table_950f6f5ae1ef43399de785df25c896e9" modify_external_catalogs: true
I20260202 02:52:20.467861 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 233) took 1874 ms. Trace:
I20260202 02:52:20.468314 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_35c5c99502d0489081751bc7403f6ad7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:20.468887 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_35c5c99502d0489081751bc7403f6ad7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:20.471108 1180 rpcz_store.cc:276] 0202 02:52:18.593683 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:18.594075 (+ 392us) service_pool.cc:225] Handling call
0202 02:52:20.438847 (+1844772us) ts_tablet_manager.cc:1151] Deleting tablet bf74f96bea2e4ab7adec42c3c5207e79
0202 02:52:20.438858 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:20.446012 (+ 7154us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:20.463148 (+ 17136us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:20.467821 (+ 4673us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1844321}
I20260202 02:52:20.472092 1180 tablet_replica.cc:333] T 0d297b6243c348ee82c146a8339cf6c4 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:20.472357 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet bf74f96bea2e4ab7adec42c3c5207e79 (table default.Table_d142ff2cb10e491997bf0e302d6a9fa5 [id=a0b8442c02974d6596db002c9292be0e]) successfully deleted
I20260202 02:52:20.472882 1180 raft_consensus.cc:2243] T 0d297b6243c348ee82c146a8339cf6c4 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:20.473358 1180 raft_consensus.cc:2272] T 0d297b6243c348ee82c146a8339cf6c4 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:20.474228 1180 ts_tablet_manager.cc:1916] T 0d297b6243c348ee82c146a8339cf6c4 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:20.474969 1173 raft_consensus.cc:359] T c63a670d5c5e489897bc6dbb142b844d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.475286 1173 raft_consensus.cc:385] T c63a670d5c5e489897bc6dbb142b844d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:20.475821 524 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_614c0ff5900b4d64b9c0af8c88a33ecb" } new_table_name: "default.Table_b88bd2e65dd140d5a1c208c9d4aa9430" modify_external_catalogs: true
I20260202 02:52:20.477758 1064 raft_consensus.cc:3060] T ff6f5710de4b4c48948aacdf5be29e8b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:20.478274 1173 raft_consensus.cc:740] T c63a670d5c5e489897bc6dbb142b844d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:20.479215 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_2e2049e67b8943acab5a0db3deb1535a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:20.479773 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2e2049e67b8943acab5a0db3deb1535a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:20.479745 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 06b80a1c9251420cbc16fc95ae98aca9 with delete_type TABLET_DATA_DELETED (replaced by tablet 2f5aae089fcc492f9ce55f84674a452c) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:20.479707 1173 consensus_queue.cc:260] T c63a670d5c5e489897bc6dbb142b844d P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.479923 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "267bd4752c5f4b99a3fe778ba8f38a2f" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:20.481031 715 leader_election.cc:343] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:20.482213 519 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 042d1ebf357b4abe8814a407891ce265 consensus state has no leader
I20260202 02:52:20.482590 519 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_87277c5d80184738b96b773f026b32a5" } modify_external_catalogs: true
I20260202 02:52:20.482764 519 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_87277c5d80184738b96b773f026b32a5" } modify_external_catalogs: true
I20260202 02:52:20.482973 1173 ts_tablet_manager.cc:1434] T c63a670d5c5e489897bc6dbb142b844d P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.030s user 0.002s sys 0.002s
I20260202 02:52:20.483400 1173 tablet_bootstrap.cc:492] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:20.483585 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_a8c4f67a58b74bf8bb3716174ed5654f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:20.484114 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a8c4f67a58b74bf8bb3716174ed5654f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:20.500011 1183 log.cc:1199] T c0937b6910534ddc8a2217f0cbd539c5 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/c0937b6910534ddc8a2217f0cbd539c5
I20260202 02:52:20.501636 1173 tablet_bootstrap.cc:654] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:20.504173 1183 ts_tablet_manager.cc:1950] T c0937b6910534ddc8a2217f0cbd539c5 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
W20260202 02:52:20.506646 524 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2b8ba9dd450948ada395207ae1c5b325 consensus state has no leader
I20260202 02:52:20.507287 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_49cb996504bf45e9a24c6f3c01f457ff"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:20.508386 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_49cb996504bf45e9a24c6f3c01f457ff in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:20.510175 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_a69dc5f8fd7e4c15b218f332ac4a74e2" } new_table_name: "default.Table_7a967fd89a5243c4a02abdaf16f6d62e" modify_external_catalogs: true
I20260202 02:52:20.512728 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_516d610640654ea4b49f7de3b04cbb0c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:20.513720 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_516d610640654ea4b49f7de3b04cbb0c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:20.517328 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet c0937b6910534ddc8a2217f0cbd539c5 (table default.Table_668e8ec9913f43fa8499eff32be8db0a [id=46b5d47377474ff9bac2acdb26b08c65]) successfully deleted
I20260202 02:52:20.523530 520 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet 78f122342dd144b48de89902faa0f94f (table default.Table_5eefa56618e84f8d92f8d87132ccd7d4 [id=b4ab2ba86e1a49338a98d855ded99f5d]). Expected version 1 got 0
W20260202 02:52:20.528687 527 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet dada209d982d48f3b74a49905f19c5b6 consensus state has no leader
I20260202 02:52:20.519219 1183 tablet_replica.cc:333] T cb0dcacdf0f1419da0b25f20cac1d62d P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:20.530992 751 tablet_service.cc:1505] Processing CreateTablet for tablet b7880cb051c24bc998003f0effed3cf8 (DEFAULT_TABLE table=default.Table_a0f1f2d30d064244bec97181c11c3a06 [id=1787be853d4a40cdada2c2735cad1a01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.532651 1183 raft_consensus.cc:2243] T cb0dcacdf0f1419da0b25f20cac1d62d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:20.534282 748 tablet_service.cc:1505] Processing CreateTablet for tablet 6b8f924aecf44e959e47d2f32e6b0bf5 (DEFAULT_TABLE table=default.Table_4532505bd70548f7b20362dc9629dbee [id=32826e501f3440589641affa98a4c82d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.536777 756 tablet_service.cc:1505] Processing CreateTablet for tablet 8bb14bf551f441dcbaaff7057b51b2be (DEFAULT_TABLE table=default.Table_7a314585590647ca8fd0f51e125b0622 [id=0e44d4587c694edc90c651e2f9db649b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.538158 1044 tablet_service.cc:1505] Processing CreateTablet for tablet b7880cb051c24bc998003f0effed3cf8 (DEFAULT_TABLE table=default.Table_a0f1f2d30d064244bec97181c11c3a06 [id=1787be853d4a40cdada2c2735cad1a01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.538607 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7880cb051c24bc998003f0effed3cf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.540531 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7880cb051c24bc998003f0effed3cf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.541003 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bb14bf551f441dcbaaff7057b51b2be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.541222 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b8f924aecf44e959e47d2f32e6b0bf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.541850 1173 tablet_bootstrap.cc:492] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:20.542162 1173 ts_tablet_manager.cc:1403] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.059s user 0.003s sys 0.010s
I20260202 02:52:20.543502 1173 raft_consensus.cc:359] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.543833 1173 raft_consensus.cc:385] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:20.543948 1173 raft_consensus.cc:740] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:20.544384 1173 consensus_queue.cc:260] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.545254 1173 ts_tablet_manager.cc:1434] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260202 02:52:20.545583 1173 tablet_bootstrap.cc:492] T b76d1837813c46e8a0925b49939d8884 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:20.547217 1173 tablet_bootstrap.cc:654] T b76d1837813c46e8a0925b49939d8884 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:20.551452 1180 ts_tablet_manager.cc:1929] T 0d297b6243c348ee82c146a8339cf6c4 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:20.535719 1183 raft_consensus.cc:2272] T cb0dcacdf0f1419da0b25f20cac1d62d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:20.559145 527 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet 78f122342dd144b48de89902faa0f94f (table default.Table_5eefa56618e84f8d92f8d87132ccd7d4 [id=b4ab2ba86e1a49338a98d855ded99f5d]). Expected version 1 got 0
I20260202 02:52:20.560894 1617 catalog_manager.cc:1269] Loaded cluster ID: bdfd876443214aa8b701e616e981c0d4
I20260202 02:52:20.561058 1617 catalog_manager.cc:1562] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb: loading cluster ID for follower catalog manager: success
I20260202 02:52:20.564914 1680 tablet.cc:1722] T ff6f5710de4b4c48948aacdf5be29e8b P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:20.568086 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 6b8f924aecf44e959e47d2f32e6b0bf5 (DEFAULT_TABLE table=default.Table_4532505bd70548f7b20362dc9629dbee [id=32826e501f3440589641affa98a4c82d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.568845 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b8f924aecf44e959e47d2f32e6b0bf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.568899 1042 tablet_service.cc:1505] Processing CreateTablet for tablet 8bb14bf551f441dcbaaff7057b51b2be (DEFAULT_TABLE table=default.Table_7a314585590647ca8fd0f51e125b0622 [id=0e44d4587c694edc90c651e2f9db649b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.569893 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bb14bf551f441dcbaaff7057b51b2be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.577575 1063 raft_consensus.cc:3060] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:20.580403 1183 ts_tablet_manager.cc:1916] T cb0dcacdf0f1419da0b25f20cac1d62d P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:20.582307 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_52a2f389faba40fea93e2bca972e81e1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:20.584708 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_52a2f389faba40fea93e2bca972e81e1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:20.586838 1173 tablet_bootstrap.cc:492] T b76d1837813c46e8a0925b49939d8884 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:20.587013 1173 ts_tablet_manager.cc:1403] T b76d1837813c46e8a0925b49939d8884 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.042s user 0.002s sys 0.001s
I20260202 02:52:20.587845 1173 raft_consensus.cc:359] T b76d1837813c46e8a0925b49939d8884 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.588057 1173 raft_consensus.cc:385] T b76d1837813c46e8a0925b49939d8884 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:20.588166 1173 raft_consensus.cc:740] T b76d1837813c46e8a0925b49939d8884 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:20.588459 1173 consensus_queue.cc:260] T b76d1837813c46e8a0925b49939d8884 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.589033 1173 ts_tablet_manager.cc:1434] T b76d1837813c46e8a0925b49939d8884 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:20.589371 1173 tablet_bootstrap.cc:492] T d2fa4c3d6ec44047abf5986e502c1590 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:20.591838 1173 tablet_bootstrap.cc:654] T d2fa4c3d6ec44047abf5986e502c1590 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:20.594461 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 9884c90cf1d245b2a7eaaf00d71b1c3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:20.598767 1600 raft_consensus.cc:493] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:20.599033 1600 raft_consensus.cc:515] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.599912 1600 leader_election.cc:290] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:20.600222 1180 log.cc:1199] T 0d297b6243c348ee82c146a8339cf6c4 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/0d297b6243c348ee82c146a8339cf6c4
I20260202 02:52:20.601934 1174 tablet_bootstrap.cc:492] T 75f7824681344d8d88df9402694fff1b P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:20.602133 1174 ts_tablet_manager.cc:1403] T 75f7824681344d8d88df9402694fff1b P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.178s user 0.010s sys 0.045s
I20260202 02:52:20.602679 528 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_f93d759396c04d25ac602a609ecc4962" } modify_external_catalogs: true
I20260202 02:52:20.602877 528 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_f93d759396c04d25ac602a609ecc4962" } modify_external_catalogs: true
I20260202 02:52:20.602937 1174 raft_consensus.cc:359] T 75f7824681344d8d88df9402694fff1b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.603123 1174 raft_consensus.cc:385] T 75f7824681344d8d88df9402694fff1b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:20.603202 1174 raft_consensus.cc:740] T 75f7824681344d8d88df9402694fff1b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:20.601955 1180 ts_tablet_manager.cc:1950] T 0d297b6243c348ee82c146a8339cf6c4 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:20.604226 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 226) took 2011 ms. Trace:
I20260202 02:52:20.602360 1057 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b428356fba0c4fb5a6033c5d39740cc5" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:20.604435 1180 rpcz_store.cc:276] 0202 02:52:18.592963 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:18.594536 (+ 1573us) service_pool.cc:225] Handling call
0202 02:52:20.472019 (+1877483us) ts_tablet_manager.cc:1151] Deleting tablet 0d297b6243c348ee82c146a8339cf6c4
0202 02:52:20.472031 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:20.494054 (+ 22023us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:20.551314 (+ 57260us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:20.604166 (+ 52852us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1877326}
I20260202 02:52:20.604938 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 0d297b6243c348ee82c146a8339cf6c4 (table default.Table_0c83932cab13463f9cca28129f18b576 [id=7c76848b98614fbf8b2950e5d55cc609]) successfully deleted
I20260202 02:52:20.605485 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 224) took 2012 ms. Trace:
I20260202 02:52:20.606724 1453 raft_consensus.cc:2749] T b428356fba0c4fb5a6033c5d39740cc5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:20.610632 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_8262b850970641b1bbe1cf9d1c3d9409"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:20.611359 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8262b850970641b1bbe1cf9d1c3d9409 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:20.616215 1453 raft_consensus.cc:2749] T 0186572e7bb9499a932a636ab280f544 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:20.616384 715 leader_election.cc:343] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:20.617969 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 227) took 1769 ms. Trace:
I20260202 02:52:20.618093 1182 rpcz_store.cc:276] 0202 02:52:18.847387 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:18.847441 (+ 54us) service_pool.cc:225] Handling call
0202 02:52:20.310830 (+1463389us) ts_tablet_manager.cc:1151] Deleting tablet 251455168e33430aafea8e0de42ca1b8
0202 02:52:20.310846 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:20.340426 (+ 29580us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:20.346315 (+ 5889us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:20.617098 (+270783us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":257096,"tablet-delete.queue_time_us":1463010}
I20260202 02:52:20.617286 1174 consensus_queue.cc:260] T 75f7824681344d8d88df9402694fff1b P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.619393 525 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 072bb7f0e4ab4935be49461bc8a3c163
I20260202 02:52:20.620180 1173 tablet_bootstrap.cc:492] T d2fa4c3d6ec44047abf5986e502c1590 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:20.620379 1173 ts_tablet_manager.cc:1403] T d2fa4c3d6ec44047abf5986e502c1590 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.031s user 0.000s sys 0.010s
I20260202 02:52:20.621256 1173 raft_consensus.cc:359] T d2fa4c3d6ec44047abf5986e502c1590 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.621454 1173 raft_consensus.cc:385] T d2fa4c3d6ec44047abf5986e502c1590 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:20.621572 1173 raft_consensus.cc:740] T d2fa4c3d6ec44047abf5986e502c1590 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:20.621812 1173 consensus_queue.cc:260] T d2fa4c3d6ec44047abf5986e502c1590 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.621582 1042 tablet_service.cc:1552] Processing DeleteTablet for tablet 072bb7f0e4ab4935be49461bc8a3c163 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:20.622635 1173 ts_tablet_manager.cc:1434] T d2fa4c3d6ec44047abf5986e502c1590 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260202 02:52:20.622946 1173 tablet_bootstrap.cc:492] T 583e547d0a1943f09f4efe82070fc777 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:20.623387 521 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 3711910e050c4515bed4294d1271d5a9
I20260202 02:52:20.625267 1173 tablet_bootstrap.cc:654] T 583e547d0a1943f09f4efe82070fc777 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:20.625771 1042 tablet_service.cc:1552] Processing DeleteTablet for tablet 3711910e050c4515bed4294d1271d5a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:20.626919 1180 rpcz_store.cc:276] 0202 02:52:18.592752 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:18.594760 (+ 2008us) service_pool.cc:225] Handling call
0202 02:52:20.605312 (+2010552us) ts_tablet_manager.cc:1151] Deleting tablet c1fc01252d95407e954cfcbf7e715912
0202 02:52:20.605324 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:20.605438 (+ 114us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2010373}
I20260202 02:52:20.629061 525 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_8e6ef9e6965c47f2ac2eb775c1e60dc8" } new_table_name: "default.Table_f93837f1f1614be4bb992e92e5237ea2" modify_external_catalogs: true
W20260202 02:52:20.629359 513 catalog_manager.cc:4952] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet c1fc01252d95407e954cfcbf7e715912 because the tablet was not found. No further retry: Not found: Tablet not found: c1fc01252d95407e954cfcbf7e715912
I20260202 02:52:20.632035 1174 ts_tablet_manager.cc:1434] T 75f7824681344d8d88df9402694fff1b P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.017s user 0.001s sys 0.001s
I20260202 02:52:20.632037 748 tablet_service.cc:1552] Processing DeleteTablet for tablet 072bb7f0e4ab4935be49461bc8a3c163 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:20.633806 1180 tablet_replica.cc:333] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:20.634172 1600 raft_consensus.cc:493] T 70306ed9dec54286885d965f2c012da4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:20.634577 1174 tablet_bootstrap.cc:492] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:20.634578 1600 raft_consensus.cc:515] T 70306ed9dec54286885d965f2c012da4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.635304 1600 leader_election.cc:290] T 70306ed9dec54286885d965f2c012da4 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:20.636394 1057 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "70306ed9dec54286885d965f2c012da4" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:20.643695 1183 ts_tablet_manager.cc:1929] T cb0dcacdf0f1419da0b25f20cac1d62d P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:20.646479 519 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 38fdbc6df7ec40b9afe4e0dfe0119e3e
I20260202 02:52:20.647673 523 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet c63a670d5c5e489897bc6dbb142b844d
I20260202 02:52:20.648171 523 catalog_manager.cc:6724] ReplaceTablet: tablet c63a670d5c5e489897bc6dbb142b844d deleted and replaced by tablet 939fbcaf75c7439d9208bbbf83a72717
I20260202 02:52:20.648654 522 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet d0671586e45e45ffb7fd9e1be6e2bbf8
I20260202 02:52:20.658504 1175 ts_tablet_manager.cc:1434] T 757492288d7c44fb91b25b7e6e21e01b P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.314s user 0.002s sys 0.001s
I20260202 02:52:20.658986 1175 tablet_bootstrap.cc:492] T ef073880898049e2be93d2cb6dc0c0bb P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:20.659883 1655 raft_consensus.cc:493] T 80d443bdb66d4cc68ae283e08c410fe6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:20.660056 1655 raft_consensus.cc:515] T 80d443bdb66d4cc68ae283e08c410fe6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.660808 1655 leader_election.cc:290] T 80d443bdb66d4cc68ae283e08c410fe6 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:20.661051 1182 tablet_replica.cc:333] T ffd8aa2bc6e344b78dce518411997fb5 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:20.661258 1182 raft_consensus.cc:2243] T ffd8aa2bc6e344b78dce518411997fb5 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:20.661487 1182 raft_consensus.cc:2272] T ffd8aa2bc6e344b78dce518411997fb5 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:20.662173 1182 ts_tablet_manager.cc:1916] T ffd8aa2bc6e344b78dce518411997fb5 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:20.664615 857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 287233us
I20260202 02:52:20.665174 857 leader_election.cc:304] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
W20260202 02:52:20.665422 857 raft_consensus.cc:2686] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1521a6d262b7451bbef8debe970dd6b5: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260202 02:52:20.666270 857 leader_election.cc:343] T 75f7824681344d8d88df9402694fff1b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:20.666421 857 leader_election.cc:304] T 75f7824681344d8d88df9402694fff1b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
W20260202 02:52:20.666790 857 leader_election.cc:343] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:20.666908 857 leader_election.cc:304] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:20.667171 1663 consensus_queue.cc:1048] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:20.667709 1663 consensus_queue.cc:1048] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:20.668105 1663 raft_consensus.cc:2749] T 75f7824681344d8d88df9402694fff1b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:20.668269 1663 raft_consensus.cc:2749] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:20.669724 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_33fc32ec03094e3c8fe78de4f6a26777"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:20.646931 1042 tablet_service.cc:1505] Processing CreateTablet for tablet d8418a9dd0944456b4d4820dde0cde02 (DEFAULT_TABLE table=default.Table_2d42dcf81d2d4b2c90fe09860b9b2318 [id=e620930eae8e4707b93e7ab16369e13d]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:20.670295 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_33fc32ec03094e3c8fe78de4f6a26777 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:20.670413 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8418a9dd0944456b4d4820dde0cde02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.670580 1183 log.cc:1199] T cb0dcacdf0f1419da0b25f20cac1d62d P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/cb0dcacdf0f1419da0b25f20cac1d62d
I20260202 02:52:20.671013 1183 ts_tablet_manager.cc:1950] T cb0dcacdf0f1419da0b25f20cac1d62d P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:20.671797 523 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_983bf3e0c5424c2588a0e754af1a5687" } new_table_name: "default.Table_cc069a0949364216b8f08d86a0d9c776" modify_external_catalogs: true
I20260202 02:52:20.671947 1183 tablet_replica.cc:333] T 0b411e8fab074aa88f8380e3f5694340 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:20.672140 1183 raft_consensus.cc:2243] T 0b411e8fab074aa88f8380e3f5694340 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:20.653040 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 38fdbc6df7ec40b9afe4e0dfe0119e3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:20.672304 1183 raft_consensus.cc:2272] T 0b411e8fab074aa88f8380e3f5694340 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:20.653236 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet d0671586e45e45ffb7fd9e1be6e2bbf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:20.653396 1037 tablet_service.cc:1552] Processing DeleteTablet for tablet c63a670d5c5e489897bc6dbb142b844d with delete_type TABLET_DATA_DELETED (replaced by tablet 939fbcaf75c7439d9208bbbf83a72717) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:20.657876 1040 tablet_service.cc:1505] Processing CreateTablet for tablet 1116872cc25a4ab5b1a3a6bbba1f0da7 (DEFAULT_TABLE table=default.Table_242f4cdb6f514121a1a113910a2aaec8 [id=52a6d31f993d4c74a520d504390cf12b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.672945 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1116872cc25a4ab5b1a3a6bbba1f0da7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.673307 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_67c55c53ca254b03b0c326672a41b559"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:20.673898 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_67c55c53ca254b03b0c326672a41b559 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:20.675663 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet cb0dcacdf0f1419da0b25f20cac1d62d (table default.Table_c245b83c1987454fabbe8646d14252d6 [id=08498a654b6a4a9ca3f968d70daad9b6]) successfully deleted
I20260202 02:52:20.678900 528 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 026e6ee0b3ae4d938fdf1550044ed1fa
W20260202 02:52:20.678929 525 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8b0368e03f644f1db28dbb57c95c48c1 has no consensus state
I20260202 02:52:20.680846 1038 tablet_service.cc:1505] Processing CreateTablet for tablet 2c31b45dc8004eae993e972e9faa3ff8 (DEFAULT_TABLE table=default.Table_ba9c16aaa48849db9fece59168c16886 [id=f1f6c97fc88340da8bab0347ad6430ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.681314 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c31b45dc8004eae993e972e9faa3ff8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.685410 1037 tablet_service.cc:1505] Processing CreateTablet for tablet f94f25f4059a43b8babedb6b45a08a19 (DEFAULT_TABLE table=default.Table_22445ccfd7b64298b92a0f70733f3625 [id=300ea019d2724bee8211d7462710dc26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.685871 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f94f25f4059a43b8babedb6b45a08a19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.686067 1617 catalog_manager.cc:1584] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb: acquiring CA information for follower catalog manager: success
I20260202 02:52:20.687351 1656 raft_consensus.cc:493] T 757492288d7c44fb91b25b7e6e21e01b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:20.687565 1656 raft_consensus.cc:515] T 757492288d7c44fb91b25b7e6e21e01b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.688270 1656 leader_election.cc:290] T 757492288d7c44fb91b25b7e6e21e01b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:20.690388 1043 tablet_service.cc:1552] Processing DeleteTablet for tablet 026e6ee0b3ae4d938fdf1550044ed1fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:20.690894 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 550895b4010f4088b2cc34acff50e03f (DEFAULT_TABLE table=default.Table_12db441a8c934f9780599755191c58f5 [id=427f58cc70ef47a6afac8de0353975f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.691339 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 550895b4010f4088b2cc34acff50e03f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.692000 524 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_25c6d490f243450ab2d9b43ec9cda7bf" } new_table_name: "default.Table_7950bdb0cab248acb118e0ebdf92d83d" modify_external_catalogs: true
I20260202 02:52:20.693193 1175 tablet_bootstrap.cc:654] T ef073880898049e2be93d2cb6dc0c0bb P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:20.698889 523 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cb634d4bae554ef3b39eeaa9958adc6c consensus state has no leader
I20260202 02:52:20.700947 1043 tablet_service.cc:1505] Processing CreateTablet for tablet ccd6bc8bd0504bf18d108c73685ef9ff (DEFAULT_TABLE table=default.Table_5ecb81038c9f48438e74fe874fc83e74 [id=c28e30416db3407b8c20c68091f392b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.700907 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_8fcc3828b43444159f0df6023ec1ba0f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:20.701380 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccd6bc8bd0504bf18d108c73685ef9ff. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:20.701557 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8fcc3828b43444159f0df6023ec1ba0f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:20.702751 1183 ts_tablet_manager.cc:1916] T 0b411e8fab074aa88f8380e3f5694340 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:20.711825 523 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_524aa17b0b964c9e985e643ee9f12e94" } new_table_name: "default.Table_c6e682a233904983978e7abb23565e41" modify_external_catalogs: true
I20260202 02:52:20.714381 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_c6515f88263042feadae47be333ed860"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:20.715032 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c6515f88263042feadae47be333ed860 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:20.716828 515 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet dada209d982d48f3b74a49905f19c5b6 (table default.Table_7a967fd89a5243c4a02abdaf16f6d62e [id=116247212fca4c0a95ad1bdbbac4787f], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet dada209d982d48f3b74a49905f19c5b6 consensus state has no leader
I20260202 02:52:20.719722 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_e984fe23a3ab488c9344620087da422d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:20.720371 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e984fe23a3ab488c9344620087da422d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:20.722569 1178 raft_consensus.cc:493] T d2fa4c3d6ec44047abf5986e502c1590 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:20.722847 919 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c63a670d5c5e489897bc6dbb142b844d" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:20.723073 919 raft_consensus.cc:3060] T c63a670d5c5e489897bc6dbb142b844d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:20.726446 919 raft_consensus.cc:2468] T c63a670d5c5e489897bc6dbb142b844d P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:20.730370 919 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0d3633f5f9f54115bc78b9c1b78aba67" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:20.730633 919 raft_consensus.cc:3060] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:20.733065 1178 raft_consensus.cc:515] T d2fa4c3d6ec44047abf5986e502c1590 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.733456 919 raft_consensus.cc:2468] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:20.733841 1178 leader_election.cc:290] T d2fa4c3d6ec44047abf5986e502c1590 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:20.739837 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_ec2d6011c0c6475d9cd5a11c6f94d5d6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:20.740536 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ec2d6011c0c6475d9cd5a11c6f94d5d6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:20.740816 1034 tablet_service.cc:1505] Processing CreateTablet for tablet 474df41e4fe24a818fa45a4ce84d5474 (DEFAULT_TABLE table=default.Table_5ed512e7fc01454392b327c54420ec55 [id=14bbe505bc8b4a4bbde264a2c0dcdf50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.741276 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 474df41e4fe24a818fa45a4ce84d5474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.742429 526 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_242f4cdb6f514121a1a113910a2aaec8" } new_table_name: "default.Table_6132c5ab92374585bde17a139c7cc390" modify_external_catalogs: true
I20260202 02:52:20.745000 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_a9b816e5b5b84151b840bfad315deb70"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:20.745513 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a9b816e5b5b84151b840bfad315deb70 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:20.718180 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_bcf2aa761e104360bb4a1251204a6608"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:20.747165 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bcf2aa761e104360bb4a1251204a6608 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:20.749989 1183 ts_tablet_manager.cc:1929] T 0b411e8fab074aa88f8380e3f5694340 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:20.758693 920 raft_consensus.cc:1275] T ff6f5710de4b4c48948aacdf5be29e8b P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Refusing update from remote peer 1f442ef85029461091de847118c34c0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260202 02:52:20.760241 920 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "80d443bdb66d4cc68ae283e08c410fe6" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:20.760504 920 raft_consensus.cc:2468] T 80d443bdb66d4cc68ae283e08c410fe6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:20.761639 920 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b428356fba0c4fb5a6033c5d39740cc5" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:20.761874 920 raft_consensus.cc:2468] T b428356fba0c4fb5a6033c5d39740cc5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:20.762421 920 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "70306ed9dec54286885d965f2c012da4" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:20.762620 920 raft_consensus.cc:2468] T 70306ed9dec54286885d965f2c012da4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
W20260202 02:52:20.764140 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet ea040c03420c4b1b98897f39160b4130 because the tablet was not found. No further retry: Not found: Tablet not found: ea040c03420c4b1b98897f39160b4130
W20260202 02:52:20.764408 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet fae7e3941c124cc2871390a0b6471f27 because the tablet was not found. No further retry: Not found: Tablet not found: fae7e3941c124cc2871390a0b6471f27
I20260202 02:52:20.767772 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet c064ea0dade840b4840a5715c2d83f69 (table default.Table_5fa327d665484811934aa3b11188a76d [id=f7c5b24c065443d89a2b1f9c34409c9d]) successfully deleted
W20260202 02:52:20.768050 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet bf5ca99708044482970cb854d2eced18 because the tablet was not found. No further retry: Not found: Tablet not found: bf5ca99708044482970cb854d2eced18
I20260202 02:52:20.768270 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 251455168e33430aafea8e0de42ca1b8 (table default.Table_ce2d44550fb5403a99372bc5d3d79265 [id=4f19275f416a425c9b9d100b2a83870a]) successfully deleted
I20260202 02:52:20.769054 1182 ts_tablet_manager.cc:1929] T ffd8aa2bc6e344b78dce518411997fb5 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:20.769285 1182 log.cc:1199] T ffd8aa2bc6e344b78dce518411997fb5 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/ffd8aa2bc6e344b78dce518411997fb5
I20260202 02:52:20.769815 1182 ts_tablet_manager.cc:1950] T ffd8aa2bc6e344b78dce518411997fb5 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:20.770264 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 228) took 1922 ms. Trace:
I20260202 02:52:20.751118 1044 tablet_service.cc:1505] Processing CreateTablet for tablet fbcb4a522df648ba9542250450c08f1b (DEFAULT_TABLE table=default.Table_499eb7d77ae34d88ad956bf4a5c55648 [id=163eefc3d38a4b28bce88b948cc5df3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.786389 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbcb4a522df648ba9542250450c08f1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.752179 1038 tablet_service.cc:1505] Processing CreateTablet for tablet 29c996781cb947f3a1aee5dd0a9c4828 (DEFAULT_TABLE table=default.Table_458f570486154241ad8c7f10a8135ace [id=d2eb691b1287473990f0a66d02f71554]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.752091 1040 tablet_service.cc:1505] Processing CreateTablet for tablet 842e54adae2a452a98521de24326eaff (DEFAULT_TABLE table=default.Table_524aa17b0b964c9e985e643ee9f12e94 [id=0d7624b55c1a4bf0a449f651ac419d77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.753809 1680 tablet.cc:1722] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260202 02:52:20.763204 999 leader_election.cc:343] T 560465c063d94ffaab33f9f79a58a357 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 560465c063d94ffaab33f9f79a58a357
I20260202 02:52:20.788767 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29c996781cb947f3a1aee5dd0a9c4828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.773252 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "80d443bdb66d4cc68ae283e08c410fe6" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:20.774019 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "757492288d7c44fb91b25b7e6e21e01b" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:20.790098 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 842e54adae2a452a98521de24326eaff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.774771 1062 raft_consensus.cc:3060] T b76d1837813c46e8a0925b49939d8884 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:20.779556 1042 tablet_service.cc:1505] Processing CreateTablet for tablet 6df3d11cf578404ca41807f0777b6c53 (DEFAULT_TABLE table=default.Table_d1815e87de114d408122224276266f69 [id=4a43e638e8854fcda2702463684c08d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.792644 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6df3d11cf578404ca41807f0777b6c53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.796011 1180 raft_consensus.cc:2243] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
W20260202 02:52:20.796504 715 leader_election.cc:343] T 70306ed9dec54286885d965f2c012da4 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:20.797787 753 tablet_service.cc:1552] Processing DeleteTablet for tablet c63a670d5c5e489897bc6dbb142b844d with delete_type TABLET_DATA_DELETED (replaced by tablet 939fbcaf75c7439d9208bbbf83a72717) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:20.798127 758 tablet_service.cc:1505] Processing CreateTablet for tablet 1116872cc25a4ab5b1a3a6bbba1f0da7 (DEFAULT_TABLE table=default.Table_242f4cdb6f514121a1a113910a2aaec8 [id=52a6d31f993d4c74a520d504390cf12b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.798202 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 3711910e050c4515bed4294d1271d5a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:42950
W20260202 02:52:20.801415 715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 159387us
I20260202 02:52:20.801461 756 tablet_service.cc:1505] Processing CreateTablet for tablet d8418a9dd0944456b4d4820dde0cde02 (DEFAULT_TABLE table=default.Table_2d42dcf81d2d4b2c90fe09860b9b2318 [id=e620930eae8e4707b93e7ab16369e13d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.801709 750 tablet_service.cc:1552] Processing DeleteTablet for tablet d0671586e45e45ffb7fd9e1be6e2bbf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:20.801327 1180 pending_rounds.cc:70] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d: Trying to abort 1 pending ops.
I20260202 02:52:20.802635 1180 pending_rounds.cc:77] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923025145180160 op_type: NO_OP noop_request { }
I20260202 02:52:20.802754 1180 raft_consensus.cc:2889] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:20.802881 1180 raft_consensus.cc:2272] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:20.803570 1180 ts_tablet_manager.cc:1916] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:20.804061 523 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 842e54adae2a452a98521de24326eaff consensus state has no leader
I20260202 02:52:20.807950 1183 log.cc:1199] T 0b411e8fab074aa88f8380e3f5694340 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/0b411e8fab074aa88f8380e3f5694340
I20260202 02:52:20.808496 1183 ts_tablet_manager.cc:1950] T 0b411e8fab074aa88f8380e3f5694340 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
W20260202 02:52:20.811215 712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260202 02:52:20.811203 748 tablet_service.cc:1552] Processing DeleteTablet for tablet 38fdbc6df7ec40b9afe4e0dfe0119e3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:20.781304 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 072bb7f0e4ab4935be49461bc8a3c163 (DEFAULT_TABLE table=default.Table_959d3e3da2814b3a986d29864e47246c [id=bdf2f19a439d4be3aa2362cfde38ab7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.785370 1694 tablet.cc:1722] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:20.816169 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 072bb7f0e4ab4935be49461bc8a3c163. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.821058 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1116872cc25a4ab5b1a3a6bbba1f0da7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.821118 778 raft_consensus.cc:1275] T d06b77ff1a6f4f78b73082b00145fcf6 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Refusing update from remote peer 1521a6d262b7451bbef8debe970dd6b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260202 02:52:20.821488 1617 catalog_manager.cc:1612] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260202 02:52:20.821642 750 tablet_service.cc:1505] Processing CreateTablet for tablet 2c31b45dc8004eae993e972e9faa3ff8 (DEFAULT_TABLE table=default.Table_ba9c16aaa48849db9fece59168c16886 [id=f1f6c97fc88340da8bab0347ad6430ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.801301 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "80d443bdb66d4cc68ae283e08c410fe6" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:20.833823 772 raft_consensus.cc:2468] T 80d443bdb66d4cc68ae283e08c410fe6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
I20260202 02:52:20.834682 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d2fa4c3d6ec44047abf5986e502c1590" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:20.839067 755 tablet_service.cc:1505] Processing CreateTablet for tablet ccd6bc8bd0504bf18d108c73685ef9ff (DEFAULT_TABLE table=default.Table_5ecb81038c9f48438e74fe874fc83e74 [id=c28e30416db3407b8c20c68091f392b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.841943 748 tablet_service.cc:1505] Processing CreateTablet for tablet 550895b4010f4088b2cc34acff50e03f (DEFAULT_TABLE table=default.Table_12db441a8c934f9780599755191c58f5 [id=427f58cc70ef47a6afac8de0353975f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.842343 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c31b45dc8004eae993e972e9faa3ff8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.858762 1044 tablet_service.cc:1505] Processing CreateTablet for tablet a0b5d82e5fb94b60b4aa324b978a5378 (DEFAULT_TABLE table=default.Table_cf37f87385ee44cea1ff80cc14050ff2 [id=697b6fe62ee845d895af350881a03556]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.859754 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0b5d82e5fb94b60b4aa324b978a5378. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.875061 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8418a9dd0944456b4d4820dde0cde02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.879732 753 tablet_service.cc:1505] Processing CreateTablet for tablet f94f25f4059a43b8babedb6b45a08a19 (DEFAULT_TABLE table=default.Table_22445ccfd7b64298b92a0f70733f3625 [id=300ea019d2724bee8211d7462710dc26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.885166 522 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_0bf3a06700844105a02b98edd499e75a" } modify_external_catalogs: true
I20260202 02:52:20.885432 522 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_0bf3a06700844105a02b98edd499e75a" } modify_external_catalogs: true
I20260202 02:52:20.885171 524 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_d1815e87de114d408122224276266f69" } modify_external_catalogs: true
I20260202 02:52:20.887601 524 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_d1815e87de114d408122224276266f69" } modify_external_catalogs: true
I20260202 02:52:20.887753 519 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_63bb1b19c34a430e94ad0d7bd9bbe795" } modify_external_catalogs: true
I20260202 02:52:20.887988 519 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_63bb1b19c34a430e94ad0d7bd9bbe795" } modify_external_catalogs: true
I20260202 02:52:20.888196 523 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_80912cc76f934f139da7f866546ae97a" } modify_external_catalogs: true
I20260202 02:52:20.888641 523 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_80912cc76f934f139da7f866546ae97a" } modify_external_catalogs: true
I20260202 02:52:20.890559 1183 tablet_replica.cc:333] T ff6f5710de4b4c48948aacdf5be29e8b P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:20.890781 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 026e6ee0b3ae4d938fdf1550044ed1fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:20.890846 1183 raft_consensus.cc:2243] T ff6f5710de4b4c48948aacdf5be29e8b P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:20.891063 1183 raft_consensus.cc:2272] T ff6f5710de4b4c48948aacdf5be29e8b P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:20.891027 747 tablet_service.cc:1505] Processing CreateTablet for tablet 29c996781cb947f3a1aee5dd0a9c4828 (DEFAULT_TABLE table=default.Table_458f570486154241ad8c7f10a8135ace [id=d2eb691b1287473990f0a66d02f71554]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.891880 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "757492288d7c44fb91b25b7e6e21e01b" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:20.891857 1182 rpcz_store.cc:276] 0202 02:52:18.847537 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:18.847580 (+ 43us) service_pool.cc:225] Handling call
0202 02:52:20.618417 (+1770837us) ts_tablet_manager.cc:1151] Deleting tablet ffd8aa2bc6e344b78dce518411997fb5
0202 02:52:20.640781 (+ 22364us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:20.680624 (+ 39843us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:20.768873 (+ 88249us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:20.770222 (+ 1349us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1770662}
I20260202 02:52:20.892294 1182 tablet_replica.cc:333] T 415d14533fd54f20adb00455a7bcad5f P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:20.892472 1174 tablet_bootstrap.cc:654] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:20.892728 1182 raft_consensus.cc:2243] T 415d14533fd54f20adb00455a7bcad5f P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:20.880259 759 tablet_service.cc:1505] Processing CreateTablet for tablet 842e54adae2a452a98521de24326eaff (DEFAULT_TABLE table=default.Table_524aa17b0b964c9e985e643ee9f12e94 [id=0d7624b55c1a4bf0a449f651ac419d77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.865717 1684 tablet.cc:1722] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:20.903623 1694 tablet.cc:1722] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260202 02:52:20.904464 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 550895b4010f4088b2cc34acff50e03f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:20.905813 859 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 130765us
W20260202 02:52:20.908177 859 leader_election.cc:343] T 80d443bdb66d4cc68ae283e08c410fe6 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:20.908557 859 leader_election.cc:343] T 757492288d7c44fb91b25b7e6e21e01b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:20.908941 1662 consensus_queue.cc:1048] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:20.911921 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 842e54adae2a452a98521de24326eaff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.912007 1180 ts_tablet_manager.cc:1929] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:20.912252 751 tablet_service.cc:1505] Processing CreateTablet for tablet fbcb4a522df648ba9542250450c08f1b (DEFAULT_TABLE table=default.Table_499eb7d77ae34d88ad956bf4a5c55648 [id=163eefc3d38a4b28bce88b948cc5df3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.912469 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 0b411e8fab074aa88f8380e3f5694340 (table default.Table_911bb209b4554ee8b9bc09edaf488d63 [id=cd07f9746ec94d9ebe956a7ca0d19b1c]) successfully deleted
I20260202 02:52:20.914708 921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d2fa4c3d6ec44047abf5986e502c1590" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:20.914713 919 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "761f5a51335a4de5977f5f7fef0d4260" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:20.915094 890 tablet_service.cc:1505] Processing CreateTablet for tablet 33150e1456144b8e9c64cc2a2ac6dcca (DEFAULT_TABLE table=default.Table_38e8a46dd2dd498baa45935fa4670b09 [id=97791994f92a47519b28e44ec02df5ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.915303 919 raft_consensus.cc:2393] T 761f5a51335a4de5977f5f7fef0d4260 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1f05820b6034228bdad884cc348716d in current term 1: Already voted for candidate 1f442ef85029461091de847118c34c0d in this term.
I20260202 02:52:20.915526 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33150e1456144b8e9c64cc2a2ac6dcca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.915874 895 tablet_service.cc:1552] Processing DeleteTablet for tablet b428356fba0c4fb5a6033c5d39740cc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:20.916427 895 tablet_service.cc:1552] Processing DeleteTablet for tablet 583e547d0a1943f09f4efe82070fc777 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:20.916869 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 38301e13e17c41fcae912c4b38c26a45 with delete_type TABLET_DATA_DELETED (replaced by tablet 79badde1001b432b95f58b5161ace15c) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:20.917238 885 tablet_service.cc:1552] Processing DeleteTablet for tablet c0937b6910534ddc8a2217f0cbd539c5 with delete_type TABLET_DATA_DELETED (replaced by tablet 44c8d5b74c8e4c499f7f68e8185efc17) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:20.919027 887 tablet_service.cc:1505] Processing CreateTablet for tablet 79c73f4a92044bafb5b8384229cb3a4d (DEFAULT_TABLE table=default.Table_8cae34d2c6f1443ba07e3f2f8abd1989 [id=dc77705cb5704fd58af1eed3edd45675]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.919498 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79c73f4a92044bafb5b8384229cb3a4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.915095 894 tablet_service.cc:1505] Processing CreateTablet for tablet dada209d982d48f3b74a49905f19c5b6 (DEFAULT_TABLE table=default.Table_1a25d92824e340579a0b522dab94fce6 [id=116247212fca4c0a95ad1bdbbac4787f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.921051 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dada209d982d48f3b74a49905f19c5b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.921116 520 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 63a9029bc39c4fbd8ac4bf4bc37d6b4e (table default.Table_90344989f4234a11a30653bdc34fe899 [id=fa95e04db2d443b5972de98c7c0e6457]). Expected version 3 got 0
I20260202 02:52:20.921337 520 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet b76d1837813c46e8a0925b49939d8884 (table default.Table_3919d090261e4b90b0672f2b863588b2 [id=fe7b88b74c864c2f98365c0956117bf1]). Expected version 1 got 0
I20260202 02:52:20.924983 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_619d2fcf726a46bfb5e939bbe6f55612"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:20.927667 1180 log.cc:1199] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/415d14533fd54f20adb00455a7bcad5f
I20260202 02:52:20.928138 1180 ts_tablet_manager.cc:1950] T 415d14533fd54f20adb00455a7bcad5f P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:20.924746 1685 tablet.cc:1722] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:20.933303 1182 pending_rounds.cc:70] T 415d14533fd54f20adb00455a7bcad5f P 1521a6d262b7451bbef8debe970dd6b5: Trying to abort 1 pending ops.
W20260202 02:52:20.934012 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_619d2fcf726a46bfb5e939bbe6f55612 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:20.945436 528 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ce2d44550fb5403a99372bc5d3d79265" } new_table_name: "default.Table_935434504bf249f8842274aa67486d8c" modify_external_catalogs: true
I20260202 02:52:20.946492 757 tablet_service.cc:1505] Processing CreateTablet for tablet 474df41e4fe24a818fa45a4ce84d5474 (DEFAULT_TABLE table=default.Table_5ed512e7fc01454392b327c54420ec55 [id=14bbe505bc8b4a4bbde264a2c0dcdf50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.950536 1183 ts_tablet_manager.cc:1916] T ff6f5710de4b4c48948aacdf5be29e8b P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:20.953377 1182 pending_rounds.cc:77] T 415d14533fd54f20adb00455a7bcad5f P 1521a6d262b7451bbef8debe970dd6b5: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923025145180160 op_type: NO_OP noop_request { }
I20260202 02:52:20.962114 525 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_84d6693620cc468e82d5cb1ec6cc69ea" } modify_external_catalogs: true
I20260202 02:52:20.962352 525 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_84d6693620cc468e82d5cb1ec6cc69ea" } modify_external_catalogs: true
I20260202 02:52:20.966413 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccd6bc8bd0504bf18d108c73685ef9ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.966887 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f94f25f4059a43b8babedb6b45a08a19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.967545 1182 raft_consensus.cc:2889] T 415d14533fd54f20adb00455a7bcad5f P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:20.967579 1042 tablet_service.cc:1552] Processing DeleteTablet for tablet d2fa4c3d6ec44047abf5986e502c1590 with delete_type TABLET_DATA_DELETED (replaced by tablet a0b5d82e5fb94b60b4aa324b978a5378) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:20.967717 1182 raft_consensus.cc:2272] T 415d14533fd54f20adb00455a7bcad5f P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:20.967883 895 tablet_service.cc:1552] Processing DeleteTablet for tablet ff6f5710de4b4c48948aacdf5be29e8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:20.968568 1182 ts_tablet_manager.cc:1916] T 415d14533fd54f20adb00455a7bcad5f P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:20.969249 1534 raft_consensus.cc:493] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:20.969422 1534 raft_consensus.cc:515] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:20.970105 1534 leader_election.cc:290] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:20.973533 1173 tablet_bootstrap.cc:492] T 583e547d0a1943f09f4efe82070fc777 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:20.973716 1173 ts_tablet_manager.cc:1403] T 583e547d0a1943f09f4efe82070fc777 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.351s user 0.079s sys 0.093s
W20260202 02:52:20.978634 526 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1116872cc25a4ab5b1a3a6bbba1f0da7 consensus state has no leader
I20260202 02:52:20.979176 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_6806575803c4468fa7b7ef990004b446"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:20.979805 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6806575803c4468fa7b7ef990004b446 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:20.983641 1173 raft_consensus.cc:359] T 583e547d0a1943f09f4efe82070fc777 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.983871 1173 raft_consensus.cc:385] T 583e547d0a1943f09f4efe82070fc777 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:20.983976 1173 raft_consensus.cc:740] T 583e547d0a1943f09f4efe82070fc777 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:20.984944 1173 consensus_queue.cc:260] T 583e547d0a1943f09f4efe82070fc777 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:20.986289 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 234) took 2332 ms. Trace:
I20260202 02:52:20.986732 752 tablet_service.cc:1505] Processing CreateTablet for tablet 072bb7f0e4ab4935be49461bc8a3c163 (DEFAULT_TABLE table=default.Table_959d3e3da2814b3a986d29864e47246c [id=bdf2f19a439d4be3aa2362cfde38ab7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.987764 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29c996781cb947f3a1aee5dd0a9c4828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.988071 754 tablet_service.cc:1505] Processing CreateTablet for tablet a0b5d82e5fb94b60b4aa324b978a5378 (DEFAULT_TABLE table=default.Table_cf37f87385ee44cea1ff80cc14050ff2 [id=697b6fe62ee845d895af350881a03556]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.989424 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "78f122342dd144b48de89902faa0f94f" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:20.992344 895 tablet_service.cc:1552] Processing DeleteTablet for tablet 06b80a1c9251420cbc16fc95ae98aca9 with delete_type TABLET_DATA_DELETED (replaced by tablet 2f5aae089fcc492f9ce55f84674a452c) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:20.993566 895 tablet_service.cc:1505] Processing CreateTablet for tablet b7880cb051c24bc998003f0effed3cf8 (DEFAULT_TABLE table=default.Table_a0f1f2d30d064244bec97181c11c3a06 [id=1787be853d4a40cdada2c2735cad1a01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.994067 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7880cb051c24bc998003f0effed3cf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.988605 1180 rpcz_store.cc:276] 0202 02:52:18.596611 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:18.596746 (+ 135us) service_pool.cc:225] Handling call
0202 02:52:20.627450 (+2030704us) ts_tablet_manager.cc:1151] Deleting tablet 415d14533fd54f20adb00455a7bcad5f
0202 02:52:20.627460 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:20.807362 (+179902us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:20.894111 (+ 86749us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:20.928962 (+ 34851us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3,"tablet-delete.queue_time_us":2030481}
I20260202 02:52:20.997355 891 tablet_service.cc:1505] Processing CreateTablet for tablet 6b8f924aecf44e959e47d2f32e6b0bf5 (DEFAULT_TABLE table=default.Table_4532505bd70548f7b20362dc9629dbee [id=32826e501f3440589641affa98a4c82d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:20.998031 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b8f924aecf44e959e47d2f32e6b0bf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:20.998288 527 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 63a9029bc39c4fbd8ac4bf4bc37d6b4e (table default.Table_90344989f4234a11a30653bdc34fe899 [id=fa95e04db2d443b5972de98c7c0e6457]). Expected version 3 got 0
I20260202 02:52:20.999503 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_2ee0b536f96e44e4a3a4a9439c7bccc9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:21.000159 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2ee0b536f96e44e4a3a4a9439c7bccc9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:21.007390 1684 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:41736 (request call id 265) took 1054 ms. Trace:
I20260202 02:52:21.014102 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 072bb7f0e4ab4935be49461bc8a3c163 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:21.015328 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 3711910e050c4515bed4294d1271d5a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:21.016147 889 tablet_service.cc:1505] Processing CreateTablet for tablet 8bb14bf551f441dcbaaff7057b51b2be (DEFAULT_TABLE table=default.Table_7a314585590647ca8fd0f51e125b0622 [id=0e44d4587c694edc90c651e2f9db649b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.016871 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbcb4a522df648ba9542250450c08f1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.017023 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 38fdbc6df7ec40b9afe4e0dfe0119e3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:21.017552 888 tablet_service.cc:1505] Processing CreateTablet for tablet d8418a9dd0944456b4d4820dde0cde02 (DEFAULT_TABLE table=default.Table_2d42dcf81d2d4b2c90fe09860b9b2318 [id=e620930eae8e4707b93e7ab16369e13d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.018194 898 tablet_service.cc:1552] Processing DeleteTablet for tablet c63a670d5c5e489897bc6dbb142b844d with delete_type TABLET_DATA_DELETED (replaced by tablet 939fbcaf75c7439d9208bbbf83a72717) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:21.018637 898 tablet_service.cc:1552] Processing DeleteTablet for tablet d0671586e45e45ffb7fd9e1be6e2bbf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:21.019451 898 tablet_service.cc:1505] Processing CreateTablet for tablet 1116872cc25a4ab5b1a3a6bbba1f0da7 (DEFAULT_TABLE table=default.Table_242f4cdb6f514121a1a113910a2aaec8 [id=52a6d31f993d4c74a520d504390cf12b]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:21.019995 715 leader_election.cc:343] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:21.020215 749 tablet_service.cc:1505] Processing CreateTablet for tablet 6df3d11cf578404ca41807f0777b6c53 (DEFAULT_TABLE table=default.Table_d1815e87de114d408122224276266f69 [id=4a43e638e8854fcda2702463684c08d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.020541 1534 raft_consensus.cc:493] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:21.020594 901 tablet_service.cc:1505] Processing CreateTablet for tablet f94f25f4059a43b8babedb6b45a08a19 (DEFAULT_TABLE table=default.Table_22445ccfd7b64298b92a0f70733f3625 [id=300ea019d2724bee8211d7462710dc26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.022248 902 tablet_service.cc:1505] Processing CreateTablet for tablet 2c31b45dc8004eae993e972e9faa3ff8 (DEFAULT_TABLE table=default.Table_ba9c16aaa48849db9fece59168c16886 [id=f1f6c97fc88340da8bab0347ad6430ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.033072 1534 raft_consensus.cc:515] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.035638 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 474df41e4fe24a818fa45a4ce84d5474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.037617 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 072bb7f0e4ab4935be49461bc8a3c163. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.039386 1173 ts_tablet_manager.cc:1434] T 583e547d0a1943f09f4efe82070fc777 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.065s user 0.000s sys 0.004s
I20260202 02:52:21.052822 521 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet b76d1837813c46e8a0925b49939d8884 (table default.Table_3919d090261e4b90b0672f2b863588b2 [id=fe7b88b74c864c2f98365c0956117bf1]). Expected version 1 got 0
I20260202 02:52:21.057375 1173 tablet_bootstrap.cc:492] T 723921dfe54c4e66af5cab76a25fee0a P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:21.059116 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9884c90cf1d245b2a7eaaf00d71b1c3e" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:21.060043 715 leader_election.cc:343] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:21.063638 1702 tablet.cc:1722] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:21.064145 890 tablet_service.cc:1552] Processing DeleteTablet for tablet 026e6ee0b3ae4d938fdf1550044ed1fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:21.065889 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0b5d82e5fb94b60b4aa324b978a5378. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.066159 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6df3d11cf578404ca41807f0777b6c53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.069017 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 248) took 2291 ms. Trace:
I20260202 02:52:21.069156 1180 rpcz_store.cc:276] 0202 02:52:18.777796 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:18.778033 (+ 237us) service_pool.cc:225] Handling call
0202 02:52:20.996255 (+2218222us) ts_tablet_manager.cc:1151] Deleting tablet 0186572e7bb9499a932a636ab280f544
0202 02:52:21.068766 (+ 72511us) spinlock_profiling.cc:243] Waited 72.2 ms on lock 0x61800000c518. stack: 00007f1e37b5aa86 00007f1e37b5a840 00007f1e36b353ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f1e37b433bd 00007f1e37b4442e 00007f1e4b7b392d 00007f1e4b731f23 00007f1e47954b06 00007f1e47955e99 00007f1e47989744 00007f1e4796a240 00007f1e47969fcd 0000000000a8970c
0202 02:52:21.068778 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:21.068926 (+ 148us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":166005632,"tablet-delete.queue_time_us":1835692}
I20260202 02:52:21.078090 1708 tablet.cc:1722] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:21.081806 1174 tablet_bootstrap.cc:492] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:21.084512 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bb14bf551f441dcbaaff7057b51b2be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.085565 1175 tablet_bootstrap.cc:492] T ef073880898049e2be93d2cb6dc0c0bb P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:21.086421 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f94f25f4059a43b8babedb6b45a08a19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.091358 522 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet d30557fbe9084cd3860820f69dda8752
I20260202 02:52:21.094543 890 tablet_service.cc:1505] Processing CreateTablet for tablet 550895b4010f4088b2cc34acff50e03f (DEFAULT_TABLE table=default.Table_12db441a8c934f9780599755191c58f5 [id=427f58cc70ef47a6afac8de0353975f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.100467 1175 ts_tablet_manager.cc:1403] T ef073880898049e2be93d2cb6dc0c0bb P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.427s user 0.000s sys 0.013s
I20260202 02:52:21.101859 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8418a9dd0944456b4d4820dde0cde02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.108572 1685 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:41736 (request call id 272) took 1102 ms. Trace:
I20260202 02:52:21.117489 522 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_115696f06e984b47bd26eab721b76bf7" } new_table_name: "default.Table_35fb6ec04ef245a88e633fedb3d9c186" modify_external_catalogs: true
I20260202 02:52:21.121424 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1116872cc25a4ab5b1a3a6bbba1f0da7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.122785 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet d30557fbe9084cd3860820f69dda8752 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:21.130141 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 249) took 2283 ms. Trace:
I20260202 02:52:21.130311 1180 rpcz_store.cc:276] 0202 02:52:18.786432 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:18.786659 (+ 227us) service_pool.cc:225] Handling call
0202 02:52:21.069480 (+2282821us) ts_tablet_manager.cc:1151] Deleting tablet 13cd1383348b458894fdecc22bf24b74
0202 02:52:21.069491 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:21.069591 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1904054}
I20260202 02:52:21.130859 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 250) took 1979 ms. Trace:
I20260202 02:52:21.130962 1180 rpcz_store.cc:276] 0202 02:52:19.151022 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.151306 (+ 284us) service_pool.cc:225] Handling call
0202 02:52:21.130631 (+1979325us) ts_tablet_manager.cc:1151] Deleting tablet 8774c6bc0fe9414ea2f17ed859d9392a
0202 02:52:21.130646 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:21.130788 (+ 142us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1914705}
I20260202 02:52:21.131414 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 251) took 1784 ms. Trace:
I20260202 02:52:21.131521 1180 rpcz_store.cc:276] 0202 02:52:19.347285 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.347354 (+ 69us) service_pool.cc:225] Handling call
0202 02:52:21.131223 (+1783869us) ts_tablet_manager.cc:1151] Deleting tablet 2f0e8c5e2fd44370ae150a206f4e6cbc
0202 02:52:21.131233 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:21.131347 (+ 114us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1756104}
I20260202 02:52:21.131817 1180 tablet_replica.cc:333] T 75f7824681344d8d88df9402694fff1b P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:21.132036 1180 raft_consensus.cc:2243] T 75f7824681344d8d88df9402694fff1b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:21.132254 1180 raft_consensus.cc:2272] T 75f7824681344d8d88df9402694fff1b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:21.133490 1180 ts_tablet_manager.cc:1916] T 75f7824681344d8d88df9402694fff1b P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:21.138775 1174 ts_tablet_manager.cc:1403] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.448s user 0.017s sys 0.004s
I20260202 02:52:21.140281 1174 raft_consensus.cc:359] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.140709 1174 raft_consensus.cc:385] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:21.126171 1685 rpcz_store.cc:276] 0202 02:52:20.006433 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:20.006491 (+ 58us) service_pool.cc:225] Handling call
0202 02:52:21.108523 (+1102032us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:20.239849 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0202 02:52:20.241147 (+ 1298us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0202 02:52:20.241157 (+ 10us) alter_schema_op.cc:66] Acquired schema lock
0202 02:52:20.241165 (+ 8us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0202 02:52:20.241234 (+ 69us) alter_schema_op.cc:128] START. Timestamp: 1770000740241171
0202 02:52:20.241238 (+ 4us) op_driver.cc:348] REPLICATION: starting
0202 02:52:20.241544 (+ 306us) log.cc:844] Serialized 173 byte log entry
0202 02:52:20.633916 (+392372us) op_driver.cc:464] REPLICATION: finished
0202 02:52:20.924680 (+290764us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0202 02:52:20.936397 (+ 11717us) log.cc:844] Serialized 5 byte log entry
0202 02:52:21.108063 (+171666us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:21.108439 (+ 376us) log.cc:844] Serialized 14 byte log entry
0202 02:52:21.108480 (+ 41us) alter_schema_op.cc:73] Released schema lock
0202 02:52:21.108485 (+ 5us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":290669,"prepare.queue_time_us":210667,"prepare.run_cpu_time_us":6352,"prepare.run_wall_time_us":71754,"raft.queue_time_us":71986,"raft.run_cpu_time_us":579,"raft.run_wall_time_us":578,"replication_time_us":392640,"thread_start_us":446257,"threads_started":4,"wal-append.queue_time_us":41157}]]}
I20260202 02:52:21.140852 1174 raft_consensus.cc:740] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:21.141099 1174 consensus_queue.cc:260] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.141702 1174 ts_tablet_manager.cc:1434] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260202 02:52:21.142194 1174 tablet_bootstrap.cc:492] T 0186572e7bb9499a932a636ab280f544 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:21.143963 1174 tablet_bootstrap.cc:654] T 0186572e7bb9499a932a636ab280f544 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:21.141582 1703 tablet.cc:1722] T 5986676f5484498da927b4c6da3f47ff P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:21.145627 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c31b45dc8004eae993e972e9faa3ff8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.149467 1180 ts_tablet_manager.cc:1929] T 75f7824681344d8d88df9402694fff1b P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:21.148388 1175 raft_consensus.cc:359] T ef073880898049e2be93d2cb6dc0c0bb P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.149857 1180 log.cc:1199] T 75f7824681344d8d88df9402694fff1b P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/75f7824681344d8d88df9402694fff1b
I20260202 02:52:21.008092 1684 rpcz_store.cc:276] 0202 02:52:19.952817 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.952862 (+ 45us) service_pool.cc:225] Handling call
0202 02:52:21.007347 (+1054485us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:20.158412 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0202 02:52:20.173523 (+ 15111us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0202 02:52:20.173534 (+ 11us) alter_schema_op.cc:66] Acquired schema lock
0202 02:52:20.173542 (+ 8us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0202 02:52:20.173590 (+ 48us) alter_schema_op.cc:128] START. Timestamp: 1770000740173542
0202 02:52:20.173593 (+ 3us) op_driver.cc:348] REPLICATION: starting
0202 02:52:20.173803 (+ 210us) log.cc:844] Serialized 173 byte log entry
0202 02:52:20.195231 (+ 21428us) spinlock_profiling.cc:243] Waited 4.83 ms on lock 0x6160001702b8. stack: 00007fb35529ca86 00007fb35529c840 00007fb3542773ad 0000000000984f94 0000000000984f5d 000000000098492a 00007fb361094e3f 00007fb361062b4b 00007fb36106dae1 00007fb36106d7cd 0000000000a8970c 00007fb3552e2daf 00007fb3552e60cb 00007fb3552e5f48 0000000000a8970c 00007fb3552c1f98
0202 02:52:20.312637 (+117406us) spinlock_profiling.cc:243] Waited 50.9 ms on lock 0x6160001702b8. stack: 00007fb35529ca86 00007fb35529c840 00007fb3542773ad 0000000000984f94 0000000000984f5d 000000000098492a 00007fb3610a3309 00007fb3610a86f4 00007fb3610a84b8 0000000000a8970c 00007fb3552e2daf 00007fb3552e60cb 00007fb3552e5f48 0000000000a8970c 00007fb3552c1f98 00007fb3588ce6da
0202 02:52:20.627529 (+314892us) op_driver.cc:464] REPLICATION: finished
0202 02:52:20.865593 (+238064us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0202 02:52:20.894861 (+ 29268us) log.cc:844] Serialized 5 byte log entry
0202 02:52:20.898433 (+ 3572us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:21.006881 (+108448us) log.cc:844] Serialized 14 byte log entry
0202 02:52:21.007300 (+ 419us) alter_schema_op.cc:73] Released schema lock
0202 02:52:21.007306 (+ 6us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":237817,"mutex_wait_us":1824,"prepare.queue_time_us":200920,"prepare.run_cpu_time_us":642,"prepare.run_wall_time_us":15492,"raft.queue_time_us":66833,"raft.run_cpu_time_us":10004,"raft.run_wall_time_us":189442,"replication_time_us":453899,"spinlock_wait_cycles":128172032,"thread_start_us":654299,"threads_started":3}]]}
I20260202 02:52:21.150648 1180 ts_tablet_manager.cc:1950] T 75f7824681344d8d88df9402694fff1b P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:21.151297 901 tablet_service.cc:1505] Processing CreateTablet for tablet ccd6bc8bd0504bf18d108c73685ef9ff (DEFAULT_TABLE table=default.Table_5ecb81038c9f48438e74fe874fc83e74 [id=c28e30416db3407b8c20c68091f392b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.151571 899 tablet_service.cc:1505] Processing CreateTablet for tablet 474df41e4fe24a818fa45a4ce84d5474 (DEFAULT_TABLE table=default.Table_5ed512e7fc01454392b327c54420ec55 [id=14bbe505bc8b4a4bbde264a2c0dcdf50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.151630 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 266) took 1547 ms. Trace:
I20260202 02:52:21.152104 894 tablet_service.cc:1505] Processing CreateTablet for tablet fbcb4a522df648ba9542250450c08f1b (DEFAULT_TABLE table=default.Table_499eb7d77ae34d88ad956bf4a5c55648 [id=163eefc3d38a4b28bce88b948cc5df3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.151894 1180 rpcz_store.cc:276] 0202 02:52:19.604608 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.604665 (+ 57us) service_pool.cc:225] Handling call
0202 02:52:21.131721 (+1527056us) ts_tablet_manager.cc:1151] Deleting tablet 75f7824681344d8d88df9402694fff1b
0202 02:52:21.131730 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:21.145281 (+ 13551us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:21.149187 (+ 3906us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:21.151561 (+ 2374us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1524992}
I20260202 02:52:21.153285 884 tablet_service.cc:1505] Processing CreateTablet for tablet 29c996781cb947f3a1aee5dd0a9c4828 (DEFAULT_TABLE table=default.Table_458f570486154241ad8c7f10a8135ace [id=d2eb691b1287473990f0a66d02f71554]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:21.156656 714 leader_election.cc:343] T c064ea0dade840b4840a5715c2d83f69 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: c064ea0dade840b4840a5715c2d83f69
I20260202 02:52:21.157840 885 tablet_service.cc:1505] Processing CreateTablet for tablet 842e54adae2a452a98521de24326eaff (DEFAULT_TABLE table=default.Table_524aa17b0b964c9e985e643ee9f12e94 [id=0d7624b55c1a4bf0a449f651ac419d77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.158663 1175 raft_consensus.cc:385] T ef073880898049e2be93d2cb6dc0c0bb P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:21.158777 1175 raft_consensus.cc:740] T ef073880898049e2be93d2cb6dc0c0bb P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:21.159035 1175 consensus_queue.cc:260] T ef073880898049e2be93d2cb6dc0c0bb P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.157827 1534 leader_election.cc:290] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:21.157173 714 leader_election.cc:304] T c064ea0dade840b4840a5715c2d83f69 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:21.162009 886 tablet_service.cc:1505] Processing CreateTablet for tablet 072bb7f0e4ab4935be49461bc8a3c163 (DEFAULT_TABLE table=default.Table_959d3e3da2814b3a986d29864e47246c [id=bdf2f19a439d4be3aa2362cfde38ab7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.162948 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 550895b4010f4088b2cc34acff50e03f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.167003 892 tablet_service.cc:1505] Processing CreateTablet for tablet 6df3d11cf578404ca41807f0777b6c53 (DEFAULT_TABLE table=default.Table_d1815e87de114d408122224276266f69 [id=4a43e638e8854fcda2702463684c08d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.167402 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet ffd8aa2bc6e344b78dce518411997fb5 (table default.Table_ab8d74f30cd14c61b85950a82d0f3787 [id=3ccb047c30974c0f9cbe778971e61644]) successfully deleted
I20260202 02:52:21.171000 520 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet bd7b0a0fcb7645fdb9edfeccb3d18268 (table default.Table_18fec323d6b04e10a687de6361b643b2 [id=1ffcc11561524fe8a7b09a7ddc92d5b2]). Expected version 2 got 0
I20260202 02:52:21.171393 520 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 5986676f5484498da927b4c6da3f47ff (table default.Table_8cb3cbab1a654f88a07d6d480fc72e52 [id=d8485f23f63b493781bea557dbe8a0c1]). Expected version 1 got 0
I20260202 02:52:21.175997 1182 ts_tablet_manager.cc:1929] T 415d14533fd54f20adb00455a7bcad5f P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:21.176855 893 tablet_service.cc:1505] Processing CreateTablet for tablet a0b5d82e5fb94b60b4aa324b978a5378 (DEFAULT_TABLE table=default.Table_cf37f87385ee44cea1ff80cc14050ff2 [id=697b6fe62ee845d895af350881a03556]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.176867 1655 consensus_queue.cc:1048] T d06b77ff1a6f4f78b73082b00145fcf6 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260202 02:52:21.176885 1175 ts_tablet_manager.cc:1434] T ef073880898049e2be93d2cb6dc0c0bb P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.068s user 0.001s sys 0.001s
I20260202 02:52:21.176877 1662 consensus_queue.cc:1048] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
W20260202 02:52:21.181578 999 leader_election.cc:343] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: bf74f96bea2e4ab7adec42c3c5207e79
W20260202 02:52:21.182055 999 leader_election.cc:343] T bf74f96bea2e4ab7adec42c3c5207e79 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: bf74f96bea2e4ab7adec42c3c5207e79
I20260202 02:52:21.182503 1063 raft_consensus.cc:3060] T d2fa4c3d6ec44047abf5986e502c1590 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:21.182564 900 tablet_service.cc:1552] Processing DeleteTablet for tablet d30557fbe9084cd3860820f69dda8752 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:21.182771 999 leader_election.cc:304] T d2fa4c3d6ec44047abf5986e502c1590 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:21.183230 999 leader_election.cc:304] T 761f5a51335a4de5977f5f7fef0d4260 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:21.183337 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29c996781cb947f3a1aee5dd0a9c4828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.184772 857 leader_election.cc:304] T 80d443bdb66d4cc68ae283e08c410fe6 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:21.185063 1182 log.cc:1199] T 415d14533fd54f20adb00455a7bcad5f P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/415d14533fd54f20adb00455a7bcad5f
I20260202 02:52:21.186870 1059 raft_consensus.cc:3060] T 583e547d0a1943f09f4efe82070fc777 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:21.190477 1175 tablet_bootstrap.cc:492] T 75b97d7e848241cf8d894b0ae92a8f8f P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:21.191617 1702 tablet.cc:1717] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1521a6d262b7451bbef8debe970dd6b5: Skipping requested alter to schema version 2, tablet already version 2
I20260202 02:52:21.197856 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6df3d11cf578404ca41807f0777b6c53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.203246 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 474df41e4fe24a818fa45a4ce84d5474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.203557 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccd6bc8bd0504bf18d108c73685ef9ff. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:21.204150 528 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fe53a8bdbd13417f9ce195c54a52a93c consensus state has no leader
I20260202 02:52:21.206516 1662 raft_consensus.cc:2804] T 80d443bdb66d4cc68ae283e08c410fe6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:21.206667 1662 raft_consensus.cc:493] T 80d443bdb66d4cc68ae283e08c410fe6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:21.206791 1662 raft_consensus.cc:3060] T 80d443bdb66d4cc68ae283e08c410fe6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:21.208681 1173 tablet_bootstrap.cc:654] T 723921dfe54c4e66af5cab76a25fee0a P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:21.209860 1662 raft_consensus.cc:515] T 80d443bdb66d4cc68ae283e08c410fe6 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.210211 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_48226e7c77da41cdba56e04221eb9689"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:21.210606 1662 leader_election.cc:290] T 80d443bdb66d4cc68ae283e08c410fe6 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:21.210889 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_48226e7c77da41cdba56e04221eb9689 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:21.210942 857 leader_election.cc:343] T 757492288d7c44fb91b25b7e6e21e01b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:21.211055 857 leader_election.cc:304] T 757492288d7c44fb91b25b7e6e21e01b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:21.211402 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbcb4a522df648ba9542250450c08f1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.212831 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 072bb7f0e4ab4935be49461bc8a3c163. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.216470 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 842e54adae2a452a98521de24326eaff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.216665 1684 tablet.cc:1722] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:21.214969 1708 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:42950 (request call id 291) took 1167 ms. Trace:
I20260202 02:52:21.218884 1180 tablet_replica.cc:333] T fe7b6004e5094c0dbe5847144a2489b9 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
W20260202 02:52:21.223136 859 consensus_peers.cc:597] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5 -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 61a9203035f2495f8f99924303ebf120. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:21.224717 1534 raft_consensus.cc:2749] T c064ea0dade840b4840a5715c2d83f69 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:21.225193 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "80d443bdb66d4cc68ae283e08c410fe6" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:21.225517 1694 tablet.cc:1722] T b76d1837813c46e8a0925b49939d8884 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:21.233708 714 leader_election.cc:304] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:21.236589 1720 raft_consensus.cc:2749] T 761f5a51335a4de5977f5f7fef0d4260 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:21.236066 715 consensus_peers.cc:597] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0937b6910534ddc8a2217f0cbd539c5. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:21.239316 1534 raft_consensus.cc:2804] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
W20260202 02:52:21.240386 715 consensus_peers.cc:597] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:21.245484 1182 ts_tablet_manager.cc:1950] T 415d14533fd54f20adb00455a7bcad5f P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:21.245424 1662 raft_consensus.cc:2749] T 757492288d7c44fb91b25b7e6e21e01b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:21.245988 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 225) took 2398 ms. Trace:
W20260202 02:52:21.246186 714 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 53911.8us
I20260202 02:52:21.246133 1182 rpcz_store.cc:276] 0202 02:52:18.847166 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:18.849547 (+ 2381us) service_pool.cc:225] Handling call
0202 02:52:20.892230 (+2042683us) ts_tablet_manager.cc:1151] Deleting tablet 415d14533fd54f20adb00455a7bcad5f
0202 02:52:20.892243 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:21.098247 (+206004us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:21.171206 (+ 72959us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:21.245942 (+ 74736us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":33370,"tablet-delete.queue_time_us":2042431}
I20260202 02:52:21.246570 1182 tablet_replica.cc:333] T eaa47f3388144e8b852804c167f232ce P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:21.246799 1182 raft_consensus.cc:2243] T eaa47f3388144e8b852804c167f232ce P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:21.246979 1182 raft_consensus.cc:2272] T eaa47f3388144e8b852804c167f232ce P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:21.247594 1182 ts_tablet_manager.cc:1916] T eaa47f3388144e8b852804c167f232ce P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:21.248106 859 leader_election.cc:343] T 80d443bdb66d4cc68ae283e08c410fe6 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:21.249063 1534 raft_consensus.cc:697] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:21.249418 1534 consensus_queue.cc:237] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.235937 1714 tablet.cc:1722] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260202 02:52:21.256049 1178 raft_consensus.cc:2749] T d2fa4c3d6ec44047abf5986e502c1590 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:21.258828 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0b5d82e5fb94b60b4aa324b978a5378. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.264143 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 415d14533fd54f20adb00455a7bcad5f (table default.Table_22489ecf56264d93993407ba7d0e6b55 [id=7c19fad4b9554c0682a01bbd3f73d198]) successfully deleted
I20260202 02:52:21.264789 968 heartbeater.cc:344] Connected to a master server at 127.0.104.188:39077
I20260202 02:52:21.271162 1183 ts_tablet_manager.cc:1929] T ff6f5710de4b4c48948aacdf5be29e8b P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:21.271332 1183 log.cc:1199] T ff6f5710de4b4c48948aacdf5be29e8b P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/ff6f5710de4b4c48948aacdf5be29e8b
I20260202 02:52:21.271725 1183 ts_tablet_manager.cc:1950] T ff6f5710de4b4c48948aacdf5be29e8b P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:21.273083 1173 tablet_bootstrap.cc:492] T 723921dfe54c4e66af5cab76a25fee0a P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:21.288647 1173 ts_tablet_manager.cc:1403] T 723921dfe54c4e66af5cab76a25fee0a P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.232s user 0.005s sys 0.003s
I20260202 02:52:21.291831 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet ff6f5710de4b4c48948aacdf5be29e8b (table default.Table_ad048ed673a5401b8ffa739c0f03b545 [id=f2276e1c2ca04a70804053da61dd3447]) successfully deleted
W20260202 02:52:21.294809 999 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 111278us
W20260202 02:52:21.295655 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 06b80a1c9251420cbc16fc95ae98aca9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 06b80a1c9251420cbc16fc95ae98aca9 already in progress: creating tablet
W20260202 02:52:21.296057 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 072bb7f0e4ab4935be49461bc8a3c163 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 072bb7f0e4ab4935be49461bc8a3c163 already in progress: creating tablet
W20260202 02:52:21.296494 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 3711910e050c4515bed4294d1271d5a9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3711910e050c4515bed4294d1271d5a9 already in progress: creating tablet
W20260202 02:52:21.296763 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 38fdbc6df7ec40b9afe4e0dfe0119e3e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 38fdbc6df7ec40b9afe4e0dfe0119e3e already in progress: creating tablet
W20260202 02:52:21.297057 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet d0671586e45e45ffb7fd9e1be6e2bbf8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d0671586e45e45ffb7fd9e1be6e2bbf8 already in progress: creating tablet
I20260202 02:52:21.298012 1183 tablet_replica.cc:333] T c63a670d5c5e489897bc6dbb142b844d P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:21.298182 1173 raft_consensus.cc:359] T 723921dfe54c4e66af5cab76a25fee0a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.299171 1173 raft_consensus.cc:385] T 723921dfe54c4e66af5cab76a25fee0a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:21.299284 1173 raft_consensus.cc:740] T 723921dfe54c4e66af5cab76a25fee0a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:21.299577 1183 raft_consensus.cc:2243] T c63a670d5c5e489897bc6dbb142b844d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:21.299711 1173 consensus_queue.cc:260] T 723921dfe54c4e66af5cab76a25fee0a P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.300803 1183 raft_consensus.cc:2272] T c63a670d5c5e489897bc6dbb142b844d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:21.301476 1183 ts_tablet_manager.cc:1916] T c63a670d5c5e489897bc6dbb142b844d P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:21.306980 1173 ts_tablet_manager.cc:1434] T 723921dfe54c4e66af5cab76a25fee0a P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.018s user 0.001s sys 0.001s
I20260202 02:52:21.307372 1173 tablet_bootstrap.cc:492] T c12ff20ca6b44ddba0c936a28cfb2eb9 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:21.309098 1173 tablet_bootstrap.cc:654] T c12ff20ca6b44ddba0c936a28cfb2eb9 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:21.310429 1694 tablet.cc:1722] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260202 02:52:21.316015 1703 tablet.cc:1722] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260202 02:52:21.323179 1175 tablet_bootstrap.cc:654] T 75b97d7e848241cf8d894b0ae92a8f8f P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:21.335350 1173 tablet_bootstrap.cc:492] T c12ff20ca6b44ddba0c936a28cfb2eb9 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:21.335525 1173 ts_tablet_manager.cc:1403] T c12ff20ca6b44ddba0c936a28cfb2eb9 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.028s user 0.003s sys 0.000s
I20260202 02:52:21.336467 1173 raft_consensus.cc:359] T c12ff20ca6b44ddba0c936a28cfb2eb9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.336663 1173 raft_consensus.cc:385] T c12ff20ca6b44ddba0c936a28cfb2eb9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:21.336768 1173 raft_consensus.cc:740] T c12ff20ca6b44ddba0c936a28cfb2eb9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:21.337018 1173 consensus_queue.cc:260] T c12ff20ca6b44ddba0c936a28cfb2eb9 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.337591 1173 ts_tablet_manager.cc:1434] T c12ff20ca6b44ddba0c936a28cfb2eb9 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260202 02:52:21.337910 1173 tablet_bootstrap.cc:492] T fe7b6004e5094c0dbe5847144a2489b9 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:21.338348 714 leader_election.cc:304] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:21.317438 1708 rpcz_store.cc:276] 0202 02:52:20.005990 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:20.166836 (+160846us) service_pool.cc:225] Handling call
0202 02:52:21.173324 (+1006488us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:20.167412 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0202 02:52:20.167506 (+ 94us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0202 02:52:20.167515 (+ 9us) alter_schema_op.cc:66] Acquired schema lock
0202 02:52:20.167525 (+ 10us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0202 02:52:20.167562 (+ 37us) alter_schema_op.cc:128] START. Timestamp: 1770000740167417
0202 02:52:20.167565 (+ 3us) op_driver.cc:348] REPLICATION: starting
0202 02:52:20.167740 (+ 175us) log.cc:844] Serialized 173 byte log entry
0202 02:52:20.972067 (+804327us) op_driver.cc:464] REPLICATION: finished
0202 02:52:21.062614 (+ 90547us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0202 02:52:21.079722 (+ 17108us) log.cc:844] Serialized 5 byte log entry
0202 02:52:21.157484 (+ 77762us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:21.173223 (+ 15739us) log.cc:844] Serialized 14 byte log entry
0202 02:52:21.173269 (+ 46us) alter_schema_op.cc:73] Released schema lock
0202 02:52:21.173274 (+ 5us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":90213,"prepare.queue_time_us":59,"prepare.run_cpu_time_us":443,"prepare.run_wall_time_us":445,"raft.queue_time_us":1153,"raft.run_cpu_time_us":340,"raft.run_wall_time_us":340,"replication_time_us":804463,"thread_start_us":52635,"threads_started":1}]]}
I20260202 02:52:21.340127 1173 tablet_bootstrap.cc:654] T fe7b6004e5094c0dbe5847144a2489b9 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:21.347868 1714 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:42950 (request call id 255) took 1970 ms. Trace:
W20260202 02:52:21.348284 714 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 97761.8us
I20260202 02:52:21.350438 1600 raft_consensus.cc:2804] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
W20260202 02:52:21.350525 1713 log.cc:927] Time spent T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1f442ef85029461091de847118c34c0d: Append to log took a long time: real 0.062s user 0.000s sys 0.000s
I20260202 02:52:21.350872 714 leader_election.cc:304] T 80d443bdb66d4cc68ae283e08c410fe6 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
W20260202 02:52:21.351660 1698 log.cc:927] Time spent T 583e547d0a1943f09f4efe82070fc777 P 1f442ef85029461091de847118c34c0d: Append to log took a long time: real 0.063s user 0.000s sys 0.000s
I20260202 02:52:21.352262 1709 consensus_queue.cc:1048] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260202 02:52:21.352448 1183 ts_tablet_manager.cc:1929] T c63a670d5c5e489897bc6dbb142b844d P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:21.352586 1183 log.cc:1199] T c63a670d5c5e489897bc6dbb142b844d P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/c63a670d5c5e489897bc6dbb142b844d
I20260202 02:52:21.353081 1183 ts_tablet_manager.cc:1950] T c63a670d5c5e489897bc6dbb142b844d P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:21.353098 1709 consensus_queue.cc:1048] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:21.354105 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet c63a670d5c5e489897bc6dbb142b844d (table default.Table_12d68a317c8e4d809dbbe77c095a1f6e [id=7219522ffc3849faab94d754a42a8511]) successfully deleted
I20260202 02:52:21.354317 1183 tablet_replica.cc:333] T d2fa4c3d6ec44047abf5986e502c1590 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
W20260202 02:52:21.354396 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 026e6ee0b3ae4d938fdf1550044ed1fa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 026e6ee0b3ae4d938fdf1550044ed1fa already in progress: creating tablet
I20260202 02:52:21.354589 1183 raft_consensus.cc:2243] T d2fa4c3d6ec44047abf5986e502c1590 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:21.354775 1183 raft_consensus.cc:2272] T d2fa4c3d6ec44047abf5986e502c1590 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:21.355947 1723 raft_consensus.cc:2804] T 80d443bdb66d4cc68ae283e08c410fe6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:21.356122 1723 raft_consensus.cc:493] T 80d443bdb66d4cc68ae283e08c410fe6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:21.357189 1723 raft_consensus.cc:3060] T 80d443bdb66d4cc68ae283e08c410fe6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:21.356264 1180 raft_consensus.cc:2243] T fe7b6004e5094c0dbe5847144a2489b9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:21.358809 1180 raft_consensus.cc:2272] T fe7b6004e5094c0dbe5847144a2489b9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:21.359624 1180 ts_tablet_manager.cc:1916] T fe7b6004e5094c0dbe5847144a2489b9 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:21.364115 1173 tablet_bootstrap.cc:492] T fe7b6004e5094c0dbe5847144a2489b9 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:21.364312 1173 ts_tablet_manager.cc:1403] T fe7b6004e5094c0dbe5847144a2489b9 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.026s user 0.001s sys 0.003s
I20260202 02:52:21.365135 1173 raft_consensus.cc:359] T fe7b6004e5094c0dbe5847144a2489b9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.365339 1173 raft_consensus.cc:385] T fe7b6004e5094c0dbe5847144a2489b9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:21.365408 1173 raft_consensus.cc:740] T fe7b6004e5094c0dbe5847144a2489b9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:21.365661 1173 consensus_queue.cc:260] T fe7b6004e5094c0dbe5847144a2489b9 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.366156 1173 ts_tablet_manager.cc:1434] T fe7b6004e5094c0dbe5847144a2489b9 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260202 02:52:21.366485 1173 tablet_bootstrap.cc:492] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:21.371816 1182 ts_tablet_manager.cc:1929] T eaa47f3388144e8b852804c167f232ce P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:21.368286 1173 tablet_bootstrap.cc:654] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:21.375741 1183 ts_tablet_manager.cc:1916] T d2fa4c3d6ec44047abf5986e502c1590 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:21.348451 1714 rpcz_store.cc:276] 0202 02:52:19.377430 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.377493 (+ 63us) service_pool.cc:225] Handling call
0202 02:52:21.347824 (+1970331us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:19.462465 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0202 02:52:19.499453 (+ 36988us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0202 02:52:19.499467 (+ 14us) alter_schema_op.cc:66] Acquired schema lock
0202 02:52:19.499475 (+ 8us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0202 02:52:19.499929 (+ 454us) alter_schema_op.cc:128] START. Timestamp: 1770000739499710
0202 02:52:19.499933 (+ 4us) op_driver.cc:348] REPLICATION: starting
0202 02:52:19.504495 (+ 4562us) log.cc:844] Serialized 173 byte log entry
0202 02:52:21.081653 (+1577158us) op_driver.cc:464] REPLICATION: finished
0202 02:52:21.226136 (+144483us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0202 02:52:21.310827 (+ 84691us) log.cc:844] Serialized 5 byte log entry
0202 02:52:21.347347 (+ 36520us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:21.347703 (+ 356us) log.cc:844] Serialized 14 byte log entry
0202 02:52:21.347768 (+ 65us) alter_schema_op.cc:73] Released schema lock
0202 02:52:21.347772 (+ 4us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":144357,"prepare.queue_time_us":78899,"prepare.run_cpu_time_us":1096,"prepare.run_wall_time_us":42157,"raft.queue_time_us":678,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":541,"replication_time_us":1581644,"thread_start_us":165931,"threads_started":2}]]}
I20260202 02:52:21.381801 1714 tablet.cc:1717] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d: Skipping requested alter to schema version 3, tablet already version 3
I20260202 02:52:21.382115 1694 tablet.cc:1717] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P b1f05820b6034228bdad884cc348716d: Skipping requested alter to schema version 3, tablet already version 3
I20260202 02:52:21.382156 1714 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:42950 (request call id 276) took 1634 ms. Trace:
I20260202 02:52:21.377614 1600 raft_consensus.cc:697] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:21.382252 1714 rpcz_store.cc:276] 0202 02:52:19.747608 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.750739 (+ 3131us) service_pool.cc:225] Handling call
0202 02:52:21.382117 (+1631378us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:19.810326 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0202 02:52:19.810445 (+ 119us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0202 02:52:21.354454 (+1544009us) alter_schema_op.cc:66] Acquired schema lock
0202 02:52:21.354465 (+ 11us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0202 02:52:21.354750 (+ 285us) alter_schema_op.cc:128] START. Timestamp: 1770000741354680
0202 02:52:21.354754 (+ 4us) op_driver.cc:348] REPLICATION: starting
0202 02:52:21.355004 (+ 250us) log.cc:844] Serialized 173 byte log entry
0202 02:52:21.381652 (+ 26648us) op_driver.cc:464] REPLICATION: finished
0202 02:52:21.381749 (+ 97us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0202 02:52:21.382032 (+ 283us) log.cc:844] Serialized 94 byte log entry
0202 02:52:21.382080 (+ 48us) alter_schema_op.cc:73] Released schema lock
0202 02:52:21.382084 (+ 4us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":36,"mutex_wait_us":1340,"prepare.queue_time_us":58,"prepare.run_cpu_time_us":96699,"prepare.run_wall_time_us":1544761,"raft.queue_time_us":22492,"replication_time_us":26849}]]}
I20260202 02:52:21.382529 1600 consensus_queue.cc:237] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.388288 1714 tablet.cc:1717] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d: Skipping requested alter to schema version 3, tablet already version 3
I20260202 02:52:21.388595 1694 tablet.cc:1717] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P b1f05820b6034228bdad884cc348716d: Skipping requested alter to schema version 3, tablet already version 3
I20260202 02:52:21.388670 1714 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:42950 (request call id 281) took 1631 ms. Trace:
I20260202 02:52:21.388782 1714 rpcz_store.cc:276] 0202 02:52:19.757361 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.757414 (+ 53us) service_pool.cc:225] Handling call
0202 02:52:21.388630 (+1631216us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:21.356496 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0202 02:52:21.356632 (+ 136us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0202 02:52:21.383686 (+ 27054us) alter_schema_op.cc:66] Acquired schema lock
0202 02:52:21.383695 (+ 9us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0202 02:52:21.383747 (+ 52us) alter_schema_op.cc:128] START. Timestamp: 1770000741383675
0202 02:52:21.383750 (+ 3us) op_driver.cc:348] REPLICATION: starting
0202 02:52:21.383884 (+ 134us) log.cc:844] Serialized 173 byte log entry
0202 02:52:21.388131 (+ 4247us) op_driver.cc:464] REPLICATION: finished
0202 02:52:21.388209 (+ 78us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0202 02:52:21.388550 (+ 341us) log.cc:844] Serialized 94 byte log entry
0202 02:52:21.388585 (+ 35us) alter_schema_op.cc:73] Released schema lock
0202 02:52:21.388591 (+ 6us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":32,"prepare.queue_time_us":1522969,"prepare.run_cpu_time_us":713,"prepare.run_wall_time_us":27475,"raft.queue_time_us":992,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":619,"replication_time_us":4349}]]}
W20260202 02:52:21.380018 713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260202 02:52:21.396258 1183 ts_tablet_manager.cc:1929] T d2fa4c3d6ec44047abf5986e502c1590 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:21.396461 1183 log.cc:1199] T d2fa4c3d6ec44047abf5986e502c1590 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/d2fa4c3d6ec44047abf5986e502c1590
I20260202 02:52:21.399772 1173 tablet_bootstrap.cc:492] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:21.399947 1173 ts_tablet_manager.cc:1403] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.034s user 0.003s sys 0.001s
I20260202 02:52:21.400776 1182 log.cc:1199] T eaa47f3388144e8b852804c167f232ce P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/eaa47f3388144e8b852804c167f232ce
I20260202 02:52:21.400843 1173 raft_consensus.cc:359] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:21.401044 1173 raft_consensus.cc:385] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:21.401136 1173 raft_consensus.cc:740] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:21.401475 1173 consensus_queue.cc:260] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:21.402079 1173 ts_tablet_manager.cc:1434] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:21.402413 1173 tablet_bootstrap.cc:492] T d06b77ff1a6f4f78b73082b00145fcf6 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:21.404208 1173 tablet_bootstrap.cc:654] T d06b77ff1a6f4f78b73082b00145fcf6 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:21.404919 1182 ts_tablet_manager.cc:1950] T eaa47f3388144e8b852804c167f232ce P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:21.404959 1183 ts_tablet_manager.cc:1950] T d2fa4c3d6ec44047abf5986e502c1590 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:21.405616 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet d2fa4c3d6ec44047abf5986e502c1590 (table default.Table_cf37f87385ee44cea1ff80cc14050ff2 [id=697b6fe62ee845d895af350881a03556]) successfully deleted
W20260202 02:52:21.405963 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet d30557fbe9084cd3860820f69dda8752 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d30557fbe9084cd3860820f69dda8752 already in progress: creating tablet
I20260202 02:52:21.412461 526 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_2ca7a7f464b54d398f58e0ca6a5e269b" } new_table_name: "default.Table_246db50484924691aa609c499b9ef15b" modify_external_catalogs: true
I20260202 02:52:21.413828 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_4fb44d5cbe71439ba073bd75711e77e0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:21.414526 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4fb44d5cbe71439ba073bd75711e77e0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:21.419495 521 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_90344989f4234a11a30653bdc34fe899 [id=fa95e04db2d443b5972de98c7c0e6457] alter complete (version 3)
I20260202 02:52:21.426574 1173 tablet_bootstrap.cc:492] T d06b77ff1a6f4f78b73082b00145fcf6 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:21.426775 1173 ts_tablet_manager.cc:1403] T d06b77ff1a6f4f78b73082b00145fcf6 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.024s user 0.005s sys 0.000s
I20260202 02:52:21.427553 1173 raft_consensus.cc:359] T d06b77ff1a6f4f78b73082b00145fcf6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.427773 1173 raft_consensus.cc:385] T d06b77ff1a6f4f78b73082b00145fcf6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:21.427847 1173 raft_consensus.cc:740] T d06b77ff1a6f4f78b73082b00145fcf6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:21.428083 1173 consensus_queue.cc:260] T d06b77ff1a6f4f78b73082b00145fcf6 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.428601 1173 ts_tablet_manager.cc:1434] T d06b77ff1a6f4f78b73082b00145fcf6 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:21.429016 1173 tablet_bootstrap.cc:492] T bbf31e5619ed4f60b4d7c8b19ec8a549 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:21.430752 1173 tablet_bootstrap.cc:654] T bbf31e5619ed4f60b4d7c8b19ec8a549 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:21.432497 1717 raft_consensus.cc:493] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:21.432703 1717 raft_consensus.cc:515] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.433631 1717 leader_election.cc:290] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:21.434453 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8095e0c6ff0041fc8b8c8b0778a20b00" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:21.435021 859 leader_election.cc:343] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:21.437253 1422 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" instance_seqno: 1770000732912730) as {username='slave'} at 127.0.104.130:38257; Asking this server to re-register.
I20260202 02:52:21.440888 968 heartbeater.cc:461] Registering TS with master...
I20260202 02:52:21.434747 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 219) took 2568 ms. Trace:
I20260202 02:52:21.445869 1173 tablet_bootstrap.cc:492] T bbf31e5619ed4f60b4d7c8b19ec8a549 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:21.446045 1173 ts_tablet_manager.cc:1403] T bbf31e5619ed4f60b4d7c8b19ec8a549 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.001s
I20260202 02:52:21.446884 1173 raft_consensus.cc:359] T bbf31e5619ed4f60b4d7c8b19ec8a549 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.447098 1173 raft_consensus.cc:385] T bbf31e5619ed4f60b4d7c8b19ec8a549 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:21.447199 1173 raft_consensus.cc:740] T bbf31e5619ed4f60b4d7c8b19ec8a549 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:21.447431 1173 consensus_queue.cc:260] T bbf31e5619ed4f60b4d7c8b19ec8a549 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.448051 1173 ts_tablet_manager.cc:1434] T bbf31e5619ed4f60b4d7c8b19ec8a549 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:21.448712 1173 tablet_bootstrap.cc:492] T 0d3633f5f9f54115bc78b9c1b78aba67 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:21.449455 1422 ts_manager.cc:194] Registered new tserver with Master: 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:21.451056 1173 tablet_bootstrap.cc:654] T 0d3633f5f9f54115bc78b9c1b78aba67 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:21.446053 1182 rpcz_store.cc:276] 0202 02:52:18.837583 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:18.846264 (+ 8681us) service_pool.cc:225] Handling call
0202 02:52:21.246468 (+2400204us) ts_tablet_manager.cc:1151] Deleting tablet eaa47f3388144e8b852804c167f232ce
0202 02:52:21.246480 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:21.368451 (+121971us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:21.371665 (+ 3214us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:21.405526 (+ 33861us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2376610}
I20260202 02:52:21.453024 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "80d443bdb66d4cc68ae283e08c410fe6" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:21.455327 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 243) took 2602 ms. Trace:
I20260202 02:52:21.456437 1182 rpcz_store.cc:276] 0202 02:52:18.852566 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:18.869893 (+ 17327us) service_pool.cc:225] Handling call
0202 02:52:21.452916 (+2583023us) ts_tablet_manager.cc:1151] Deleting tablet 13cd1383348b458894fdecc22bf24b74
0202 02:52:21.452932 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:21.454882 (+ 1950us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2581701}
W20260202 02:52:21.458658 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 13cd1383348b458894fdecc22bf24b74 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 13cd1383348b458894fdecc22bf24b74 already in progress: creating tablet
I20260202 02:52:21.461777 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 244) took 2608 ms. Trace:
I20260202 02:52:21.462482 1173 tablet_bootstrap.cc:492] T 0d3633f5f9f54115bc78b9c1b78aba67 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:21.462785 1173 ts_tablet_manager.cc:1403] T 0d3633f5f9f54115bc78b9c1b78aba67 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.002s
I20260202 02:52:21.463752 1173 raft_consensus.cc:359] T 0d3633f5f9f54115bc78b9c1b78aba67 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.463994 1173 raft_consensus.cc:385] T 0d3633f5f9f54115bc78b9c1b78aba67 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:21.464076 1173 raft_consensus.cc:740] T 0d3633f5f9f54115bc78b9c1b78aba67 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:21.464424 1175 tablet_bootstrap.cc:492] T 75b97d7e848241cf8d894b0ae92a8f8f P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:21.464383 1173 consensus_queue.cc:260] T 0d3633f5f9f54115bc78b9c1b78aba67 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.464731 1175 ts_tablet_manager.cc:1403] T 75b97d7e848241cf8d894b0ae92a8f8f P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.287s user 0.006s sys 0.006s
I20260202 02:52:21.464916 1173 ts_tablet_manager.cc:1434] T 0d3633f5f9f54115bc78b9c1b78aba67 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260202 02:52:21.465271 1173 tablet_bootstrap.cc:492] T f58a7b25815c4b5bac568aa11c19610c P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:21.465417 1174 tablet_bootstrap.cc:492] T 0186572e7bb9499a932a636ab280f544 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:21.465583 1174 ts_tablet_manager.cc:1403] T 0186572e7bb9499a932a636ab280f544 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.324s user 0.001s sys 0.005s
I20260202 02:52:21.466375 1174 raft_consensus.cc:359] T 0186572e7bb9499a932a636ab280f544 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.466369 1175 raft_consensus.cc:359] T 75b97d7e848241cf8d894b0ae92a8f8f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.466567 1174 raft_consensus.cc:385] T 0186572e7bb9499a932a636ab280f544 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:21.466660 1174 raft_consensus.cc:740] T 0186572e7bb9499a932a636ab280f544 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:21.466701 1175 raft_consensus.cc:385] T 75b97d7e848241cf8d894b0ae92a8f8f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:21.466876 1175 raft_consensus.cc:740] T 75b97d7e848241cf8d894b0ae92a8f8f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:21.466894 1174 consensus_queue.cc:260] T 0186572e7bb9499a932a636ab280f544 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.467248 1175 consensus_queue.cc:260] T 75b97d7e848241cf8d894b0ae92a8f8f P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.468237 1723 raft_consensus.cc:515] T 80d443bdb66d4cc68ae283e08c410fe6 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.463091 1182 rpcz_store.cc:276] 0202 02:52:18.852743 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:18.870814 (+ 18071us) service_pool.cc:225] Handling call
0202 02:52:21.459185 (+2588371us) ts_tablet_manager.cc:1151] Deleting tablet 8774c6bc0fe9414ea2f17ed859d9392a
0202 02:52:21.459699 (+ 514us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:21.461285 (+ 1586us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2588193}
I20260202 02:52:21.468672 779 raft_consensus.cc:2393] T 80d443bdb66d4cc68ae283e08c410fe6 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1521a6d262b7451bbef8debe970dd6b5 in current term 1: Already voted for candidate 1f442ef85029461091de847118c34c0d in this term.
I20260202 02:52:21.470057 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 245) took 2617 ms. Trace:
W20260202 02:52:21.470227 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 8774c6bc0fe9414ea2f17ed859d9392a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8774c6bc0fe9414ea2f17ed859d9392a already in progress: creating tablet
I20260202 02:52:21.471329 1723 leader_election.cc:290] T 80d443bdb66d4cc68ae283e08c410fe6 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:21.471303 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "80d443bdb66d4cc68ae283e08c410fe6" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:21.471772 1173 tablet_bootstrap.cc:654] T f58a7b25815c4b5bac568aa11c19610c P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:21.471848 715 leader_election.cc:343] T 80d443bdb66d4cc68ae283e08c410fe6 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:21.474129 528 master_service.cc:954] ReplaceTablet: received request to replace tablet 6154460838eb4e508f3646d3e3e4cf21 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:21.475193 522 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_028fb132e4894101b8470628a6041c07" } new_table_name: "default.Table_d0e51ff8e83f4da19ce3c119dbec1ea7" modify_external_catalogs: true
I20260202 02:52:21.476022 1175 ts_tablet_manager.cc:1434] T 75b97d7e848241cf8d894b0ae92a8f8f P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.011s user 0.003s sys 0.000s
I20260202 02:52:21.478503 1714 tablet.cc:1722] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:21.478853 1708 tablet.cc:1722] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260202 02:52:21.484597 526 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6154460838eb4e508f3646d3e3e4cf21 consensus state has no leader
I20260202 02:52:21.476382 1182 rpcz_store.cc:276] 0202 02:52:18.852830 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:18.870995 (+ 18165us) service_pool.cc:225] Handling call
0202 02:52:21.469537 (+2598542us) ts_tablet_manager.cc:1151] Deleting tablet 2f0e8c5e2fd44370ae150a206f4e6cbc
0202 02:52:21.469548 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:21.469921 (+ 373us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2587571}
I20260202 02:52:21.485071 1175 tablet_bootstrap.cc:492] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:21.486043 1182 tablet_replica.cc:333] T 0186572e7bb9499a932a636ab280f544 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
W20260202 02:52:21.487681 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 2f0e8c5e2fd44370ae150a206f4e6cbc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2f0e8c5e2fd44370ae150a206f4e6cbc already in progress: creating tablet
I20260202 02:52:21.488215 899 tablet_service.cc:1505] Processing CreateTablet for tablet 1f379a29e2454f689d68f5951837512b (DEFAULT_TABLE table=default.Table_67c55c53ca254b03b0c326672a41b559 [id=7e80e30271434635a17d0034580aeca8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.488512 901 tablet_service.cc:1505] Processing CreateTablet for tablet 1d8143f529b74e6ab779dd695920413c (DEFAULT_TABLE table=default.Table_33fc32ec03094e3c8fe78de4f6a26777 [id=85340c1716bd4235bdad7042b53ce58c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.488649 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f379a29e2454f689d68f5951837512b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.488847 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d8143f529b74e6ab779dd695920413c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.492257 885 tablet_service.cc:1505] Processing CreateTablet for tablet d0ce2121d65b4a228596e0ca9018883d (DEFAULT_TABLE table=default.Table_8fcc3828b43444159f0df6023ec1ba0f [id=ca632e6d14ca41e2ab9444dc362a4265]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.492537 1038 tablet_service.cc:1505] Processing CreateTablet for tablet 587cfd0ae62f4cfd937faf6890e78f4a (DEFAULT_TABLE table=default.Table_2be7fcc0f92846059283a7691fcda407 [id=9f2441734085417fb56dc05a42df54db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.492609 884 tablet_service.cc:1505] Processing CreateTablet for tablet 44c8d5b74c8e4c499f7f68e8185efc17 (DEFAULT_TABLE table=default.Table_668e8ec9913f43fa8499eff32be8db0a [id=46b5d47377474ff9bac2acdb26b08c65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.492687 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0ce2121d65b4a228596e0ca9018883d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.492802 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 1f379a29e2454f689d68f5951837512b (DEFAULT_TABLE table=default.Table_67c55c53ca254b03b0c326672a41b559 [id=7e80e30271434635a17d0034580aeca8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.492964 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 587cfd0ae62f4cfd937faf6890e78f4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.492965 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44c8d5b74c8e4c499f7f68e8185efc17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.493145 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f379a29e2454f689d68f5951837512b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.492563 1044 tablet_service.cc:1505] Processing CreateTablet for tablet d0ce2121d65b4a228596e0ca9018883d (DEFAULT_TABLE table=default.Table_8fcc3828b43444159f0df6023ec1ba0f [id=ca632e6d14ca41e2ab9444dc362a4265]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.493605 1042 tablet_service.cc:1505] Processing CreateTablet for tablet 1d8143f529b74e6ab779dd695920413c (DEFAULT_TABLE table=default.Table_33fc32ec03094e3c8fe78de4f6a26777 [id=85340c1716bd4235bdad7042b53ce58c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.494485 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d8143f529b74e6ab779dd695920413c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.497018 1040 tablet_service.cc:1505] Processing CreateTablet for tablet 44c8d5b74c8e4c499f7f68e8185efc17 (DEFAULT_TABLE table=default.Table_668e8ec9913f43fa8499eff32be8db0a [id=46b5d47377474ff9bac2acdb26b08c65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.497442 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44c8d5b74c8e4c499f7f68e8185efc17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.487582 1182 raft_consensus.cc:2243] T 0186572e7bb9499a932a636ab280f544 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:21.503647 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0ce2121d65b4a228596e0ca9018883d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.503472 1182 raft_consensus.cc:2272] T 0186572e7bb9499a932a636ab280f544 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:21.507579 1173 tablet_bootstrap.cc:492] T f58a7b25815c4b5bac568aa11c19610c P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:21.507777 1173 ts_tablet_manager.cc:1403] T f58a7b25815c4b5bac568aa11c19610c P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.043s user 0.001s sys 0.006s
I20260202 02:52:21.509111 1173 raft_consensus.cc:359] T f58a7b25815c4b5bac568aa11c19610c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.509325 1173 raft_consensus.cc:385] T f58a7b25815c4b5bac568aa11c19610c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:21.509420 1173 raft_consensus.cc:740] T f58a7b25815c4b5bac568aa11c19610c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:21.509400 1034 tablet_service.cc:1505] Processing CreateTablet for tablet ac2ab12f5040488aa306cb69b107b485 (DEFAULT_TABLE table=default.Table_35c5c99502d0489081751bc7403f6ad7 [id=640ad152bb4243cda5a7e7242996e2fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.509858 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac2ab12f5040488aa306cb69b107b485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.509819 1173 consensus_queue.cc:260] T f58a7b25815c4b5bac568aa11c19610c P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.510633 1173 ts_tablet_manager.cc:1434] T f58a7b25815c4b5bac568aa11c19610c P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260202 02:52:21.510965 1173 tablet_bootstrap.cc:492] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:21.511281 1182 ts_tablet_manager.cc:1916] T 0186572e7bb9499a932a636ab280f544 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:21.509138 1043 tablet_service.cc:1505] Processing CreateTablet for tablet 8b0368e03f644f1db28dbb57c95c48c1 (DEFAULT_TABLE table=default.Table_f93837f1f1614be4bb992e92e5237ea2 [id=8882e1fbac2949838c99adb1290f273e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.512837 1173 tablet_bootstrap.cc:654] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:21.513679 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b0368e03f644f1db28dbb57c95c48c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.516098 1173 tablet_bootstrap.cc:492] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:21.516287 1173 ts_tablet_manager.cc:1403] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260202 02:52:21.517095 1039 tablet_service.cc:1505] Processing CreateTablet for tablet 65bd1c1a382e42e4a68077264563c55b (DEFAULT_TABLE table=default.Table_1cc0209a40ad471fb1c41012a476df79 [id=b941bc2edc03444394d1ba8a596a0d27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.517140 1173 raft_consensus.cc:359] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.517367 1173 raft_consensus.cc:385] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:21.517434 1173 raft_consensus.cc:740] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:21.517525 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65bd1c1a382e42e4a68077264563c55b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.517659 1173 consensus_queue.cc:260] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.518299 1173 ts_tablet_manager.cc:1434] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:21.518637 1173 tablet_bootstrap.cc:492] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:21.520560 1173 tablet_bootstrap.cc:654] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:21.522969 1178 raft_consensus.cc:493] T 0d3633f5f9f54115bc78b9c1b78aba67 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:21.523138 1178 raft_consensus.cc:515] T 0d3633f5f9f54115bc78b9c1b78aba67 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.523299 1175 tablet_bootstrap.cc:654] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:21.524310 1178 leader_election.cc:290] T 0d3633f5f9f54115bc78b9c1b78aba67 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:21.525383 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0d3633f5f9f54115bc78b9c1b78aba67" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:21.525893 1039 tablet_service.cc:1505] Processing CreateTablet for tablet 939fbcaf75c7439d9208bbbf83a72717 (DEFAULT_TABLE table=default.Table_12d68a317c8e4d809dbbe77c095a1f6e [id=7219522ffc3849faab94d754a42a8511]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.526319 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 939fbcaf75c7439d9208bbbf83a72717. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.534476 1178 raft_consensus.cc:493] T d06b77ff1a6f4f78b73082b00145fcf6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:21.534646 1178 raft_consensus.cc:515] T d06b77ff1a6f4f78b73082b00145fcf6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.535533 1178 leader_election.cc:290] T d06b77ff1a6f4f78b73082b00145fcf6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:21.536260 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d06b77ff1a6f4f78b73082b00145fcf6" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:21.539515 1182 ts_tablet_manager.cc:1929] T 0186572e7bb9499a932a636ab280f544 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:21.539669 1182 log.cc:1199] T 0186572e7bb9499a932a636ab280f544 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/0186572e7bb9499a932a636ab280f544
I20260202 02:52:21.540086 1182 ts_tablet_manager.cc:1950] T 0186572e7bb9499a932a636ab280f544 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:21.545786 894 tablet_service.cc:1505] Processing CreateTablet for tablet ac2ab12f5040488aa306cb69b107b485 (DEFAULT_TABLE table=default.Table_35c5c99502d0489081751bc7403f6ad7 [id=640ad152bb4243cda5a7e7242996e2fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.546324 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac2ab12f5040488aa306cb69b107b485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.546379 1036 tablet_service.cc:1505] Processing CreateTablet for tablet da5435f969074364bc8f8a6ddc5e4f8a (DEFAULT_TABLE table=default.Table_51648e2a42614c3d9c5820901d153932 [id=ad256e6219784dff9aa916cf07f67fbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.546518 893 tablet_service.cc:1505] Processing CreateTablet for tablet 65bd1c1a382e42e4a68077264563c55b (DEFAULT_TABLE table=default.Table_1cc0209a40ad471fb1c41012a476df79 [id=b941bc2edc03444394d1ba8a596a0d27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.546803 1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da5435f969074364bc8f8a6ddc5e4f8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.546873 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65bd1c1a382e42e4a68077264563c55b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.551539 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_06273452ab3e4d2e8bdb1e3a41721dcc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:21.552126 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_06273452ab3e4d2e8bdb1e3a41721dcc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:21.552686 892 tablet_service.cc:1505] Processing CreateTablet for tablet 939fbcaf75c7439d9208bbbf83a72717 (DEFAULT_TABLE table=default.Table_12d68a317c8e4d809dbbe77c095a1f6e [id=7219522ffc3849faab94d754a42a8511]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.553069 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 939fbcaf75c7439d9208bbbf83a72717. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.556659 890 tablet_service.cc:1505] Processing CreateTablet for tablet 93f958924885457ab15f2aac5509e0aa (DEFAULT_TABLE table=default.Table_2e2049e67b8943acab5a0db3deb1535a [id=0db38635f6af4f168f221f4749adf0dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.557133 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93f958924885457ab15f2aac5509e0aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.557446 527 master_service.cc:954] ReplaceTablet: received request to replace tablet 8095e0c6ff0041fc8b8c8b0778a20b00 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:21.560727 898 tablet_service.cc:1505] Processing CreateTablet for tablet 7709ea34429e4e91ad4922a3d73ebd14 (DEFAULT_TABLE table=default.Table_8262b850970641b1bbe1cf9d1c3d9409 [id=c6e5418bf48e43e2adb6d1b60445f0b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.561165 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7709ea34429e4e91ad4922a3d73ebd14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.563326 521 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_10aa0dafec0547069eb92ecf12b3c780 [id=1f9c48c8e2484856a66a0719268f22ae] alter complete (version 1)
I20260202 02:52:21.565554 520 master_service.cc:954] ReplaceTablet: received request to replace tablet c12ff20ca6b44ddba0c936a28cfb2eb9 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:21.566032 520 master_service.cc:954] ReplaceTablet: received request to replace tablet ffd80badce1048f3ac95366cad05ae53 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:21.565976 884 tablet_service.cc:1505] Processing CreateTablet for tablet 587cfd0ae62f4cfd937faf6890e78f4a (DEFAULT_TABLE table=default.Table_2be7fcc0f92846059283a7691fcda407 [id=9f2441734085417fb56dc05a42df54db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.566408 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 587cfd0ae62f4cfd937faf6890e78f4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.567041 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_fbe3212129904307af7133ead4053a59"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:21.567574 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fbe3212129904307af7133ead4053a59 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:21.568650 901 tablet_service.cc:1505] Processing CreateTablet for tablet 8b0368e03f644f1db28dbb57c95c48c1 (DEFAULT_TABLE table=default.Table_f93837f1f1614be4bb992e92e5237ea2 [id=8882e1fbac2949838c99adb1290f273e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.569029 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b0368e03f644f1db28dbb57c95c48c1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:21.569691 522 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 70306ed9dec54286885d965f2c012da4 consensus state has no leader
I20260202 02:52:21.570068 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_c9f4a79d03a04c2e8135ba9950104fc8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:21.570531 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c9f4a79d03a04c2e8135ba9950104fc8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:21.571496 418 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskYXxklX/build/asan/bin/kudu pid 426
I20260202 02:52:21.572733 487 generic_service.cc:196] Checking for leaks (request via RPC)
I20260202 02:52:21.574057 1035 tablet_service.cc:1505] Processing CreateTablet for tablet 79badde1001b432b95f58b5161ace15c (DEFAULT_TABLE table=default.Table_80912cc76f934f139da7f866546ae97a [id=2f3bf7e0cf4c41ac8a2554360b57214a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.574249 1039 tablet_service.cc:1505] Processing CreateTablet for tablet 2b352d9a815e4bccafb7e2251500c5bc (DEFAULT_TABLE table=default.Table_408b8f21ff294ce9880d0b9a4d681f93 [id=c0fa167c7d1f47ea994fcf7dfdd4100f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.574472 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79badde1001b432b95f58b5161ace15c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.575141 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b352d9a815e4bccafb7e2251500c5bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.576864 1041 tablet_service.cc:1505] Processing CreateTablet for tablet 3284213e4b0447b49a7718ba896d3fa8 (DEFAULT_TABLE table=default.Table_e71a7347855d447cabe6ea2cc64b293a [id=e490dd882e8e4e51ad5a2073c891ba91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.577288 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3284213e4b0447b49a7718ba896d3fa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.579196 899 tablet_service.cc:1505] Processing CreateTablet for tablet 0138ba7326674741b0729324735f1de7 (DEFAULT_TABLE table=default.Table_516d610640654ea4b49f7de3b04cbb0c [id=5c8367fbb78840d18a26a026d75c3d6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.579660 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0138ba7326674741b0729324735f1de7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.581015 1033 tablet_service.cc:1505] Processing CreateTablet for tablet 2f5aae089fcc492f9ce55f84674a452c (DEFAULT_TABLE table=default.Table_84d6693620cc468e82d5cb1ec6cc69ea [id=f6625b21b6064828b11acdfdb694cd71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.581533 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f5aae089fcc492f9ce55f84674a452c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.585218 1032 tablet_service.cc:1505] Processing CreateTablet for tablet 75a24a1bf54d4c799938a5723ac6fa99 (DEFAULT_TABLE table=default.Table_52a2f389faba40fea93e2bca972e81e1 [id=70ac373200604907b00132eb4727ea5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.585884 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75a24a1bf54d4c799938a5723ac6fa99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.589032 1031 tablet_service.cc:1505] Processing CreateTablet for tablet 7ea7a5d8a92b4ac0903483627258ae66 (DEFAULT_TABLE table=default.Table_d040708acb92480496bac8b012890006 [id=1c26d457c06345b7abeb1941bcee6c31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.589579 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ea7a5d8a92b4ac0903483627258ae66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.599761 1173 tablet_bootstrap.cc:492] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:21.599777 525 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 2f5aae089fcc492f9ce55f84674a452c
I20260202 02:52:21.599933 1173 ts_tablet_manager.cc:1403] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.081s user 0.004s sys 0.003s
I20260202 02:52:21.600965 519 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 0d3633f5f9f54115bc78b9c1b78aba67
I20260202 02:52:21.601020 528 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 6154460838eb4e508f3646d3e3e4cf21
I20260202 02:52:21.601389 528 catalog_manager.cc:6724] ReplaceTablet: tablet 6154460838eb4e508f3646d3e3e4cf21 deleted and replaced by tablet 5e20842de8cb4355b732685605c88834
I20260202 02:52:21.602416 1173 raft_consensus.cc:359] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:21.602603 1173 raft_consensus.cc:385] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:21.602687 1173 raft_consensus.cc:740] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:21.602903 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 242) took 2750 ms. Trace:
I20260202 02:52:21.602927 1173 consensus_queue.cc:260] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:21.603070 1182 rpcz_store.cc:276] 0202 02:52:18.852184 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:18.853700 (+ 1516us) service_pool.cc:225] Handling call
0202 02:52:21.485524 (+2631824us) ts_tablet_manager.cc:1151] Deleting tablet 0186572e7bb9499a932a636ab280f544
0202 02:52:21.485538 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:21.536044 (+ 50506us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:21.539369 (+ 3325us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:21.602857 (+ 63488us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2601511}
I20260202 02:52:21.603586 1173 ts_tablet_manager.cc:1434] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260202 02:52:21.603713 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 224) took 2756 ms. Trace:
I20260202 02:52:21.603910 1173 tablet_bootstrap.cc:492] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:21.603876 1182 rpcz_store.cc:276] 0202 02:52:18.847017 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:18.849965 (+ 2948us) service_pool.cc:225] Handling call
0202 02:52:21.603565 (+2753600us) ts_tablet_manager.cc:1151] Deleting tablet 23c38ef4e3c840808ec36f859cf6f6de
0202 02:52:21.603578 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:21.603666 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2718913}
I20260202 02:52:21.604435 1182 tablet_replica.cc:333] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:21.604648 1182 raft_consensus.cc:2243] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:21.604821 1182 raft_consensus.cc:2272] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:21.605572 1182 ts_tablet_manager.cc:1916] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:21.606206 1173 tablet_bootstrap.cc:654] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:21.607657 1175 tablet_bootstrap.cc:492] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:21.607836 1175 ts_tablet_manager.cc:1403] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.124s user 0.006s sys 0.001s
I20260202 02:52:21.608775 1175 raft_consensus.cc:359] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.608979 1175 raft_consensus.cc:385] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:21.609066 1175 raft_consensus.cc:740] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:21.609350 1175 consensus_queue.cc:260] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.612105 1175 ts_tablet_manager.cc:1434] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260202 02:52:21.619514 1173 tablet_bootstrap.cc:492] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:21.619503 1178 raft_consensus.cc:493] T bbf31e5619ed4f60b4d7c8b19ec8a549 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:21.619735 1173 ts_tablet_manager.cc:1403] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.000s
I20260202 02:52:21.619760 1178 raft_consensus.cc:515] T bbf31e5619ed4f60b4d7c8b19ec8a549 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.620867 1178 leader_election.cc:290] T bbf31e5619ed4f60b4d7c8b19ec8a549 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:21.621374 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bbf31e5619ed4f60b4d7c8b19ec8a549" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:21.621675 778 raft_consensus.cc:2393] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1f05820b6034228bdad884cc348716d in current term 1: Already voted for candidate 1f442ef85029461091de847118c34c0d in this term.
I20260202 02:52:21.621973 1030 tablet_service.cc:1552] Processing DeleteTablet for tablet 0d3633f5f9f54115bc78b9c1b78aba67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:21.622344 1183 tablet_replica.cc:333] T 0d3633f5f9f54115bc78b9c1b78aba67 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:21.622519 1030 tablet_service.cc:1552] Processing DeleteTablet for tablet 6154460838eb4e508f3646d3e3e4cf21 with delete_type TABLET_DATA_DELETED (replaced by tablet 5e20842de8cb4355b732685605c88834) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:21.622601 1183 raft_consensus.cc:2243] T 0d3633f5f9f54115bc78b9c1b78aba67 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:21.622824 1183 raft_consensus.cc:2272] T 0d3633f5f9f54115bc78b9c1b78aba67 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:21.623310 1039 tablet_service.cc:1552] Processing DeleteTablet for tablet 2f5aae089fcc492f9ce55f84674a452c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:21.623353 1030 tablet_service.cc:1505] Processing CreateTablet for tablet 8153510d5d154c7895b1ee794585612e (DEFAULT_TABLE table=default.Table_881157f1bd2945a5be1340a99a941bef [id=c0c6499675894350ac008b534d030c93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.623960 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8153510d5d154c7895b1ee794585612e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.624025 1183 ts_tablet_manager.cc:1916] T 0d3633f5f9f54115bc78b9c1b78aba67 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:21.624670 1175 tablet_bootstrap.cc:492] T 1f54c1dfe0a942b3bafd85522953a032 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:21.629247 1029 tablet_service.cc:1505] Processing CreateTablet for tablet e2edc6660b9a41e39e9529627fafcb03 (DEFAULT_TABLE table=default.Table_9d4fbf619567432a88aa9a0c1b19cb3e [id=fff85ebd1bd1415c932f7c9c5a0da42a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.629809 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2edc6660b9a41e39e9529627fafcb03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.631203 1039 tablet_service.cc:1505] Processing CreateTablet for tablet b427c7e897c64977b924e7d586420e69 (DEFAULT_TABLE table=default.Table_7d0cce2fe6094a54a4b51acf6b1d42da [id=98ee1a7044cd46d8b39dfda755ed8a7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.631632 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b427c7e897c64977b924e7d586420e69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.634166 1183 ts_tablet_manager.cc:1929] T 0d3633f5f9f54115bc78b9c1b78aba67 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:21.634300 1183 log.cc:1199] T 0d3633f5f9f54115bc78b9c1b78aba67 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/0d3633f5f9f54115bc78b9c1b78aba67
I20260202 02:52:21.633850 1182 ts_tablet_manager.cc:1929] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:21.634413 1175 tablet_bootstrap.cc:654] T 1f54c1dfe0a942b3bafd85522953a032 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:21.634596 1182 log.cc:1199] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/9884c90cf1d245b2a7eaaf00d71b1c3e
I20260202 02:52:21.634668 1183 ts_tablet_manager.cc:1950] T 0d3633f5f9f54115bc78b9c1b78aba67 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:21.635703 918 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0d3633f5f9f54115bc78b9c1b78aba67" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:21.635783 1182 ts_tablet_manager.cc:1950] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:21.635995 918 raft_consensus.cc:2393] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1f05820b6034228bdad884cc348716d in current term 1: Already voted for candidate 1f442ef85029461091de847118c34c0d in this term.
I20260202 02:52:21.636402 917 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d06b77ff1a6f4f78b73082b00145fcf6" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:21.637097 892 tablet_service.cc:1505] Processing CreateTablet for tablet da5435f969074364bc8f8a6ddc5e4f8a (DEFAULT_TABLE table=default.Table_51648e2a42614c3d9c5820901d153932 [id=ad256e6219784dff9aa916cf07f67fbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.637635 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da5435f969074364bc8f8a6ddc5e4f8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.640288 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 0186572e7bb9499a932a636ab280f544 (table default.Table_8cae34d2c6f1443ba07e3f2f8abd1989 [id=dc77705cb5704fd58af1eed3edd45675]) successfully deleted
W20260202 02:52:21.640592 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 23c38ef4e3c840808ec36f859cf6f6de because the tablet was not found. No further retry: Not found: Tablet not found: 23c38ef4e3c840808ec36f859cf6f6de
I20260202 02:52:21.641566 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 0d3633f5f9f54115bc78b9c1b78aba67 (table default.Table_63bb1b19c34a430e94ad0d7bd9bbe795 [id=af3058593c2747bf99db3b8da1aaedc7]) successfully deleted
W20260202 02:52:21.641824 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 6154460838eb4e508f3646d3e3e4cf21 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6154460838eb4e508f3646d3e3e4cf21 already in progress: creating tablet
W20260202 02:52:21.642043 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 2f5aae089fcc492f9ce55f84674a452c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2f5aae089fcc492f9ce55f84674a452c already in progress: creating tablet
I20260202 02:52:21.642509 999 leader_election.cc:304] T 0d3633f5f9f54115bc78b9c1b78aba67 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
W20260202 02:52:21.642756 999 raft_consensus.cc:2686] T 0d3633f5f9f54115bc78b9c1b78aba67 P b1f05820b6034228bdad884cc348716d: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260202 02:52:21.643424 999 leader_election.cc:343] T bbf31e5619ed4f60b4d7c8b19ec8a549 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: bbf31e5619ed4f60b4d7c8b19ec8a549
I20260202 02:52:21.643584 999 leader_election.cc:304] T bbf31e5619ed4f60b4d7c8b19ec8a549 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:21.643852 1178 raft_consensus.cc:3060] T bbf31e5619ed4f60b4d7c8b19ec8a549 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:21.644888 1173 raft_consensus.cc:359] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.645156 1173 raft_consensus.cc:385] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:21.645885 1173 raft_consensus.cc:740] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:21.646206 1173 consensus_queue.cc:260] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.646801 1173 ts_tablet_manager.cc:1434] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.027s user 0.001s sys 0.001s
I20260202 02:52:21.647351 1178 raft_consensus.cc:2749] T bbf31e5619ed4f60b4d7c8b19ec8a549 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:21.647814 999 leader_election.cc:304] T d06b77ff1a6f4f78b73082b00145fcf6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:21.648432 1178 raft_consensus.cc:2749] T d06b77ff1a6f4f78b73082b00145fcf6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:21.649115 1031 tablet_service.cc:1505] Processing CreateTablet for tablet abfe7f76b2424c4d9ab07b29b66ec103 (DEFAULT_TABLE table=default.Table_ff54266e8b5a4334ad1fbeda5270f0f2 [id=b6c09db68b4542a3bb11bf20b1d54571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.650884 1034 tablet_service.cc:1505] Processing CreateTablet for tablet 7709ea34429e4e91ad4922a3d73ebd14 (DEFAULT_TABLE table=default.Table_8262b850970641b1bbe1cf9d1c3d9409 [id=c6e5418bf48e43e2adb6d1b60445f0b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.651428 1042 tablet_service.cc:1505] Processing CreateTablet for tablet f3eebefa74584c1791d52377f5ba1290 (DEFAULT_TABLE table=default.Table_80e6b91d5978450dbfd9bb2b4b410d3c [id=e0cea4989f66443faf074eacf434658c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.652400 1038 tablet_service.cc:1505] Processing CreateTablet for tablet 0138ba7326674741b0729324735f1de7 (DEFAULT_TABLE table=default.Table_516d610640654ea4b49f7de3b04cbb0c [id=5c8367fbb78840d18a26a026d75c3d6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.653107 1040 tablet_service.cc:1505] Processing CreateTablet for tablet 07580ad1d049475a9645acfad0b1d219 (DEFAULT_TABLE table=default.Table_444485558ccc4c6ba771ab686bc9e1c1 [id=c4207a758ebb4784a67c662cfa4333d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.653550 1041 tablet_service.cc:1505] Processing CreateTablet for tablet c8a273ac8cd344719a48fd81945bed0a (DEFAULT_TABLE table=default.Table_0601f3c5b49446b3a7ec44443f89a38c [id=62c3c39fb73249349cee49b86d535612]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.654013 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 93f958924885457ab15f2aac5509e0aa (DEFAULT_TABLE table=default.Table_2e2049e67b8943acab5a0db3deb1535a [id=0db38635f6af4f168f221f4749adf0dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.654716 1173 tablet_bootstrap.cc:492] T 8095e0c6ff0041fc8b8c8b0778a20b00 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:21.656756 1173 tablet_bootstrap.cc:654] T 8095e0c6ff0041fc8b8c8b0778a20b00 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:21.656750 1043 tablet_service.cc:1505] Processing CreateTablet for tablet 99e0b5e9a9874aec886155e0d614dec2 (DEFAULT_TABLE table=default.Table_a8c4f67a58b74bf8bb3716174ed5654f [id=e7975854ed2f4810a2d438b590267584]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.657232 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99e0b5e9a9874aec886155e0d614dec2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.658002 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07580ad1d049475a9645acfad0b1d219. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.662139 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8a273ac8cd344719a48fd81945bed0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.663749 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abfe7f76b2424c4d9ab07b29b66ec103. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.665421 1178 raft_consensus.cc:493] T fe7b6004e5094c0dbe5847144a2489b9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:21.666181 1178 raft_consensus.cc:515] T fe7b6004e5094c0dbe5847144a2489b9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.667116 1178 leader_election.cc:290] T fe7b6004e5094c0dbe5847144a2489b9 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:21.668753 1708 tablet.cc:1722] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260202 02:52:21.670750 714 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 320483us
W20260202 02:52:21.671196 714 leader_election.cc:343] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: f93e8d3c4faf48eaa98c3cbf88d585e2
I20260202 02:52:21.671411 714 leader_election.cc:304] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:21.673187 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8095e0c6ff0041fc8b8c8b0778a20b00" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:21.673467 779 raft_consensus.cc:2468] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
I20260202 02:52:21.674532 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93f958924885457ab15f2aac5509e0aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.676443 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3eebefa74584c1791d52377f5ba1290. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.678344 1032 tablet_service.cc:1505] Processing CreateTablet for tablet 59ad016e15ec4a488921eb564029b689 (DEFAULT_TABLE table=default.Table_49cb996504bf45e9a24c6f3c01f457ff [id=ec1d1bc3e38643f7ad1c57e98ef66388]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.678840 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7709ea34429e4e91ad4922a3d73ebd14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.679873 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0138ba7326674741b0729324735f1de7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.683897 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 246) took 1921 ms. Trace:
I20260202 02:52:21.684099 1182 rpcz_store.cc:276] 0202 02:52:19.717260 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.717303 (+ 43us) service_pool.cc:225] Handling call
0202 02:52:21.604362 (+1887059us) ts_tablet_manager.cc:1151] Deleting tablet 9884c90cf1d245b2a7eaaf00d71b1c3e
0202 02:52:21.604372 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:21.618709 (+ 14337us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:21.633588 (+ 14879us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:21.638470 (+ 4882us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1886805}
I20260202 02:52:21.684872 1182 tablet_replica.cc:333] T 526f2c7b95184a0987e7aa94efc8475a P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:21.685097 1182 raft_consensus.cc:2243] T 526f2c7b95184a0987e7aa94efc8475a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:21.685781 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 9884c90cf1d245b2a7eaaf00d71b1c3e (table default.Table_22dd5a0be7944811bf358ec584461157 [id=7c8c316a6dd74622a257e70c3299ea1b]) successfully deleted
I20260202 02:52:21.685974 1182 raft_consensus.cc:2272] T 526f2c7b95184a0987e7aa94efc8475a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:21.686650 1182 ts_tablet_manager.cc:1916] T 526f2c7b95184a0987e7aa94efc8475a P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:21.689033 857 leader_election.cc:304] T 80d443bdb66d4cc68ae283e08c410fe6 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:21.689762 1717 raft_consensus.cc:2749] T 80d443bdb66d4cc68ae283e08c410fe6 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260202 02:52:21.689797 857 leader_election.cc:304] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:21.690653 901 tablet_service.cc:1505] Processing CreateTablet for tablet 2b352d9a815e4bccafb7e2251500c5bc (DEFAULT_TABLE table=default.Table_408b8f21ff294ce9880d0b9a4d681f93 [id=c0fa167c7d1f47ea994fcf7dfdd4100f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.691080 1656 raft_consensus.cc:2804] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:21.692411 1656 raft_consensus.cc:493] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:21.693334 1656 raft_consensus.cc:3060] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:21.694025 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b352d9a815e4bccafb7e2251500c5bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.703735 898 tablet_service.cc:1505] Processing CreateTablet for tablet 75a24a1bf54d4c799938a5723ac6fa99 (DEFAULT_TABLE table=default.Table_52a2f389faba40fea93e2bca972e81e1 [id=70ac373200604907b00132eb4727ea5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.704298 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75a24a1bf54d4c799938a5723ac6fa99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.704660 893 tablet_service.cc:1505] Processing CreateTablet for tablet 79badde1001b432b95f58b5161ace15c (DEFAULT_TABLE table=default.Table_80912cc76f934f139da7f866546ae97a [id=2f3bf7e0cf4c41ac8a2554360b57214a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.705137 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79badde1001b432b95f58b5161ace15c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.708510 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 415d14533fd54f20adb00455a7bcad5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:16 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:21.709045 890 tablet_service.cc:1505] Processing CreateTablet for tablet 3284213e4b0447b49a7718ba896d3fa8 (DEFAULT_TABLE table=default.Table_e71a7347855d447cabe6ea2cc64b293a [id=e490dd882e8e4e51ad5a2073c891ba91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.709496 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3284213e4b0447b49a7718ba896d3fa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.712118 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 2f5aae089fcc492f9ce55f84674a452c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:21.712230 894 tablet_service.cc:1552] Processing DeleteTablet for tablet 6154460838eb4e508f3646d3e3e4cf21 with delete_type TABLET_DATA_DELETED (replaced by tablet 5e20842de8cb4355b732685605c88834) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:21.713235 899 tablet_service.cc:1505] Processing CreateTablet for tablet 2f5aae089fcc492f9ce55f84674a452c (DEFAULT_TABLE table=default.Table_84d6693620cc468e82d5cb1ec6cc69ea [id=f6625b21b6064828b11acdfdb694cd71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.713688 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f5aae089fcc492f9ce55f84674a452c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.714033 885 tablet_service.cc:1505] Processing CreateTablet for tablet abfe7f76b2424c4d9ab07b29b66ec103 (DEFAULT_TABLE table=default.Table_ff54266e8b5a4334ad1fbeda5270f0f2 [id=b6c09db68b4542a3bb11bf20b1d54571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.714411 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abfe7f76b2424c4d9ab07b29b66ec103. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.714730 1708 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:42950 (request call id 257) took 2164 ms. Trace:
I20260202 02:52:21.714856 1708 rpcz_store.cc:276] 0202 02:52:19.550722 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.550789 (+ 67us) service_pool.cc:225] Handling call
0202 02:52:21.714690 (+2163901us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:19.600101 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0202 02:52:19.600900 (+ 799us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0202 02:52:19.600910 (+ 10us) alter_schema_op.cc:66] Acquired schema lock
0202 02:52:19.600917 (+ 7us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0202 02:52:19.600960 (+ 43us) alter_schema_op.cc:128] START. Timestamp: 1770000739600767
0202 02:52:19.600963 (+ 3us) op_driver.cc:348] REPLICATION: starting
0202 02:52:19.601278 (+ 315us) log.cc:844] Serialized 173 byte log entry
0202 02:52:21.668548 (+2067270us) op_driver.cc:464] REPLICATION: finished
0202 02:52:21.668681 (+ 133us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0202 02:52:21.669210 (+ 529us) log.cc:844] Serialized 5 byte log entry
0202 02:52:21.703984 (+ 34774us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:21.704307 (+ 323us) log.cc:844] Serialized 14 byte log entry
0202 02:52:21.714633 (+ 10326us) alter_schema_op.cc:73] Released schema lock
0202 02:52:21.714639 (+ 6us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":72,"prepare.queue_time_us":48821,"prepare.run_cpu_time_us":4008,"prepare.run_wall_time_us":84006,"raft.queue_time_us":46,"raft.run_cpu_time_us":308,"raft.run_wall_time_us":6336,"replication_time_us":2067529,"thread_start_us":82606,"threads_started":1,"wal-append.queue_time_us":81220,"wal-append.run_cpu_time_us":451,"wal-append.run_wall_time_us":1000686}]]}
I20260202 02:52:21.715786 1174 ts_tablet_manager.cc:1434] T 0186572e7bb9499a932a636ab280f544 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.250s user 0.001s sys 0.001s
I20260202 02:52:21.716099 1174 tablet_bootstrap.cc:492] T eaa47f3388144e8b852804c167f232ce P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:21.716945 886 tablet_service.cc:1505] Processing CreateTablet for tablet 8153510d5d154c7895b1ee794585612e (DEFAULT_TABLE table=default.Table_881157f1bd2945a5be1340a99a941bef [id=c0c6499675894350ac008b534d030c93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.717283 884 tablet_service.cc:1505] Processing CreateTablet for tablet c8a273ac8cd344719a48fd81945bed0a (DEFAULT_TABLE table=default.Table_0601f3c5b49446b3a7ec44443f89a38c [id=62c3c39fb73249349cee49b86d535612]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.717374 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8153510d5d154c7895b1ee794585612e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.717633 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8a273ac8cd344719a48fd81945bed0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.717811 1174 tablet_bootstrap.cc:654] T eaa47f3388144e8b852804c167f232ce P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:21.713244 894 tablet_service.cc:1552] Processing DeleteTablet for tablet 0d3633f5f9f54115bc78b9c1b78aba67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:21.721249 524 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 6df3d11cf578404ca41807f0777b6c53
I20260202 02:52:21.720984 902 tablet_service.cc:1505] Processing CreateTablet for tablet b427c7e897c64977b924e7d586420e69 (DEFAULT_TABLE table=default.Table_7d0cce2fe6094a54a4b51acf6b1d42da [id=98ee1a7044cd46d8b39dfda755ed8a7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.722062 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b427c7e897c64977b924e7d586420e69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.722086 524 master_service.cc:954] ReplaceTablet: received request to replace tablet 929eb48881b141718873d0e359e768a8 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:21.723402 523 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 79badde1001b432b95f58b5161ace15c
I20260202 02:52:21.727084 1180 ts_tablet_manager.cc:1929] T fe7b6004e5094c0dbe5847144a2489b9 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:21.727257 1180 log.cc:1199] T fe7b6004e5094c0dbe5847144a2489b9 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/fe7b6004e5094c0dbe5847144a2489b9
I20260202 02:52:21.727288 1656 raft_consensus.cc:515] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.727617 1180 ts_tablet_manager.cc:1950] T fe7b6004e5094c0dbe5847144a2489b9 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:21.728027 519 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_f52d623ce2cb43bf9598994d5298752f" } new_table_name: "default.Table_e8e9b4cdaf434b6cbbfed1a65bff1fc0" modify_external_catalogs: true
I20260202 02:52:21.728904 527 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 8095e0c6ff0041fc8b8c8b0778a20b00
I20260202 02:52:21.729131 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8095e0c6ff0041fc8b8c8b0778a20b00" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:21.729373 527 catalog_manager.cc:6724] ReplaceTablet: tablet 8095e0c6ff0041fc8b8c8b0778a20b00 deleted and replaced by tablet f5727f9b9bbb4e77abfef065c51606a1
I20260202 02:52:21.729713 527 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_935434504bf249f8842274aa67486d8c" } modify_external_catalogs: true
I20260202 02:52:21.729882 527 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_935434504bf249f8842274aa67486d8c" } modify_external_catalogs: true
I20260202 02:52:21.730549 521 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_f97ead9efdf047a2a4fc35c017853fb5 [id=1229c6d37b994a669d3363fdf6f8c6c8] alter complete (version 2)
I20260202 02:52:21.731408 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_93c8b5e956214e88abc353bfb46c8757"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:21.731917 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_93c8b5e956214e88abc353bfb46c8757 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:21.734005 526 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_11d7a6e6268042a7a26c8ecc00bc9f6e" } new_table_name: "default.Table_9913ba1a0b0c4f30bc3dd302ee61b92d" modify_external_catalogs: true
I20260202 02:52:21.735937 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_28e7d9aa8131413e9a174756c246d164"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:21.736719 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_28e7d9aa8131413e9a174756c246d164 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:21.736655 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_61741238f8ad47cbb8fe712419268ff0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:21.737464 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_61741238f8ad47cbb8fe712419268ff0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:21.737968 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_c2f3ac0894bd4f4f9702b6429da397bc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:21.738721 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c2f3ac0894bd4f4f9702b6429da397bc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:21.743773 1174 tablet_bootstrap.cc:492] T eaa47f3388144e8b852804c167f232ce P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:21.744103 1174 ts_tablet_manager.cc:1403] T eaa47f3388144e8b852804c167f232ce P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.028s user 0.004s sys 0.000s
I20260202 02:52:21.744316 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 268) took 2103 ms. Trace:
I20260202 02:52:21.748560 1180 rpcz_store.cc:276] 0202 02:52:19.641223 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.641312 (+ 89us) service_pool.cc:225] Handling call
0202 02:52:21.175727 (+1534415us) ts_tablet_manager.cc:1151] Deleting tablet fe7b6004e5094c0dbe5847144a2489b9
0202 02:52:21.175745 (+ 18us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:21.687758 (+512013us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:21.726942 (+ 39184us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:21.744251 (+ 17309us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1491857}
I20260202 02:52:21.750053 1180 tablet_replica.cc:333] T 723921dfe54c4e66af5cab76a25fee0a P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:21.750960 1180 raft_consensus.cc:2243] T 723921dfe54c4e66af5cab76a25fee0a P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:21.756775 1180 raft_consensus.cc:2272] T 723921dfe54c4e66af5cab76a25fee0a P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:21.758673 1180 ts_tablet_manager.cc:1916] T 723921dfe54c4e66af5cab76a25fee0a P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:21.758765 524 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 929eb48881b141718873d0e359e768a8
W20260202 02:52:21.758888 519 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ef073880898049e2be93d2cb6dc0c0bb consensus state has no leader
I20260202 02:52:21.759279 524 catalog_manager.cc:6724] ReplaceTablet: tablet 929eb48881b141718873d0e359e768a8 deleted and replaced by tablet cd1b6662da254889ae4bbe8d0b1ef1bb
I20260202 02:52:21.759404 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_783c68f0f6ad41899a919edefc2e85df"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:21.759627 524 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_8cb3cbab1a654f88a07d6d480fc72e52" } new_table_name: "default.Table_755b6ce0796b45b8be9df04e59dfe565" modify_external_catalogs: true
W20260202 02:52:21.760092 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_783c68f0f6ad41899a919edefc2e85df in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:21.764405 1656 leader_election.cc:290] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:21.764421 859 leader_election.cc:343] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:21.764720 527 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet fe53a8bdbd13417f9ce195c54a52a93c
W20260202 02:52:21.767557 526 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 874ca4ba4ebc4721bb4b60cba8e01841 consensus state has no leader
I20260202 02:52:21.771283 892 tablet_service.cc:1505] Processing CreateTablet for tablet e2edc6660b9a41e39e9529627fafcb03 (DEFAULT_TABLE table=default.Table_9d4fbf619567432a88aa9a0c1b19cb3e [id=fff85ebd1bd1415c932f7c9c5a0da42a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.771747 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2edc6660b9a41e39e9529627fafcb03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.771905 891 tablet_service.cc:1505] Processing CreateTablet for tablet 59ad016e15ec4a488921eb564029b689 (DEFAULT_TABLE table=default.Table_49cb996504bf45e9a24c6f3c01f457ff [id=ec1d1bc3e38643f7ad1c57e98ef66388]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.772236 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59ad016e15ec4a488921eb564029b689. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.780251 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_67001aa310ae44c0bb9e2b24bb0b57b8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:21.781301 521 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_246db50484924691aa609c499b9ef15b" } new_table_name: "default.Table_6e6ab095d9a6497d8a89bcc865b6f42f" modify_external_catalogs: true
I20260202 02:52:21.782785 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_3dcd1dea44ed4e54a89f4413e7f17dbe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:21.784108 898 tablet_service.cc:1505] Processing CreateTablet for tablet 7ea7a5d8a92b4ac0903483627258ae66 (DEFAULT_TABLE table=default.Table_d040708acb92480496bac8b012890006 [id=1c26d457c06345b7abeb1941bcee6c31]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:21.784968 512 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 70306ed9dec54286885d965f2c012da4 (table default.Table_d0e51ff8e83f4da19ce3c119dbec1ea7 [id=31abb0facd164caaa15591259eb951c2], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 70306ed9dec54286885d965f2c012da4 consensus state has no leader
W20260202 02:52:21.785080 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3dcd1dea44ed4e54a89f4413e7f17dbe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:21.785243 525 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_3919d090261e4b90b0672f2b863588b2 [id=fe7b88b74c864c2f98365c0956117bf1] alter complete (version 1)
W20260202 02:52:21.786339 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_67001aa310ae44c0bb9e2b24bb0b57b8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:21.787810 526 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_9dfc2d7b50e94228b7aad2a3f575f625" } new_table_name: "default.Table_6d32a9b50b2644b1a5b287ce8882ebab" modify_external_catalogs: true
I20260202 02:52:21.790302 889 tablet_service.cc:1505] Processing CreateTablet for tablet 07580ad1d049475a9645acfad0b1d219 (DEFAULT_TABLE table=default.Table_444485558ccc4c6ba771ab686bc9e1c1 [id=c4207a758ebb4784a67c662cfa4333d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.791466 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59ad016e15ec4a488921eb564029b689. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.792646 1039 tablet_service.cc:1552] Processing DeleteTablet for tablet 6df3d11cf578404ca41807f0777b6c53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:21.795074 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ea7a5d8a92b4ac0903483627258ae66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.795651 894 tablet_service.cc:1505] Processing CreateTablet for tablet f3eebefa74584c1791d52377f5ba1290 (DEFAULT_TABLE table=default.Table_80e6b91d5978450dbfd9bb2b4b410d3c [id=e0cea4989f66443faf074eacf434658c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.796207 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8095e0c6ff0041fc8b8c8b0778a20b00" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:21.796471 775 raft_consensus.cc:3060] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:21.797410 888 tablet_service.cc:1505] Processing CreateTablet for tablet 99e0b5e9a9874aec886155e0d614dec2 (DEFAULT_TABLE table=default.Table_a8c4f67a58b74bf8bb3716174ed5654f [id=e7975854ed2f4810a2d438b590267584]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.800056 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07580ad1d049475a9645acfad0b1d219. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.800572 1277 raft_consensus.cc:493] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:21.800770 1277 raft_consensus.cc:515] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.801658 1277 leader_election.cc:290] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:21.747130 1174 raft_consensus.cc:359] T eaa47f3388144e8b852804c167f232ce P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.803558 1174 raft_consensus.cc:385] T eaa47f3388144e8b852804c167f232ce P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:21.803660 1174 raft_consensus.cc:740] T eaa47f3388144e8b852804c167f232ce P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:21.804095 528 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_662111850b514df5ad7c69aa21783397" } modify_external_catalogs: true
I20260202 02:52:21.804414 528 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_662111850b514df5ad7c69aa21783397" } modify_external_catalogs: true
I20260202 02:52:21.804502 520 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_d4ad0ff5676e4af2a187497eda81d7a2" } modify_external_catalogs: true
I20260202 02:52:21.804702 520 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_d4ad0ff5676e4af2a187497eda81d7a2" } modify_external_catalogs: true
I20260202 02:52:21.803952 1174 consensus_queue.cc:260] T eaa47f3388144e8b852804c167f232ce P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.806356 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99e0b5e9a9874aec886155e0d614dec2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.806445 1174 ts_tablet_manager.cc:1434] T eaa47f3388144e8b852804c167f232ce P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.062s user 0.002s sys 0.000s
I20260202 02:52:21.806592 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3eebefa74584c1791d52377f5ba1290. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:21.808203 712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260202 02:52:21.812740 714 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 141528us
W20260202 02:52:21.813192 714 leader_election.cc:343] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 267bd4752c5f4b99a3fe778ba8f38a2f
I20260202 02:52:21.813473 714 leader_election.cc:304] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:21.813973 714 leader_election.cc:304] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:21.814406 714 leader_election.cc:304] T 70306ed9dec54286885d965f2c012da4 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:21.814719 1600 raft_consensus.cc:2749] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:21.814913 1600 raft_consensus.cc:2749] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:21.815106 1600 raft_consensus.cc:2804] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:21.815203 1600 raft_consensus.cc:493] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:21.815286 1600 raft_consensus.cc:3060] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:21.817606 1174 tablet_bootstrap.cc:492] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:21.818989 1600 raft_consensus.cc:515] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.820106 1174 tablet_bootstrap.cc:654] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:21.820158 1600 leader_election.cc:290] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:21.820448 1277 raft_consensus.cc:2804] T 70306ed9dec54286885d965f2c012da4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:21.820612 1277 raft_consensus.cc:493] T 70306ed9dec54286885d965f2c012da4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:21.820801 1277 raft_consensus.cc:3060] T 70306ed9dec54286885d965f2c012da4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:21.823827 1277 raft_consensus.cc:515] T 70306ed9dec54286885d965f2c012da4 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.826273 1175 tablet_bootstrap.cc:492] T 1f54c1dfe0a942b3bafd85522953a032 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:21.826534 1175 ts_tablet_manager.cc:1403] T 1f54c1dfe0a942b3bafd85522953a032 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.214s user 0.017s sys 0.034s
I20260202 02:52:21.830888 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 415d14533fd54f20adb00455a7bcad5f (table default.Table_22489ecf56264d93993407ba7d0e6b55 [id=7c19fad4b9554c0682a01bbd3f73d198]) successfully deleted
W20260202 02:52:21.831367 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 0186572e7bb9499a932a636ab280f544 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0186572e7bb9499a932a636ab280f544 already in progress: creating tablet
W20260202 02:52:21.831899 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 13cd1383348b458894fdecc22bf24b74 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 13cd1383348b458894fdecc22bf24b74 already in progress: creating tablet
W20260202 02:52:21.832114 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 8774c6bc0fe9414ea2f17ed859d9392a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8774c6bc0fe9414ea2f17ed859d9392a already in progress: creating tablet
I20260202 02:52:21.832101 890 tablet_service.cc:1552] Processing DeleteTablet for tablet 8095e0c6ff0041fc8b8c8b0778a20b00 with delete_type TABLET_DATA_DELETED (replaced by tablet f5727f9b9bbb4e77abfef065c51606a1) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:21.832369 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 2f0e8c5e2fd44370ae150a206f4e6cbc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2f0e8c5e2fd44370ae150a206f4e6cbc already in progress: creating tablet
I20260202 02:52:21.832554 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 75f7824681344d8d88df9402694fff1b (table default.Table_ae49654353a64141b6d7f0e6afe13d9d [id=431440e879c04f0f9ad63c64d8741b56]) successfully deleted
W20260202 02:52:21.836899 999 leader_election.cc:343] T fe7b6004e5094c0dbe5847144a2489b9 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: fe7b6004e5094c0dbe5847144a2489b9
I20260202 02:52:21.837800 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet fe7b6004e5094c0dbe5847144a2489b9 (table default.Table_a6b938c21e164294bd8b5fb205673f75 [id=1abad082d47d4f31be7c697ed72e241c]) successfully deleted
W20260202 02:52:21.842160 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 6df3d11cf578404ca41807f0777b6c53 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6df3d11cf578404ca41807f0777b6c53 already in progress: creating tablet
W20260202 02:52:21.842687 998 leader_election.cc:343] T fe7b6004e5094c0dbe5847144a2489b9 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Not found: Tablet not found: fe7b6004e5094c0dbe5847144a2489b9
I20260202 02:52:21.843142 998 leader_election.cc:304] T fe7b6004e5094c0dbe5847144a2489b9 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:21.843468 1039 tablet_service.cc:1552] Processing DeleteTablet for tablet 79badde1001b432b95f58b5161ace15c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:21.843596 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 6df3d11cf578404ca41807f0777b6c53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:21.845811 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet 929eb48881b141718873d0e359e768a8 with delete_type TABLET_DATA_DELETED (replaced by tablet cd1b6662da254889ae4bbe8d0b1ef1bb) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:21.846063 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 929eb48881b141718873d0e359e768a8 with delete_type TABLET_DATA_DELETED (replaced by tablet cd1b6662da254889ae4bbe8d0b1ef1bb) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:21.846309 1656 raft_consensus.cc:493] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260202 02:52:21.846832 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 79badde1001b432b95f58b5161ace15c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 79badde1001b432b95f58b5161ace15c already in progress: creating tablet
W20260202 02:52:21.848682 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 929eb48881b141718873d0e359e768a8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 929eb48881b141718873d0e359e768a8 already in progress: creating tablet
I20260202 02:52:21.848901 1656 raft_consensus.cc:515] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.849610 1656 leader_election.cc:290] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:21.851128 892 tablet_service.cc:1552] Processing DeleteTablet for tablet 79badde1001b432b95f58b5161ace15c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:21.854836 526 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 16b47a2831f54d809ed0b16deaec7e3d consensus state has no leader
W20260202 02:52:21.855577 521 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5e20842de8cb4355b732685605c88834 has no consensus state
I20260202 02:52:21.860432 1037 tablet_service.cc:1552] Processing DeleteTablet for tablet 8095e0c6ff0041fc8b8c8b0778a20b00 with delete_type TABLET_DATA_DELETED (replaced by tablet f5727f9b9bbb4e77abfef065c51606a1) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:21.861435 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4e79cd48fefb4ce29ef48beacfc4d26f" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:21.861862 1182 ts_tablet_manager.cc:1929] T 526f2c7b95184a0987e7aa94efc8475a P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:21.862026 1182 log.cc:1199] T 526f2c7b95184a0987e7aa94efc8475a P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/526f2c7b95184a0987e7aa94efc8475a
I20260202 02:52:21.862560 884 tablet_service.cc:1552] Processing DeleteTablet for tablet fe53a8bdbd13417f9ce195c54a52a93c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:21 UTC) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:21.863126 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 8095e0c6ff0041fc8b8c8b0778a20b00 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8095e0c6ff0041fc8b8c8b0778a20b00 already in progress: creating tablet
W20260202 02:52:21.863402 859 leader_election.cc:343] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:21.865579 917 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "78f122342dd144b48de89902faa0f94f" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:21.868927 917 raft_consensus.cc:2468] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:21.872025 1037 tablet_service.cc:1505] Processing CreateTablet for tablet f1c8f7e1dc9644a6b6dd2bc264cce8ef (DEFAULT_TABLE table=default.Table_06273452ab3e4d2e8bdb1e3a41721dcc [id=4096475017214ffdba27d20aca4e8d64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.872519 1039 tablet_service.cc:1505] Processing CreateTablet for tablet 25a407001f964da6b51f42375136c79a (DEFAULT_TABLE table=default.Table_48226e7c77da41cdba56e04221eb9689 [id=05e52ce8e94e4975b51849059b904272]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.876416 1173 tablet_bootstrap.cc:492] T 8095e0c6ff0041fc8b8c8b0778a20b00 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:21.876638 1173 ts_tablet_manager.cc:1403] T 8095e0c6ff0041fc8b8c8b0778a20b00 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.222s user 0.012s sys 0.016s
I20260202 02:52:21.878355 1182 ts_tablet_manager.cc:1950] T 526f2c7b95184a0987e7aa94efc8475a P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:21.890079 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 247) took 2167 ms. Trace:
I20260202 02:52:21.896662 1182 rpcz_store.cc:276] 0202 02:52:19.717912 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.717955 (+ 43us) service_pool.cc:225] Handling call
0202 02:52:21.684494 (+1966539us) ts_tablet_manager.cc:1151] Deleting tablet 526f2c7b95184a0987e7aa94efc8475a
0202 02:52:21.684526 (+ 32us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:21.755057 (+ 70531us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:21.850377 (+ 95320us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:21.884835 (+ 34458us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1966334}
I20260202 02:52:21.897123 1182 tablet_replica.cc:333] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:21.898082 921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "80d443bdb66d4cc68ae283e08c410fe6" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:21.898047 1175 raft_consensus.cc:359] T 1f54c1dfe0a942b3bafd85522953a032 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.898247 1175 raft_consensus.cc:385] T 1f54c1dfe0a942b3bafd85522953a032 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:21.898308 921 raft_consensus.cc:2393] T 80d443bdb66d4cc68ae283e08c410fe6 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f442ef85029461091de847118c34c0d in current term 1: Already voted for candidate 1521a6d262b7451bbef8debe970dd6b5 in this term.
I20260202 02:52:21.898852 1175 raft_consensus.cc:740] T 1f54c1dfe0a942b3bafd85522953a032 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:21.899138 1175 consensus_queue.cc:260] T 1f54c1dfe0a942b3bafd85522953a032 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.901907 1708 tablet.cc:1722] T 80eb3013f5354022a2dd78bb294a8497 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260202 02:52:21.902140 1277 leader_election.cc:290] T 70306ed9dec54286885d965f2c012da4 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:21.903218 1175 ts_tablet_manager.cc:1434] T 1f54c1dfe0a942b3bafd85522953a032 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.073s user 0.001s sys 0.001s
I20260202 02:52:21.903268 921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b428356fba0c4fb5a6033c5d39740cc5" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:21.903458 921 raft_consensus.cc:3060] T b428356fba0c4fb5a6033c5d39740cc5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:21.901896 775 raft_consensus.cc:2468] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
I20260202 02:52:21.904120 1175 tablet_bootstrap.cc:492] T d21fca9a5b234c26879717f51c8ea0f0 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:21.904935 1030 tablet_service.cc:1552] Processing DeleteTablet for tablet fe53a8bdbd13417f9ce195c54a52a93c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:21 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:21.905033 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f58a7b25815c4b5bac568aa11c19610c" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:21.888365 999 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 58462.5us
I20260202 02:52:21.906270 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 526f2c7b95184a0987e7aa94efc8475a (table default.Table_0842fd6a82e145e7a98b3fed2c0c6993 [id=4e076b2023be4c888ac1c27babde1607]) successfully deleted
I20260202 02:52:21.906626 1178 raft_consensus.cc:2749] T fe7b6004e5094c0dbe5847144a2489b9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:21.907272 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4e79cd48fefb4ce29ef48beacfc4d26f" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:21.908171 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet fe53a8bdbd13417f9ce195c54a52a93c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fe53a8bdbd13417f9ce195c54a52a93c already in progress: creating tablet
I20260202 02:52:21.908802 1062 raft_consensus.cc:2468] T f58a7b25815c4b5bac568aa11c19610c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:21.909301 1723 raft_consensus.cc:493] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:21.909813 750 tablet_service.cc:1552] Processing DeleteTablet for tablet d30557fbe9084cd3860820f69dda8752 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:21.910228 756 tablet_service.cc:1505] Processing CreateTablet for tablet d0ce2121d65b4a228596e0ca9018883d (DEFAULT_TABLE table=default.Table_8fcc3828b43444159f0df6023ec1ba0f [id=ca632e6d14ca41e2ab9444dc362a4265]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.910636 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0ce2121d65b4a228596e0ca9018883d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.911201 1721 raft_consensus.cc:493] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260202 02:52:21.910180 1059 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.606s user 0.015s sys 0.043s
I20260202 02:52:21.911459 1178 raft_consensus.cc:493] T c12ff20ca6b44ddba0c936a28cfb2eb9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:21.911931 751 tablet_service.cc:1505] Processing CreateTablet for tablet 587cfd0ae62f4cfd937faf6890e78f4a (DEFAULT_TABLE table=default.Table_2be7fcc0f92846059283a7691fcda407 [id=9f2441734085417fb56dc05a42df54db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.909723 759 tablet_service.cc:1505] Processing CreateTablet for tablet 1d8143f529b74e6ab779dd695920413c (DEFAULT_TABLE table=default.Table_33fc32ec03094e3c8fe78de4f6a26777 [id=85340c1716bd4235bdad7042b53ce58c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.913031 747 tablet_service.cc:1505] Processing CreateTablet for tablet 1f379a29e2454f689d68f5951837512b (DEFAULT_TABLE table=default.Table_67c55c53ca254b03b0c326672a41b559 [id=7e80e30271434635a17d0034580aeca8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.913076 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "70306ed9dec54286885d965f2c012da4" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:21.913401 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b428356fba0c4fb5a6033c5d39740cc5" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:21.913651 1036 tablet_service.cc:1505] Processing CreateTablet for tablet 2616e1bbce4049dbb584550ccc150288 (DEFAULT_TABLE table=default.Table_4fb44d5cbe71439ba073bd75711e77e0 [id=8c2d0f27f5d04a98bfad01483c783ecb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.913995 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1c8f7e1dc9644a6b6dd2bc264cce8ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.914499 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25a407001f964da6b51f42375136c79a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.915123 1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2616e1bbce4049dbb584550ccc150288. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.916150 755 tablet_service.cc:1505] Processing CreateTablet for tablet 44c8d5b74c8e4c499f7f68e8185efc17 (DEFAULT_TABLE table=default.Table_668e8ec9913f43fa8499eff32be8db0a [id=46b5d47377474ff9bac2acdb26b08c65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.916118 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 7a44585a217b48f39f247bae3ca2e255 (DEFAULT_TABLE table=default.Table_2ee0b536f96e44e4a3a4a9439c7bccc9 [id=d95486b11c494863be7380146ebcf0c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.916393 1178 raft_consensus.cc:515] T c12ff20ca6b44ddba0c936a28cfb2eb9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.916558 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a44585a217b48f39f247bae3ca2e255. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.918061 1723 raft_consensus.cc:515] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.918147 1721 raft_consensus.cc:515] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:21.918880 1723 leader_election.cc:290] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:21.919870 1721 leader_election.cc:290] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:21.920012 1059 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.671s user 0.015s sys 0.043s
I20260202 02:52:21.920246 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d8143f529b74e6ab779dd695920413c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.923959 1182 raft_consensus.cc:2243] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:21.924453 1182 raft_consensus.cc:2272] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:21.925940 526 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_49cb996504bf45e9a24c6f3c01f457ff" } modify_external_catalogs: true
I20260202 02:52:21.926152 526 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_49cb996504bf45e9a24c6f3c01f457ff" } modify_external_catalogs: true
I20260202 02:52:21.927551 521 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_3d26f7a34d0c48379e3cebda66e1ef13" } modify_external_catalogs: true
I20260202 02:52:21.927711 521 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_3d26f7a34d0c48379e3cebda66e1ef13" } modify_external_catalogs: true
I20260202 02:52:21.928618 1173 raft_consensus.cc:359] T 8095e0c6ff0041fc8b8c8b0778a20b00 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.928864 1173 raft_consensus.cc:385] T 8095e0c6ff0041fc8b8c8b0778a20b00 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:21.929522 750 tablet_service.cc:1505] Processing CreateTablet for tablet 0138ba7326674741b0729324735f1de7 (DEFAULT_TABLE table=default.Table_516d610640654ea4b49f7de3b04cbb0c [id=5c8367fbb78840d18a26a026d75c3d6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.929998 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bc1539e5c8aa4d6e98f687728b891530" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:21.934332 527 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet d06b77ff1a6f4f78b73082b00145fcf6 (table default.Table_63e36ff851f849849172c3c8e473214e [id=915b211258ad498286448a5b2598a289]). Expected version 3 got 0
I20260202 02:52:21.936229 1178 leader_election.cc:290] T c12ff20ca6b44ddba0c936a28cfb2eb9 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:21.936265 1173 raft_consensus.cc:740] T 8095e0c6ff0041fc8b8c8b0778a20b00 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:21.938066 1173 consensus_queue.cc:260] T 8095e0c6ff0041fc8b8c8b0778a20b00 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:21.938696 777 raft_consensus.cc:2468] T bc1539e5c8aa4d6e98f687728b891530 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:21.940598 757 tablet_service.cc:1505] Processing CreateTablet for tablet ac2ab12f5040488aa306cb69b107b485 (DEFAULT_TABLE table=default.Table_35c5c99502d0489081751bc7403f6ad7 [id=640ad152bb4243cda5a7e7242996e2fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.942035 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f379a29e2454f689d68f5951837512b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.942704 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_cc8d0231630743f180159c4d3a4efab8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:21.942970 998 leader_election.cc:304] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
W20260202 02:52:21.943290 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cc8d0231630743f180159c4d3a4efab8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:21.944022 1173 ts_tablet_manager.cc:1434] T 8095e0c6ff0041fc8b8c8b0778a20b00 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.066s user 0.000s sys 0.003s
I20260202 02:52:21.944450 1173 tablet_bootstrap.cc:492] T b428356fba0c4fb5a6033c5d39740cc5 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:21.946074 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_eeb507a1a49d407196bff72270b0834f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:21.946628 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eeb507a1a49d407196bff72270b0834f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:21.952226 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c12ff20ca6b44ddba0c936a28cfb2eb9" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:21.953474 749 tablet_service.cc:1505] Processing CreateTablet for tablet 939fbcaf75c7439d9208bbbf83a72717 (DEFAULT_TABLE table=default.Table_12d68a317c8e4d809dbbe77c095a1f6e [id=7219522ffc3849faab94d754a42a8511]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.953900 754 tablet_service.cc:1505] Processing CreateTablet for tablet 65bd1c1a382e42e4a68077264563c55b (DEFAULT_TABLE table=default.Table_1cc0209a40ad471fb1c41012a476df79 [id=b941bc2edc03444394d1ba8a596a0d27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.954413 748 tablet_service.cc:1505] Processing CreateTablet for tablet 8b0368e03f644f1db28dbb57c95c48c1 (DEFAULT_TABLE table=default.Table_f93837f1f1614be4bb992e92e5237ea2 [id=8882e1fbac2949838c99adb1290f273e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.955193 1178 raft_consensus.cc:2804] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:21.955752 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44c8d5b74c8e4c499f7f68e8185efc17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.956899 1178 raft_consensus.cc:493] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:21.960336 1173 tablet_bootstrap.cc:654] T b428356fba0c4fb5a6033c5d39740cc5 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:21.960388 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65bd1c1a382e42e4a68077264563c55b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.961978 1178 raft_consensus.cc:3060] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:21.961990 524 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ba9c16aaa48849db9fece59168c16886" } new_table_name: "default.Table_5db99844c1bb484a863a9e289a19da3e" modify_external_catalogs: true
I20260202 02:52:21.965427 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_89983dadd3fc4966bf26418544c1729a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:21.966022 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_89983dadd3fc4966bf26418544c1729a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:21.970745 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac2ab12f5040488aa306cb69b107b485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.970682 1178 raft_consensus.cc:515] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:21.972247 1178 leader_election.cc:290] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:21.974829 752 tablet_service.cc:1505] Processing CreateTablet for tablet 93f958924885457ab15f2aac5509e0aa (DEFAULT_TABLE table=default.Table_2e2049e67b8943acab5a0db3deb1535a [id=0db38635f6af4f168f221f4749adf0dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.974817 902 tablet_service.cc:1505] Processing CreateTablet for tablet f1c8f7e1dc9644a6b6dd2bc264cce8ef (DEFAULT_TABLE table=default.Table_06273452ab3e4d2e8bdb1e3a41721dcc [id=4096475017214ffdba27d20aca4e8d64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.975068 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0138ba7326674741b0729324735f1de7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.975533 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1c8f7e1dc9644a6b6dd2bc264cce8ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.975508 1182 ts_tablet_manager.cc:1916] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:21.976992 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 587cfd0ae62f4cfd937faf6890e78f4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.979307 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 51ba0bcd498043c383c493f45ad1fe17 (DEFAULT_TABLE table=default.Table_619d2fcf726a46bfb5e939bbe6f55612 [id=038692c44a0043bab69ab29f77aa13be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.980494 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51ba0bcd498043c383c493f45ad1fe17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.981707 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 939fbcaf75c7439d9208bbbf83a72717. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.981150 758 tablet_service.cc:1505] Processing CreateTablet for tablet 7709ea34429e4e91ad4922a3d73ebd14 (DEFAULT_TABLE table=default.Table_8262b850970641b1bbe1cf9d1c3d9409 [id=c6e5418bf48e43e2adb6d1b60445f0b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.983129 746 tablet_service.cc:1505] Processing CreateTablet for tablet 2b352d9a815e4bccafb7e2251500c5bc (DEFAULT_TABLE table=default.Table_408b8f21ff294ce9880d0b9a4d681f93 [id=c0fa167c7d1f47ea994fcf7dfdd4100f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.984841 753 tablet_service.cc:1505] Processing CreateTablet for tablet da5435f969074364bc8f8a6ddc5e4f8a (DEFAULT_TABLE table=default.Table_51648e2a42614c3d9c5820901d153932 [id=ad256e6219784dff9aa916cf07f67fbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.985116 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bc1539e5c8aa4d6e98f687728b891530" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:21.990711 893 tablet_service.cc:1505] Processing CreateTablet for tablet 2616e1bbce4049dbb584550ccc150288 (DEFAULT_TABLE table=default.Table_4fb44d5cbe71439ba073bd75711e77e0 [id=8c2d0f27f5d04a98bfad01483c783ecb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.991331 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93f958924885457ab15f2aac5509e0aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:21.993896 775 raft_consensus.cc:3060] T bc1539e5c8aa4d6e98f687728b891530 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:21.996234 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da5435f969074364bc8f8a6ddc5e4f8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.000891 886 tablet_service.cc:1505] Processing CreateTablet for tablet 25a407001f964da6b51f42375136c79a (DEFAULT_TABLE table=default.Table_48226e7c77da41cdba56e04221eb9689 [id=05e52ce8e94e4975b51849059b904272]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:21.999827 745 tablet_service.cc:1505] Processing CreateTablet for tablet 79badde1001b432b95f58b5161ace15c (DEFAULT_TABLE table=default.Table_80912cc76f934f139da7f866546ae97a [id=2f3bf7e0cf4c41ac8a2554360b57214a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.003340 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7709ea34429e4e91ad4922a3d73ebd14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.005059 899 tablet_service.cc:1505] Processing CreateTablet for tablet 7a44585a217b48f39f247bae3ca2e255 (DEFAULT_TABLE table=default.Table_2ee0b536f96e44e4a3a4a9439c7bccc9 [id=d95486b11c494863be7380146ebcf0c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.006125 1723 raft_consensus.cc:493] T eaa47f3388144e8b852804c167f232ce P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:22.005800 741 tablet_service.cc:1505] Processing CreateTablet for tablet 2f5aae089fcc492f9ce55f84674a452c (DEFAULT_TABLE table=default.Table_84d6693620cc468e82d5cb1ec6cc69ea [id=f6625b21b6064828b11acdfdb694cd71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.008145 740 tablet_service.cc:1505] Processing CreateTablet for tablet 75a24a1bf54d4c799938a5723ac6fa99 (DEFAULT_TABLE table=default.Table_52a2f389faba40fea93e2bca972e81e1 [id=70ac373200604907b00132eb4727ea5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.010632 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79badde1001b432b95f58b5161ace15c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.010790 1723 raft_consensus.cc:515] T eaa47f3388144e8b852804c167f232ce P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.013963 1723 leader_election.cc:290] T eaa47f3388144e8b852804c167f232ce P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:22.008775 742 tablet_service.cc:1505] Processing CreateTablet for tablet 3284213e4b0447b49a7718ba896d3fa8 (DEFAULT_TABLE table=default.Table_e71a7347855d447cabe6ea2cc64b293a [id=e490dd882e8e4e51ad5a2073c891ba91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.016374 743 tablet_service.cc:1505] Processing CreateTablet for tablet 7ea7a5d8a92b4ac0903483627258ae66 (DEFAULT_TABLE table=default.Table_d040708acb92480496bac8b012890006 [id=1c26d457c06345b7abeb1941bcee6c31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.025264 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f5aae089fcc492f9ce55f84674a452c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.027119 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2616e1bbce4049dbb584550ccc150288. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.033943 894 tablet_service.cc:1505] Processing CreateTablet for tablet 51ba0bcd498043c383c493f45ad1fe17 (DEFAULT_TABLE table=default.Table_619d2fcf726a46bfb5e939bbe6f55612 [id=038692c44a0043bab69ab29f77aa13be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.034444 1175 tablet_bootstrap.cc:654] T d21fca9a5b234c26879717f51c8ea0f0 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:22.042446 1029 tablet_service.cc:1505] Processing CreateTablet for tablet dc90eeb79ded4739883ad98928476680 (DEFAULT_TABLE table=default.Table_a9b816e5b5b84151b840bfad315deb70 [id=b46bd49d8fe14af094e69597521a1a22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.044449 1039 tablet_service.cc:1505] Processing CreateTablet for tablet 7ff6d2fa085343a6b11be0898da6233d (DEFAULT_TABLE table=default.Table_bcf2aa761e104360bb4a1251204a6608 [id=5eaf1a210fa44066b81bb5b01db4d7cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.046870 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc90eeb79ded4739883ad98928476680. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.046994 921 raft_consensus.cc:2468] T b428356fba0c4fb5a6033c5d39740cc5 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:22.048120 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ff6d2fa085343a6b11be0898da6233d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.050580 1173 tablet_bootstrap.cc:492] T b428356fba0c4fb5a6033c5d39740cc5 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:22.052546 775 raft_consensus.cc:2468] T bc1539e5c8aa4d6e98f687728b891530 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:22.053511 998 leader_election.cc:304] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:22.053905 1173 ts_tablet_manager.cc:1403] T b428356fba0c4fb5a6033c5d39740cc5 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.106s user 0.009s sys 0.005s
I20260202 02:52:22.054337 1178 raft_consensus.cc:2804] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:22.054998 1178 raft_consensus.cc:697] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:22.055307 1178 consensus_queue.cc:237] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:22.055651 1173 raft_consensus.cc:359] T b428356fba0c4fb5a6033c5d39740cc5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.043094 744 tablet_service.cc:1505] Processing CreateTablet for tablet c8a273ac8cd344719a48fd81945bed0a (DEFAULT_TABLE table=default.Table_0601f3c5b49446b3a7ec44443f89a38c [id=62c3c39fb73249349cee49b86d535612]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.060425 1173 raft_consensus.cc:385] T b428356fba0c4fb5a6033c5d39740cc5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.060576 1173 raft_consensus.cc:740] T b428356fba0c4fb5a6033c5d39740cc5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.060814 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b352d9a815e4bccafb7e2251500c5bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.058530 1708 tablet.cc:1722] T 583e547d0a1943f09f4efe82070fc777 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:22.062594 1173 consensus_queue.cc:260] T b428356fba0c4fb5a6033c5d39740cc5 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.066521 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b0368e03f644f1db28dbb57c95c48c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.072132 715 leader_election.cc:304] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
W20260202 02:52:22.072577 715 leader_election.cc:343] T 70306ed9dec54286885d965f2c012da4 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:22.073643 715 leader_election.cc:343] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:22.075073 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 528d4269f91a44679acfad16e9fbbe32 (DEFAULT_TABLE table=default.Table_c6515f88263042feadae47be333ed860 [id=c2c454eb645044839fa81b33b6460558]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.076115 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3284213e4b0447b49a7718ba896d3fa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.076961 1721 raft_consensus.cc:493] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:22.077023 1173 ts_tablet_manager.cc:1434] T b428356fba0c4fb5a6033c5d39740cc5 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.020s user 0.002s sys 0.002s
I20260202 02:52:22.077193 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 528d4269f91a44679acfad16e9fbbe32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.081799 1173 tablet_bootstrap.cc:492] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:22.082557 1059 raft_consensus.cc:3060] T d06b77ff1a6f4f78b73082b00145fcf6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:22.085606 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a44585a217b48f39f247bae3ca2e255. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.085649 1776 raft_consensus.cc:493] T 1f54c1dfe0a942b3bafd85522953a032 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:22.085809 1776 raft_consensus.cc:515] T 1f54c1dfe0a942b3bafd85522953a032 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.086561 1776 leader_election.cc:290] T 1f54c1dfe0a942b3bafd85522953a032 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:22.088677 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51ba0bcd498043c383c493f45ad1fe17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.089598 1721 raft_consensus.cc:515] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.090258 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8095e0c6ff0041fc8b8c8b0778a20b00" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:22.090968 1721 leader_election.cc:290] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:22.091301 1063 raft_consensus.cc:2468] T 8095e0c6ff0041fc8b8c8b0778a20b00 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:22.092090 1764 raft_consensus.cc:493] T 723921dfe54c4e66af5cab76a25fee0a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:22.093160 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "eaa47f3388144e8b852804c167f232ce" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:22.093228 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "267bd4752c5f4b99a3fe778ba8f38a2f" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:22.093442 775 raft_consensus.cc:2468] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:22.093843 998 leader_election.cc:304] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:22.094002 1723 raft_consensus.cc:2804] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:22.094184 1721 raft_consensus.cc:2804] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:22.094261 1721 raft_consensus.cc:493] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:22.094348 1721 raft_consensus.cc:3060] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:22.097414 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25a407001f964da6b51f42375136c79a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.097642 1723 raft_consensus.cc:493] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:22.097668 1721 raft_consensus.cc:515] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.097784 1723 raft_consensus.cc:3060] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:22.098986 1180 ts_tablet_manager.cc:1929] T 723921dfe54c4e66af5cab76a25fee0a P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:22.099324 1180 log.cc:1199] T 723921dfe54c4e66af5cab76a25fee0a P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/723921dfe54c4e66af5cab76a25fee0a
I20260202 02:52:22.099175 1721 leader_election.cc:290] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:22.099735 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "267bd4752c5f4b99a3fe778ba8f38a2f" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:22.099992 775 raft_consensus.cc:3060] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:22.103747 715 leader_election.cc:304] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
W20260202 02:52:22.104203 715 consensus_peers.cc:597] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c63a670d5c5e489897bc6dbb142b844d. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:22.104379 715 leader_election.cc:343] T eaa47f3388144e8b852804c167f232ce P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:22.104653 1277 raft_consensus.cc:2804] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260202 02:52:22.104753 1277 raft_consensus.cc:493] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:22.104851 1277 raft_consensus.cc:3060] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:22.103569 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 2f5aae089fcc492f9ce55f84674a452c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:22.107102 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 6154460838eb4e508f3646d3e3e4cf21 with delete_type TABLET_DATA_DELETED (replaced by tablet 5e20842de8cb4355b732685605c88834) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:22.107540 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 0d3633f5f9f54115bc78b9c1b78aba67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:22.110042 1182 ts_tablet_manager.cc:1929] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:22.111701 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75a24a1bf54d4c799938a5723ac6fa99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.113366 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8a273ac8cd344719a48fd81945bed0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.116925 743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ea7a5d8a92b4ac0903483627258ae66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.119202 1178 raft_consensus.cc:493] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:22.119401 1178 raft_consensus.cc:515] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:22.121063 1178 leader_election.cc:290] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:22.121354 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c184d44d7f434b39a997a7387e5d3de0" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:22.121574 774 raft_consensus.cc:2468] T c184d44d7f434b39a997a7387e5d3de0 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:22.120440 1781 tablet.cc:1722] T d06b77ff1a6f4f78b73082b00145fcf6 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260202 02:52:22.123600 1182 log.cc:1199] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/723921dfe54c4e66af5cab76a25fee0a
I20260202 02:52:22.125762 1032 tablet_service.cc:1505] Processing CreateTablet for tablet b4a4c75a906f40beb896bf6fff705ae6 (DEFAULT_TABLE table=default.Table_6806575803c4468fa7b7ef990004b446 [id=944fbe1acd894017bae3c54f135ee9b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.126329 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4a4c75a906f40beb896bf6fff705ae6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.126297 1656 raft_consensus.cc:493] T ef073880898049e2be93d2cb6dc0c0bb P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:22.084750 1769 tablet.cc:1722] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:22.127264 998 leader_election.cc:304] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:22.102825 1764 raft_consensus.cc:515] T 723921dfe54c4e66af5cab76a25fee0a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.129140 1783 tablet.cc:1722] T ff6f5710de4b4c48948aacdf5be29e8b P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:22.129644 1178 raft_consensus.cc:2804] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:22.129772 1173 tablet_bootstrap.cc:654] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:22.133114 1656 raft_consensus.cc:515] T ef073880898049e2be93d2cb6dc0c0bb P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.135099 1178 raft_consensus.cc:493] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:22.135393 1764 leader_election.cc:290] T 723921dfe54c4e66af5cab76a25fee0a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:22.135545 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f54c1dfe0a942b3bafd85522953a032" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:22.135885 753 tablet_service.cc:1505] Processing CreateTablet for tablet b427c7e897c64977b924e7d586420e69 (DEFAULT_TABLE table=default.Table_7d0cce2fe6094a54a4b51acf6b1d42da [id=98ee1a7044cd46d8b39dfda755ed8a7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.141983 1178 raft_consensus.cc:3060] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:22.142097 859 leader_election.cc:343] T 1f54c1dfe0a942b3bafd85522953a032 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:22.142691 1723 raft_consensus.cc:515] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.143687 1723 leader_election.cc:290] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:22.144760 1781 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:41736 (request call id 269) took 2174 ms. Trace:
I20260202 02:52:22.145167 1182 ts_tablet_manager.cc:1950] T 723921dfe54c4e66af5cab76a25fee0a P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:22.146240 756 tablet_service.cc:1505] Processing CreateTablet for tablet f3eebefa74584c1791d52377f5ba1290 (DEFAULT_TABLE table=default.Table_80e6b91d5978450dbfd9bb2b4b410d3c [id=e0cea4989f66443faf074eacf434658c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.149966 1277 raft_consensus.cc:515] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.151618 1277 leader_election.cc:290] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:22.152217 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f58a7b25815c4b5bac568aa11c19610c" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:22.152465 1062 raft_consensus.cc:3060] T f58a7b25815c4b5bac568aa11c19610c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:22.153026 1057 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ef073880898049e2be93d2cb6dc0c0bb" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:22.156705 1039 tablet_service.cc:1505] Processing CreateTablet for tablet f5727f9b9bbb4e77abfef065c51606a1 (DEFAULT_TABLE table=default.Table_4e6c3886f58b49698a00d45b957f4c9a [id=b5cc03b77f414304b29c5fb4bb7dc3ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.157743 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5727f9b9bbb4e77abfef065c51606a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.158416 752 tablet_service.cc:1505] Processing CreateTablet for tablet 07580ad1d049475a9645acfad0b1d219 (DEFAULT_TABLE table=default.Table_444485558ccc4c6ba771ab686bc9e1c1 [id=c4207a758ebb4784a67c662cfa4333d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.159600 1656 leader_election.cc:290] T ef073880898049e2be93d2cb6dc0c0bb P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:22.159979 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 258) took 2256 ms. Trace:
I20260202 02:52:22.160132 1182 rpcz_store.cc:276] 0202 02:52:19.900168 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.900242 (+ 74us) service_pool.cc:225] Handling call
0202 02:52:21.897038 (+1996796us) ts_tablet_manager.cc:1151] Deleting tablet 723921dfe54c4e66af5cab76a25fee0a
0202 02:52:21.897057 (+ 19us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.072865 (+175808us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.109898 (+ 37033us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.156205 (+ 46307us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1996348}
I20260202 02:52:22.170653 755 tablet_service.cc:1505] Processing CreateTablet for tablet e2edc6660b9a41e39e9529627fafcb03 (DEFAULT_TABLE table=default.Table_9d4fbf619567432a88aa9a0c1b19cb3e [id=fff85ebd1bd1415c932f7c9c5a0da42a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.173280 1062 raft_consensus.cc:2468] T f58a7b25815c4b5bac568aa11c19610c P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
W20260202 02:52:22.176595 859 leader_election.cc:343] T ef073880898049e2be93d2cb6dc0c0bb P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:22.165776 1781 rpcz_store.cc:276] 0202 02:52:19.970496 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.970563 (+ 67us) service_pool.cc:225] Handling call
0202 02:52:22.144723 (+2174160us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:20.173935 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0202 02:52:20.174023 (+ 88us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0202 02:52:20.174031 (+ 8us) alter_schema_op.cc:66] Acquired schema lock
0202 02:52:20.174037 (+ 6us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0202 02:52:20.174068 (+ 31us) alter_schema_op.cc:128] START. Timestamp: 1770000740174029
0202 02:52:20.174071 (+ 3us) op_driver.cc:348] REPLICATION: starting
0202 02:52:20.174205 (+ 134us) log.cc:844] Serialized 173 byte log entry
0202 02:52:22.018966 (+1844761us) op_driver.cc:464] REPLICATION: finished
0202 02:52:22.069008 (+ 50042us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0202 02:52:22.122949 (+ 53941us) log.cc:844] Serialized 5 byte log entry
0202 02:52:22.143793 (+ 20844us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.144109 (+ 316us) log.cc:844] Serialized 14 byte log entry
0202 02:52:22.144398 (+ 289us) alter_schema_op.cc:73] Released schema lock
0202 02:52:22.144403 (+ 5us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":49674,"prepare.queue_time_us":201965,"prepare.run_cpu_time_us":14574,"prepare.run_wall_time_us":79402,"raft.queue_time_us":6388,"raft.run_cpu_time_us":453,"raft.run_wall_time_us":5443,"replication_time_us":1844823,"thread_start_us":445490,"threads_started":3}]]}
I20260202 02:52:22.178262 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 259) took 2263 ms. Trace:
I20260202 02:52:22.178450 1182 rpcz_store.cc:276] 0202 02:52:19.901772 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.903708 (+ 1936us) service_pool.cc:225] Handling call
0202 02:52:22.164636 (+2260928us) ts_tablet_manager.cc:1151] Deleting tablet 94bada52db6a47179415d302b541d01f
0202 02:52:22.164646 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.165571 (+ 925us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2255983}
I20260202 02:52:22.180112 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 261) took 2264 ms. Trace:
I20260202 02:52:22.180176 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b427c7e897c64977b924e7d586420e69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.180295 1182 rpcz_store.cc:276] 0202 02:52:19.914219 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.914366 (+ 147us) service_pool.cc:225] Handling call
0202 02:52:22.178780 (+2264414us) ts_tablet_manager.cc:1151] Deleting tablet 59990791123c4d4eb620090a1e8e31e1
0202 02:52:22.178794 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.178923 (+ 129us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2259841}
I20260202 02:52:22.180979 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 260) took 2275 ms. Trace:
I20260202 02:52:22.181146 1182 rpcz_store.cc:276] 0202 02:52:19.905314 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.905381 (+ 67us) service_pool.cc:225] Handling call
0202 02:52:22.180655 (+2275274us) ts_tablet_manager.cc:1151] Deleting tablet ffd80badce1048f3ac95366cad05ae53
0202 02:52:22.180667 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.180789 (+ 122us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2261263}
I20260202 02:52:22.181640 1182 tablet_replica.cc:333] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:22.181896 1182 raft_consensus.cc:2243] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:22.181927 1029 tablet_service.cc:1505] Processing CreateTablet for tablet ee04563d3b8e4cb2b989bb8d6ee9b20e (DEFAULT_TABLE table=default.Table_c9f4a79d03a04c2e8135ba9950104fc8 [id=78feba5e47064271ac41a928580d1f02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.182328 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8095e0c6ff0041fc8b8c8b0778a20b00" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:22.182621 1182 raft_consensus.cc:2272] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:22.183439 1180 ts_tablet_manager.cc:1950] T 723921dfe54c4e66af5cab76a25fee0a P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:22.183455 1182 ts_tablet_manager.cc:1916] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:22.184300 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee04563d3b8e4cb2b989bb8d6ee9b20e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.185974 1062 raft_consensus.cc:3060] T 8095e0c6ff0041fc8b8c8b0778a20b00 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 2
I20260202 02:52:22.188201 715 leader_election.cc:304] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:22.188802 1277 raft_consensus.cc:2804] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:22.189036 1709 raft_consensus.cc:493] T 0186572e7bb9499a932a636ab280f544 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:22.189159 1709 raft_consensus.cc:515] T 0186572e7bb9499a932a636ab280f544 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.189391 1277 raft_consensus.cc:697] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:22.189670 1277 consensus_queue.cc:237] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.192370 1781 tablet.cc:1722] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260202 02:52:22.193892 917 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c12ff20ca6b44ddba0c936a28cfb2eb9" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:22.195341 775 raft_consensus.cc:2468] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:22.196743 857 leader_election.cc:304] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:22.197340 1778 raft_consensus.cc:2804] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:22.197528 1778 raft_consensus.cc:697] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:22.197865 1778 consensus_queue.cc:237] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.198067 998 leader_election.cc:304] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:22.198386 1764 raft_consensus.cc:2804] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:22.198547 1173 tablet_bootstrap.cc:492] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:22.198849 1764 raft_consensus.cc:697] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:22.199127 1764 consensus_queue.cc:237] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.204406 1173 ts_tablet_manager.cc:1403] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.121s user 0.011s sys 0.003s
I20260202 02:52:22.210500 1783 tablet.cc:1722] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:22.211300 1769 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:41736 (request call id 287) took 1195 ms. Trace:
I20260202 02:52:22.211504 1769 rpcz_store.cc:276] 0202 02:52:20.969515 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:20.985277 (+ 15762us) service_pool.cc:225] Handling call
0202 02:52:22.164818 (+1179541us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:21.012525 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0202 02:52:21.122545 (+110020us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0202 02:52:21.122554 (+ 9us) alter_schema_op.cc:66] Acquired schema lock
0202 02:52:21.122559 (+ 5us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0202 02:52:21.142269 (+ 19710us) spinlock_profiling.cc:243] Waited 1.18 ms on lock 0x60f000019388. stack: 00007fb35529ca86 00007fb35529c840 00007fb3542773ad 0000000000984f94 0000000000984f5d 000000000098492a 00007fb35e24cd54 00007fb35e24c271 00007fb35e24c07a 00007fb3611c69a9 00007fb3611c62c4 00007fb3625421df 00007fb362541399 00007fb3625497c0 00007fb3625495cd 0000000000a8970c
0202 02:52:21.142305 (+ 36us) alter_schema_op.cc:128] START. Timestamp: 1770000741125009
0202 02:52:21.142308 (+ 3us) op_driver.cc:348] REPLICATION: starting
0202 02:52:21.151113 (+ 8805us) log.cc:844] Serialized 173 byte log entry
0202 02:52:21.901587 (+750474us) op_driver.cc:464] REPLICATION: finished
0202 02:52:22.014553 (+112966us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0202 02:52:22.129448 (+114895us) log.cc:844] Serialized 5 byte log entry
0202 02:52:22.163731 (+ 34283us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.164140 (+ 409us) log.cc:844] Serialized 14 byte log entry
0202 02:52:22.164745 (+ 605us) alter_schema_op.cc:73] Released schema lock
0202 02:52:22.164750 (+ 5us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":90091,"prepare.queue_time_us":26696,"prepare.run_cpu_time_us":6356,"prepare.run_wall_time_us":181122,"replication_time_us":759243,"spinlock_wait_cycles":2723072,"thread_start_us":154351,"threads_started":3,"wal-append.queue_time_us":35479}]]}
I20260202 02:52:22.214191 1175 tablet_bootstrap.cc:492] T d21fca9a5b234c26879717f51c8ea0f0 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:22.214607 1175 ts_tablet_manager.cc:1403] T d21fca9a5b234c26879717f51c8ea0f0 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.311s user 0.010s sys 0.005s
I20260202 02:52:22.215799 759 tablet_service.cc:1505] Processing CreateTablet for tablet 8153510d5d154c7895b1ee794585612e (DEFAULT_TABLE table=default.Table_881157f1bd2945a5be1340a99a941bef [id=c0c6499675894350ac008b534d030c93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.201327 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "723921dfe54c4e66af5cab76a25fee0a" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:22.219219 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 267) took 2565 ms. Trace:
W20260202 02:52:22.222640 998 leader_election.cc:343] T 723921dfe54c4e66af5cab76a25fee0a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: Tablet not RUNNING: SHUTDOWN
I20260202 02:52:22.223434 1037 tablet_service.cc:1505] Processing CreateTablet for tablet eea6bb04ddf24276b46270d009ed78b4 (DEFAULT_TABLE table=default.Table_ec2d6011c0c6475d9cd5a11c6f94d5d6 [id=b7139f5062824d66af2dbd62469a9c03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.222640 1173 raft_consensus.cc:359] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.224807 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eea6bb04ddf24276b46270d009ed78b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.225978 1174 tablet_bootstrap.cc:492] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:22.227514 1178 raft_consensus.cc:515] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:22.228360 1173 raft_consensus.cc:385] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.228600 1062 raft_consensus.cc:2468] T 8095e0c6ff0041fc8b8c8b0778a20b00 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 2.
I20260202 02:52:22.228843 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 5e20842de8cb4355b732685605c88834 (DEFAULT_TABLE table=default.Table_6e6ab095d9a6497d8a89bcc865b6f42f [id=7d8f7ac4c2ee4b72b2ff25aff060010f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.229300 1173 raft_consensus.cc:740] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.229640 1031 tablet_service.cc:1505] Processing CreateTablet for tablet 6a3dac5cd1ce47488a50b4c32eca67c7 (DEFAULT_TABLE table=default.Table_e984fe23a3ab488c9344620087da422d [id=e24ffad12f514d60907b6cadfd09dbe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.231143 1174 ts_tablet_manager.cc:1403] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.409s user 0.008s sys 0.008s
I20260202 02:52:22.231276 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e20842de8cb4355b732685605c88834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.231638 754 tablet_service.cc:1505] Processing CreateTablet for tablet abfe7f76b2424c4d9ab07b29b66ec103 (DEFAULT_TABLE table=default.Table_ff54266e8b5a4334ad1fbeda5270f0f2 [id=b6c09db68b4542a3bb11bf20b1d54571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.238487 715 leader_election.cc:304] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:22.230726 1173 consensus_queue.cc:260] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.238991 1178 leader_election.cc:290] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:22.243763 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c184d44d7f434b39a997a7387e5d3de0" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:22.247439 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a3dac5cd1ce47488a50b4c32eca67c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.218984 1777 raft_consensus.cc:493] T 75b97d7e848241cf8d894b0ae92a8f8f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:22.247910 1777 raft_consensus.cc:515] T 75b97d7e848241cf8d894b0ae92a8f8f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.248988 1777 leader_election.cc:290] T 75b97d7e848241cf8d894b0ae92a8f8f P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:22.249579 747 tablet_service.cc:1505] Processing CreateTablet for tablet 99e0b5e9a9874aec886155e0d614dec2 (DEFAULT_TABLE table=default.Table_a8c4f67a58b74bf8bb3716174ed5654f [id=e7975854ed2f4810a2d438b590267584]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.250957 1709 leader_election.cc:290] T 0186572e7bb9499a932a636ab280f544 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:22.251272 1173 ts_tablet_manager.cc:1434] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.046s user 0.002s sys 0.002s
I20260202 02:52:22.251643 1173 tablet_bootstrap.cc:492] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
W20260202 02:52:22.230255 857 leader_election.cc:343] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:22.252460 857 leader_election.cc:304] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:22.252978 1777 raft_consensus.cc:2749] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:22.254393 758 tablet_service.cc:1552] Processing DeleteTablet for tablet 79badde1001b432b95f58b5161ace15c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:22.255398 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07580ad1d049475a9645acfad0b1d219. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.216856 1175 raft_consensus.cc:359] T d21fca9a5b234c26879717f51c8ea0f0 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.256858 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0186572e7bb9499a932a636ab280f544" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:22.258008 1175 raft_consensus.cc:385] T d21fca9a5b234c26879717f51c8ea0f0 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.259552 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "75b97d7e848241cf8d894b0ae92a8f8f" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:22.259588 1175 raft_consensus.cc:740] T d21fca9a5b234c26879717f51c8ea0f0 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
W20260202 02:52:22.260581 859 leader_election.cc:343] T 75b97d7e848241cf8d894b0ae92a8f8f P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:22.263952 1173 tablet_bootstrap.cc:654] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:22.262403 1175 consensus_queue.cc:260] T d21fca9a5b234c26879717f51c8ea0f0 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.269909 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 6df3d11cf578404ca41807f0777b6c53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:22.270016 1783 tablet.cc:1717] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5: Skipping requested alter to schema version 3, tablet already version 3
I20260202 02:52:22.271437 1769 tablet.cc:1717] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5: Skipping requested alter to schema version 3, tablet already version 3
I20260202 02:52:22.273773 1175 ts_tablet_manager.cc:1434] T d21fca9a5b234c26879717f51c8ea0f0 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.059s user 0.000s sys 0.004s
I20260202 02:52:22.275143 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 723921dfe54c4e66af5cab76a25fee0a (table default.Table_bdfbf5f796b949e196b1c1609fbb3655 [id=c30593053ed7490588ff567e30984807]) successfully deleted
W20260202 02:52:22.275835 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 94bada52db6a47179415d302b541d01f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 94bada52db6a47179415d302b541d01f already in progress: creating tablet
W20260202 02:52:22.276525 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 59990791123c4d4eb620090a1e8e31e1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 59990791123c4d4eb620090a1e8e31e1 already in progress: creating tablet
I20260202 02:52:22.276503 1175 tablet_bootstrap.cc:492] T d64f42d187bc4c9f9381c931449ec0cc P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
W20260202 02:52:22.276870 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet ffd80badce1048f3ac95366cad05ae53 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ffd80badce1048f3ac95366cad05ae53 already in progress: creating tablet
W20260202 02:52:22.277227 999 leader_election.cc:343] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: bc1539e5c8aa4d6e98f687728b891530
I20260202 02:52:22.277700 999 leader_election.cc:304] T c12ff20ca6b44ddba0c936a28cfb2eb9 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
W20260202 02:52:22.278263 999 leader_election.cc:343] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: bc1539e5c8aa4d6e98f687728b891530
W20260202 02:52:22.278735 999 leader_election.cc:343] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 267bd4752c5f4b99a3fe778ba8f38a2f
W20260202 02:52:22.279084 999 leader_election.cc:343] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 267bd4752c5f4b99a3fe778ba8f38a2f
W20260202 02:52:22.279373 999 leader_election.cc:343] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: c184d44d7f434b39a997a7387e5d3de0
W20260202 02:52:22.279644 999 leader_election.cc:343] T 723921dfe54c4e66af5cab76a25fee0a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 723921dfe54c4e66af5cab76a25fee0a
I20260202 02:52:22.279733 999 leader_election.cc:304] T 723921dfe54c4e66af5cab76a25fee0a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:22.280112 1721 raft_consensus.cc:2749] T c12ff20ca6b44ddba0c936a28cfb2eb9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:22.280297 1721 raft_consensus.cc:2749] T 723921dfe54c4e66af5cab76a25fee0a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:22.280858 1721 raft_consensus.cc:493] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:22.281019 1721 raft_consensus.cc:515] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.281636 1721 leader_election.cc:290] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:22.277453 1180 rpcz_store.cc:276] 0202 02:52:19.637905 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.640837 (+ 2932us) service_pool.cc:225] Handling call
0202 02:52:21.749951 (+2109114us) ts_tablet_manager.cc:1151] Deleting tablet 723921dfe54c4e66af5cab76a25fee0a
0202 02:52:21.749965 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:21.782589 (+ 32624us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.098754 (+316165us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.203002 (+104248us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2065304}
I20260202 02:52:22.285781 1723 raft_consensus.cc:2804] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Leader election won for term 2
I20260202 02:52:22.285964 1723 raft_consensus.cc:697] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d [term 2 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:22.286090 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c064ea0dade840b4840a5715c2d83f69" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:22.288525 894 tablet_service.cc:1505] Processing CreateTablet for tablet 7ff6d2fa085343a6b11be0898da6233d (DEFAULT_TABLE table=default.Table_bcf2aa761e104360bb4a1251204a6608 [id=5eaf1a210fa44066b81bb5b01db4d7cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.288570 1708 tablet.cc:1722] T d06b77ff1a6f4f78b73082b00145fcf6 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260202 02:52:22.289029 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ff6d2fa085343a6b11be0898da6233d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.288446 740 tablet_service.cc:1505] Processing CreateTablet for tablet 59ad016e15ec4a488921eb564029b689 (DEFAULT_TABLE table=default.Table_49cb996504bf45e9a24c6f3c01f457ff [id=ec1d1bc3e38643f7ad1c57e98ef66388]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.291713 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ef073880898049e2be93d2cb6dc0c0bb" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:22.289189 1723 consensus_queue.cc:237] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.299012 894 tablet_service.cc:1505] Processing CreateTablet for tablet dc90eeb79ded4739883ad98928476680 (DEFAULT_TABLE table=default.Table_a9b816e5b5b84151b840bfad315deb70 [id=b46bd49d8fe14af094e69597521a1a22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.299582 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc90eeb79ded4739883ad98928476680. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.302623 1798 tablet.cc:1722] T 583e547d0a1943f09f4efe82070fc777 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:22.303645 893 tablet_service.cc:1505] Processing CreateTablet for tablet 528d4269f91a44679acfad16e9fbbe32 (DEFAULT_TABLE table=default.Table_c6515f88263042feadae47be333ed860 [id=c2c454eb645044839fa81b33b6460558]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.304113 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 528d4269f91a44679acfad16e9fbbe32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.309103 1173 tablet_bootstrap.cc:492] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:22.309294 1173 ts_tablet_manager.cc:1403] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.058s user 0.005s sys 0.013s
I20260202 02:52:22.310079 1173 raft_consensus.cc:359] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.310293 1173 raft_consensus.cc:385] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.310364 1173 raft_consensus.cc:740] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.310606 1173 consensus_queue.cc:260] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.311115 1173 ts_tablet_manager.cc:1434] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260202 02:52:22.311431 1173 tablet_bootstrap.cc:492] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:22.313107 1173 tablet_bootstrap.cc:654] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:22.313453 893 tablet_service.cc:1505] Processing CreateTablet for tablet b4a4c75a906f40beb896bf6fff705ae6 (DEFAULT_TABLE table=default.Table_6806575803c4468fa7b7ef990004b446 [id=944fbe1acd894017bae3c54f135ee9b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.314076 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4a4c75a906f40beb896bf6fff705ae6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.320817 758 tablet_service.cc:1552] Processing DeleteTablet for tablet 8095e0c6ff0041fc8b8c8b0778a20b00 with delete_type TABLET_DATA_DELETED (replaced by tablet f5727f9b9bbb4e77abfef065c51606a1) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:22.320983 1175 tablet_bootstrap.cc:654] T d64f42d187bc4c9f9381c931449ec0cc P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:22.324175 715 leader_election.cc:343] T 0186572e7bb9499a932a636ab280f544 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:22.324391 715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 65677.5us
I20260202 02:52:22.324769 899 tablet_service.cc:1505] Processing CreateTablet for tablet ee04563d3b8e4cb2b989bb8d6ee9b20e (DEFAULT_TABLE table=default.Table_c9f4a79d03a04c2e8135ba9950104fc8 [id=78feba5e47064271ac41a928580d1f02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.325909 1798 tablet.cc:1722] T c050a2882f764dda8df2a7df6547f149 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:22.327040 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee04563d3b8e4cb2b989bb8d6ee9b20e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.329492 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f54c1dfe0a942b3bafd85522953a032" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:22.332531 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8153510d5d154c7895b1ee794585612e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.332829 1180 tablet_replica.cc:333] T c064ea0dade840b4840a5715c2d83f69 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:22.333273 1173 tablet_bootstrap.cc:492] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:22.333436 1173 ts_tablet_manager.cc:1403] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.000s
I20260202 02:52:22.333482 758 tablet_service.cc:1505] Processing CreateTablet for tablet 2616e1bbce4049dbb584550ccc150288 (DEFAULT_TABLE table=default.Table_4fb44d5cbe71439ba073bd75711e77e0 [id=8c2d0f27f5d04a98bfad01483c783ecb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.334288 1173 raft_consensus.cc:359] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.334482 1173 raft_consensus.cc:385] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.334582 1173 raft_consensus.cc:740] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.334568 886 tablet_service.cc:1505] Processing CreateTablet for tablet eea6bb04ddf24276b46270d009ed78b4 (DEFAULT_TABLE table=default.Table_ec2d6011c0c6475d9cd5a11c6f94d5d6 [id=b7139f5062824d66af2dbd62469a9c03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.334869 1173 consensus_queue.cc:260] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.335021 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eea6bb04ddf24276b46270d009ed78b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.335412 1173 ts_tablet_manager.cc:1434] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260202 02:52:22.335486 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3eebefa74584c1791d52377f5ba1290. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.335711 1173 tablet_bootstrap.cc:492] T 75f7824681344d8d88df9402694fff1b P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:22.337540 1173 tablet_bootstrap.cc:654] T 75f7824681344d8d88df9402694fff1b P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:22.340664 1178 raft_consensus.cc:493] T b428356fba0c4fb5a6033c5d39740cc5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:22.340852 1178 raft_consensus.cc:515] T b428356fba0c4fb5a6033c5d39740cc5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.341526 1178 leader_election.cc:290] T b428356fba0c4fb5a6033c5d39740cc5 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:22.342335 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b428356fba0c4fb5a6033c5d39740cc5" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:22.343626 922 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b428356fba0c4fb5a6033c5d39740cc5" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:22.343890 922 raft_consensus.cc:2393] T b428356fba0c4fb5a6033c5d39740cc5 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1f05820b6034228bdad884cc348716d in current term 1: Already voted for candidate 1f442ef85029461091de847118c34c0d in this term.
W20260202 02:52:22.344726 999 leader_election.cc:343] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: c184d44d7f434b39a997a7387e5d3de0
I20260202 02:52:22.344952 886 tablet_service.cc:1505] Processing CreateTablet for tablet 6a3dac5cd1ce47488a50b4c32eca67c7 (DEFAULT_TABLE table=default.Table_e984fe23a3ab488c9344620087da422d [id=e24ffad12f514d60907b6cadfd09dbe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.345471 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a3dac5cd1ce47488a50b4c32eca67c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.348734 899 tablet_service.cc:1505] Processing CreateTablet for tablet 5e20842de8cb4355b732685605c88834 (DEFAULT_TABLE table=default.Table_6e6ab095d9a6497d8a89bcc865b6f42f [id=7d8f7ac4c2ee4b72b2ff25aff060010f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.349421 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e20842de8cb4355b732685605c88834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.351471 1798 tablet.cc:1722] T d06b77ff1a6f4f78b73082b00145fcf6 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260202 02:52:22.354979 1173 tablet_bootstrap.cc:492] T 75f7824681344d8d88df9402694fff1b P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:22.355194 1173 ts_tablet_manager.cc:1403] T 75f7824681344d8d88df9402694fff1b P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.000s
I20260202 02:52:22.355760 1182 ts_tablet_manager.cc:1929] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:22.355991 1182 log.cc:1199] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/d2fa4c3d6ec44047abf5986e502c1590
I20260202 02:52:22.355989 1173 raft_consensus.cc:359] T 75f7824681344d8d88df9402694fff1b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.356191 1173 raft_consensus.cc:385] T 75f7824681344d8d88df9402694fff1b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.356261 1173 raft_consensus.cc:740] T 75f7824681344d8d88df9402694fff1b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.356513 1173 consensus_queue.cc:260] T 75f7824681344d8d88df9402694fff1b P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.357016 1173 ts_tablet_manager.cc:1434] T 75f7824681344d8d88df9402694fff1b P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:22.357333 1173 tablet_bootstrap.cc:492] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:22.358799 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 929eb48881b141718873d0e359e768a8 with delete_type TABLET_DATA_DELETED (replaced by tablet cd1b6662da254889ae4bbe8d0b1ef1bb) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:22.359557 1173 tablet_bootstrap.cc:654] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:22.364586 1182 ts_tablet_manager.cc:1950] T d2fa4c3d6ec44047abf5986e502c1590 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:22.366062 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 266) took 2405 ms. Trace:
I20260202 02:52:22.366410 1182 rpcz_store.cc:276] 0202 02:52:19.960438 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.960676 (+ 238us) service_pool.cc:225] Handling call
0202 02:52:22.181425 (+2220749us) ts_tablet_manager.cc:1151] Deleting tablet d2fa4c3d6ec44047abf5986e502c1590
0202 02:52:22.181435 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.265065 (+ 83630us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.355589 (+ 90524us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.365948 (+ 10359us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2216963}
I20260202 02:52:22.368048 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 271) took 2394 ms. Trace:
I20260202 02:52:22.368186 779 raft_consensus.cc:2468] T c064ea0dade840b4840a5715c2d83f69 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:22.368867 998 leader_election.cc:304] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:22.369024 775 raft_consensus.cc:3060] T c184d44d7f434b39a997a7387e5d3de0 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:22.369269 999 leader_election.cc:343] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: c064ea0dade840b4840a5715c2d83f69
I20260202 02:52:22.369680 1178 raft_consensus.cc:2804] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:22.369787 1178 raft_consensus.cc:493] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:22.369863 1178 raft_consensus.cc:3060] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:22.369946 744 tablet_service.cc:1552] Processing DeleteTablet for tablet fe53a8bdbd13417f9ce195c54a52a93c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:21 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:22.370950 1779 tablet.cc:1722] T c050a2882f764dda8df2a7df6547f149 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:22.371315 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "75b97d7e848241cf8d894b0ae92a8f8f" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:22.372016 1174 raft_consensus.cc:359] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:22.372285 1174 raft_consensus.cc:385] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.372500 1174 raft_consensus.cc:740] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.372733 1178 raft_consensus.cc:515] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.373015 1174 consensus_queue.cc:260] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
W20260202 02:52:22.373759 857 leader_election.cc:343] T ef073880898049e2be93d2cb6dc0c0bb P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:22.374030 857 leader_election.cc:304] T ef073880898049e2be93d2cb6dc0c0bb P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:22.374228 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c064ea0dade840b4840a5715c2d83f69" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:22.374477 772 raft_consensus.cc:3060] T c064ea0dade840b4840a5715c2d83f69 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:22.374531 1778 raft_consensus.cc:2749] T ef073880898049e2be93d2cb6dc0c0bb P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:22.374552 1178 leader_election.cc:290] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:22.376201 857 leader_election.cc:343] T 1f54c1dfe0a942b3bafd85522953a032 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:22.376578 857 leader_election.cc:304] T 1f54c1dfe0a942b3bafd85522953a032 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:22.374042 1174 ts_tablet_manager.cc:1434] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.143s user 0.000s sys 0.003s
I20260202 02:52:22.376974 774 raft_consensus.cc:2393] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1f05820b6034228bdad884cc348716d in current term 1: Already voted for candidate 1f442ef85029461091de847118c34c0d in this term.
I20260202 02:52:22.377501 998 leader_election.cc:304] T b428356fba0c4fb5a6033c5d39740cc5 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:22.378022 1178 raft_consensus.cc:3060] T b428356fba0c4fb5a6033c5d39740cc5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:22.382449 1178 raft_consensus.cc:2749] T b428356fba0c4fb5a6033c5d39740cc5 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:22.383035 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2edc6660b9a41e39e9529627fafcb03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.387037 751 tablet_service.cc:1505] Processing CreateTablet for tablet 51ba0bcd498043c383c493f45ad1fe17 (DEFAULT_TABLE table=default.Table_619d2fcf726a46bfb5e939bbe6f55612 [id=038692c44a0043bab69ab29f77aa13be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.389648 1175 tablet_bootstrap.cc:492] T d64f42d187bc4c9f9381c931449ec0cc P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:22.389830 1175 ts_tablet_manager.cc:1403] T d64f42d187bc4c9f9381c931449ec0cc P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.114s user 0.003s sys 0.002s
I20260202 02:52:22.390262 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51ba0bcd498043c383c493f45ad1fe17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.392637 1778 raft_consensus.cc:2749] T 1f54c1dfe0a942b3bafd85522953a032 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:22.392918 1175 raft_consensus.cc:359] T d64f42d187bc4c9f9381c931449ec0cc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:22.393111 1175 raft_consensus.cc:385] T d64f42d187bc4c9f9381c931449ec0cc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.393208 1175 raft_consensus.cc:740] T d64f42d187bc4c9f9381c931449ec0cc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.392637 741 tablet_service.cc:1505] Processing CreateTablet for tablet 7ff6d2fa085343a6b11be0898da6233d (DEFAULT_TABLE table=default.Table_bcf2aa761e104360bb4a1251204a6608 [id=5eaf1a210fa44066b81bb5b01db4d7cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.393442 1175 consensus_queue.cc:260] T d64f42d187bc4c9f9381c931449ec0cc P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:22.393898 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ff6d2fa085343a6b11be0898da6233d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.394016 1175 ts_tablet_manager.cc:1434] T d64f42d187bc4c9f9381c931449ec0cc P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.004s user 0.000s sys 0.000s
W20260202 02:52:22.394016 857 leader_election.cc:343] T 75b97d7e848241cf8d894b0ae92a8f8f P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:22.394268 857 leader_election.cc:304] T 75b97d7e848241cf8d894b0ae92a8f8f P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:22.394400 1175 tablet_bootstrap.cc:492] T ffd80badce1048f3ac95366cad05ae53 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:22.394665 1777 raft_consensus.cc:2749] T 75b97d7e848241cf8d894b0ae92a8f8f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:22.396179 1175 tablet_bootstrap.cc:654] T ffd80badce1048f3ac95366cad05ae53 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:22.396229 999 leader_election.cc:343] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: c064ea0dade840b4840a5715c2d83f69
I20260202 02:52:22.397718 757 tablet_service.cc:1505] Processing CreateTablet for tablet dc90eeb79ded4739883ad98928476680 (DEFAULT_TABLE table=default.Table_a9b816e5b5b84151b840bfad315deb70 [id=b46bd49d8fe14af094e69597521a1a22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.398319 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc90eeb79ded4739883ad98928476680. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.399403 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet d2fa4c3d6ec44047abf5986e502c1590 (table default.Table_cf37f87385ee44cea1ff80cc14050ff2 [id=697b6fe62ee845d895af350881a03556]) successfully deleted
I20260202 02:52:22.399955 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59ad016e15ec4a488921eb564029b689. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.368384 1182 rpcz_store.cc:276] 0202 02:52:19.978339 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.978406 (+ 67us) service_pool.cc:225] Handling call
0202 02:52:22.367678 (+2389272us) ts_tablet_manager.cc:1151] Deleting tablet 02b434fa058b45529ff3938b053263c6
0202 02:52:22.367721 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.367944 (+ 223us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2357281}
I20260202 02:52:22.402220 746 tablet_service.cc:1505] Processing CreateTablet for tablet b4a4c75a906f40beb896bf6fff705ae6 (DEFAULT_TABLE table=default.Table_6806575803c4468fa7b7ef990004b446 [id=944fbe1acd894017bae3c54f135ee9b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.403049 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2616e1bbce4049dbb584550ccc150288. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.402994 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 262) took 2465 ms. Trace:
I20260202 02:52:22.403327 1182 rpcz_store.cc:276] 0202 02:52:19.937226 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.937337 (+ 111us) service_pool.cc:225] Handling call
0202 02:52:22.402372 (+2465035us) ts_tablet_manager.cc:1151] Deleting tablet d1e0e74f32a24729b250c5751677cfb0
0202 02:52:22.402383 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.402666 (+ 283us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2377130}
I20260202 02:52:22.403839 1182 tablet_replica.cc:333] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:22.403920 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99e0b5e9a9874aec886155e0d614dec2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.404206 1182 raft_consensus.cc:2243] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:22.404966 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abfe7f76b2424c4d9ab07b29b66ec103. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.404924 1182 raft_consensus.cc:2272] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:22.405737 1174 tablet_bootstrap.cc:492] T 75b97d7e848241cf8d894b0ae92a8f8f P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:22.405961 1182 ts_tablet_manager.cc:1916] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:22.407102 1175 tablet_bootstrap.cc:492] T ffd80badce1048f3ac95366cad05ae53 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:22.407291 1175 ts_tablet_manager.cc:1403] T ffd80badce1048f3ac95366cad05ae53 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.001s
I20260202 02:52:22.407902 893 tablet_service.cc:1505] Processing CreateTablet for tablet f5727f9b9bbb4e77abfef065c51606a1 (DEFAULT_TABLE table=default.Table_4e6c3886f58b49698a00d45b957f4c9a [id=b5cc03b77f414304b29c5fb4bb7dc3ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.408457 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5727f9b9bbb4e77abfef065c51606a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.409598 1175 raft_consensus.cc:359] T ffd80badce1048f3ac95366cad05ae53 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.409842 745 tablet_service.cc:1505] Processing CreateTablet for tablet f1c8f7e1dc9644a6b6dd2bc264cce8ef (DEFAULT_TABLE table=default.Table_06273452ab3e4d2e8bdb1e3a41721dcc [id=4096475017214ffdba27d20aca4e8d64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.410169 1175 raft_consensus.cc:385] T ffd80badce1048f3ac95366cad05ae53 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.410269 1175 raft_consensus.cc:740] T ffd80badce1048f3ac95366cad05ae53 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.411056 1173 tablet_bootstrap.cc:492] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
W20260202 02:52:22.411018 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 02b434fa058b45529ff3938b053263c6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 02b434fa058b45529ff3938b053263c6 already in progress: creating tablet
I20260202 02:52:22.411213 1173 ts_tablet_manager.cc:1403] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.054s user 0.003s sys 0.010s
I20260202 02:52:22.411907 1173 raft_consensus.cc:359] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.412034 1783 tablet.cc:1717] T d06b77ff1a6f4f78b73082b00145fcf6 P 1521a6d262b7451bbef8debe970dd6b5: Skipping requested alter to schema version 3, tablet already version 3
I20260202 02:52:22.412099 1173 raft_consensus.cc:385] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.412182 1173 raft_consensus.cc:740] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.412433 1173 consensus_queue.cc:260] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:22.412568 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet d1e0e74f32a24729b250c5751677cfb0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d1e0e74f32a24729b250c5751677cfb0 already in progress: creating tablet
I20260202 02:52:22.413018 1173 ts_tablet_manager.cc:1434] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:22.413328 1173 tablet_bootstrap.cc:492] T 80d443bdb66d4cc68ae283e08c410fe6 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:22.413856 1783 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:41736 (request call id 285) took 1494 ms. Trace:
I20260202 02:52:22.413957 1783 rpcz_store.cc:276] 0202 02:52:20.917493 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:20.917537 (+ 44us) service_pool.cc:225] Handling call
0202 02:52:22.412441 (+1494904us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:20.986333 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0202 02:52:21.122948 (+136615us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0202 02:52:22.144431 (+1021483us) alter_schema_op.cc:66] Acquired schema lock
0202 02:52:22.144440 (+ 9us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0202 02:52:22.144492 (+ 52us) alter_schema_op.cc:128] START. Timestamp: 1770000742144460
0202 02:52:22.144495 (+ 3us) op_driver.cc:348] REPLICATION: starting
0202 02:52:22.158278 (+ 13783us) log.cc:844] Serialized 173 byte log entry
0202 02:52:22.411325 (+253047us) op_driver.cc:464] REPLICATION: finished
0202 02:52:22.411450 (+ 125us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0202 02:52:22.412300 (+ 850us) log.cc:844] Serialized 94 byte log entry
0202 02:52:22.412392 (+ 92us) alter_schema_op.cc:73] Released schema lock
0202 02:52:22.412398 (+ 6us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":70,"prepare.queue_time_us":68272,"prepare.run_cpu_time_us":68544,"prepare.run_wall_time_us":1172369,"replication_time_us":266794,"thread_start_us":110013,"threads_started":1}]]}
I20260202 02:52:22.415076 1173 tablet_bootstrap.cc:654] T 80d443bdb66d4cc68ae283e08c410fe6 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:22.415522 775 raft_consensus.cc:2468] T c184d44d7f434b39a997a7387e5d3de0 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:22.415511 1175 consensus_queue.cc:260] T ffd80badce1048f3ac95366cad05ae53 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.416107 1175 ts_tablet_manager.cc:1434] T ffd80badce1048f3ac95366cad05ae53 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260202 02:52:22.416077 998 leader_election.cc:304] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:22.416486 1175 tablet_bootstrap.cc:492] T 622a113af04a41c6852c29fc8a8acd78 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:22.416785 1178 raft_consensus.cc:2804] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:22.416927 1178 raft_consensus.cc:697] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:22.417260 1178 consensus_queue.cc:237] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:22.418588 1175 tablet_bootstrap.cc:654] T 622a113af04a41c6852c29fc8a8acd78 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:22.414728 749 tablet_service.cc:1505] Processing CreateTablet for tablet 25a407001f964da6b51f42375136c79a (DEFAULT_TABLE table=default.Table_48226e7c77da41cdba56e04221eb9689 [id=05e52ce8e94e4975b51849059b904272]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.422441 744 tablet_service.cc:1505] Processing CreateTablet for tablet 7a44585a217b48f39f247bae3ca2e255 (DEFAULT_TABLE table=default.Table_2ee0b536f96e44e4a3a4a9439c7bccc9 [id=d95486b11c494863be7380146ebcf0c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.425570 1178 raft_consensus.cc:493] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:22.425777 1178 raft_consensus.cc:515] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.426522 1178 leader_election.cc:290] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:22.430174 715 consensus_peers.cc:597] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d3633f5f9f54115bc78b9c1b78aba67. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:22.430615 772 raft_consensus.cc:2468] T c064ea0dade840b4840a5715c2d83f69 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:22.431207 998 leader_election.cc:304] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:22.432358 1178 raft_consensus.cc:2804] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:22.432559 1178 raft_consensus.cc:697] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:22.433002 1178 consensus_queue.cc:237] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:22.437831 999 leader_election.cc:343] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 0186572e7bb9499a932a636ab280f544
I20260202 02:52:22.440014 1173 tablet_bootstrap.cc:492] T 80d443bdb66d4cc68ae283e08c410fe6 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:22.440222 1173 ts_tablet_manager.cc:1403] T 80d443bdb66d4cc68ae283e08c410fe6 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.027s user 0.003s sys 0.002s
I20260202 02:52:22.441133 1173 raft_consensus.cc:359] T 80d443bdb66d4cc68ae283e08c410fe6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.441359 1173 raft_consensus.cc:385] T 80d443bdb66d4cc68ae283e08c410fe6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.441426 1173 raft_consensus.cc:740] T 80d443bdb66d4cc68ae283e08c410fe6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.441661 1173 consensus_queue.cc:260] T 80d443bdb66d4cc68ae283e08c410fe6 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.442145 1173 ts_tablet_manager.cc:1434] T 80d443bdb66d4cc68ae283e08c410fe6 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:22.442468 1173 tablet_bootstrap.cc:492] T 526f2c7b95184a0987e7aa94efc8475a P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:22.443831 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0186572e7bb9499a932a636ab280f544" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:22.444060 775 raft_consensus.cc:2468] T 0186572e7bb9499a932a636ab280f544 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:22.444152 1173 tablet_bootstrap.cc:654] T 526f2c7b95184a0987e7aa94efc8475a P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:22.444414 1783 tablet.cc:1717] T d06b77ff1a6f4f78b73082b00145fcf6 P 1521a6d262b7451bbef8debe970dd6b5: Skipping requested alter to schema version 3, tablet already version 3
I20260202 02:52:22.444509 998 leader_election.cc:304] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:22.444823 1783 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:41736 (request call id 314) took 1107 ms. Trace:
I20260202 02:52:22.445053 1764 raft_consensus.cc:2804] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:22.445163 1764 raft_consensus.cc:493] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:22.444952 1783 rpcz_store.cc:276] 0202 02:52:21.337679 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:21.353922 (+ 16243us) service_pool.cc:225] Handling call
0202 02:52:22.444794 (+1090872us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:22.158650 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0202 02:52:22.158765 (+ 115us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0202 02:52:22.412983 (+254218us) alter_schema_op.cc:66] Acquired schema lock
0202 02:52:22.412996 (+ 13us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0202 02:52:22.413044 (+ 48us) alter_schema_op.cc:128] START. Timestamp: 1770000742412985
0202 02:52:22.413047 (+ 3us) op_driver.cc:348] REPLICATION: starting
0202 02:52:22.413266 (+ 219us) log.cc:844] Serialized 173 byte log entry
0202 02:52:22.444184 (+ 30918us) op_driver.cc:464] REPLICATION: finished
0202 02:52:22.444302 (+ 118us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0202 02:52:22.444707 (+ 405us) log.cc:844] Serialized 94 byte log entry
0202 02:52:22.444750 (+ 43us) alter_schema_op.cc:73] Released schema lock
0202 02:52:22.444756 (+ 6us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":66,"prepare.queue_time_us":804293,"prepare.run_cpu_time_us":15947,"prepare.run_wall_time_us":255034,"replication_time_us":31099}]]}
I20260202 02:52:22.445272 1764 raft_consensus.cc:3060] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:22.448110 1764 raft_consensus.cc:515] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.453493 1180 raft_consensus.cc:2243] T c064ea0dade840b4840a5715c2d83f69 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:22.453941 1180 raft_consensus.cc:2272] T c064ea0dade840b4840a5715c2d83f69 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:22.454583 1180 ts_tablet_manager.cc:1916] T c064ea0dade840b4840a5715c2d83f69 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:22.455719 742 tablet_service.cc:1505] Processing CreateTablet for tablet f5727f9b9bbb4e77abfef065c51606a1 (DEFAULT_TABLE table=default.Table_4e6c3886f58b49698a00d45b957f4c9a [id=b5cc03b77f414304b29c5fb4bb7dc3ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.457355 1173 tablet_bootstrap.cc:492] T 526f2c7b95184a0987e7aa94efc8475a P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:22.457479 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0186572e7bb9499a932a636ab280f544" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:22.457710 775 raft_consensus.cc:3060] T 0186572e7bb9499a932a636ab280f544 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:22.457511 1173 ts_tablet_manager.cc:1403] T 526f2c7b95184a0987e7aa94efc8475a P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.003s
I20260202 02:52:22.459039 1173 raft_consensus.cc:359] T 526f2c7b95184a0987e7aa94efc8475a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:22.459237 1173 raft_consensus.cc:385] T 526f2c7b95184a0987e7aa94efc8475a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.459317 1173 raft_consensus.cc:740] T 526f2c7b95184a0987e7aa94efc8475a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.459528 1173 consensus_queue.cc:260] T 526f2c7b95184a0987e7aa94efc8475a P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:22.460235 1173 ts_tablet_manager.cc:1434] T 526f2c7b95184a0987e7aa94efc8475a P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260202 02:52:22.460636 1173 tablet_bootstrap.cc:492] T eb0ac55fbb534b29a86d6b3e8aa72ebc P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:22.460670 1182 ts_tablet_manager.cc:1929] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:22.460963 1182 log.cc:1199] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/61a9203035f2495f8f99924303ebf120
I20260202 02:52:22.461510 1182 ts_tablet_manager.cc:1950] T 61a9203035f2495f8f99924303ebf120 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:22.462596 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 263) took 2513 ms. Trace:
I20260202 02:52:22.464866 1182 rpcz_store.cc:276] 0202 02:52:19.949224 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.951637 (+ 2413us) service_pool.cc:225] Handling call
0202 02:52:22.403737 (+2452100us) ts_tablet_manager.cc:1151] Deleting tablet 61a9203035f2495f8f99924303ebf120
0202 02:52:22.403748 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.446355 (+ 42607us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.460527 (+ 14172us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.462522 (+ 1995us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2377072}
I20260202 02:52:22.465580 1182 tablet_replica.cc:333] T 75f7824681344d8d88df9402694fff1b P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:22.466079 1182 raft_consensus.cc:2243] T 75f7824681344d8d88df9402694fff1b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:22.466317 1182 raft_consensus.cc:2272] T 75f7824681344d8d88df9402694fff1b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:22.470288 1173 tablet_bootstrap.cc:654] T eb0ac55fbb534b29a86d6b3e8aa72ebc P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:22.471163 1764 leader_election.cc:290] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:22.471793 1182 ts_tablet_manager.cc:1916] T 75f7824681344d8d88df9402694fff1b P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:22.471451 999 leader_election.cc:343] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 0186572e7bb9499a932a636ab280f544
I20260202 02:52:22.477826 1173 tablet_bootstrap.cc:492] T eb0ac55fbb534b29a86d6b3e8aa72ebc P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:22.478034 1173 ts_tablet_manager.cc:1403] T eb0ac55fbb534b29a86d6b3e8aa72ebc P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.001s
I20260202 02:52:22.478788 1173 raft_consensus.cc:359] T eb0ac55fbb534b29a86d6b3e8aa72ebc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.479007 1173 raft_consensus.cc:385] T eb0ac55fbb534b29a86d6b3e8aa72ebc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.479085 1173 raft_consensus.cc:740] T eb0ac55fbb534b29a86d6b3e8aa72ebc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.479331 1173 consensus_queue.cc:260] T eb0ac55fbb534b29a86d6b3e8aa72ebc P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.479839 1173 ts_tablet_manager.cc:1434] T eb0ac55fbb534b29a86d6b3e8aa72ebc P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260202 02:52:22.480186 1173 tablet_bootstrap.cc:492] T 75b97d7e848241cf8d894b0ae92a8f8f P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:22.484583 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4a4c75a906f40beb896bf6fff705ae6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.488159 1708 tablet.cc:1722] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:22.488621 1779 tablet.cc:1717] T d06b77ff1a6f4f78b73082b00145fcf6 P 1f442ef85029461091de847118c34c0d: Skipping requested alter to schema version 3, tablet already version 3
I20260202 02:52:22.490448 1173 tablet_bootstrap.cc:654] T 75b97d7e848241cf8d894b0ae92a8f8f P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:22.492600 1182 ts_tablet_manager.cc:1929] T 75f7824681344d8d88df9402694fff1b P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:22.493573 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a44585a217b48f39f247bae3ca2e255. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.498505 1182 log.cc:1199] T 75f7824681344d8d88df9402694fff1b P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/75f7824681344d8d88df9402694fff1b
I20260202 02:52:22.499095 1182 ts_tablet_manager.cc:1950] T 75f7824681344d8d88df9402694fff1b P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:22.499541 1173 tablet_bootstrap.cc:492] T 75b97d7e848241cf8d894b0ae92a8f8f P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:22.499709 1173 ts_tablet_manager.cc:1403] T 75b97d7e848241cf8d894b0ae92a8f8f P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.000s
I20260202 02:52:22.500133 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 257) took 2615 ms. Trace:
I20260202 02:52:22.500319 1182 rpcz_store.cc:276] 0202 02:52:19.884368 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.885748 (+ 1380us) service_pool.cc:225] Handling call
0202 02:52:22.465474 (+2579726us) ts_tablet_manager.cc:1151] Deleting tablet 75f7824681344d8d88df9402694fff1b
0202 02:52:22.465488 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.475564 (+ 10076us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.492445 (+ 16881us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.500093 (+ 7648us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2436672}
I20260202 02:52:22.500608 1173 raft_consensus.cc:359] T 75b97d7e848241cf8d894b0ae92a8f8f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.500826 1173 raft_consensus.cc:385] T 75b97d7e848241cf8d894b0ae92a8f8f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.500929 1173 raft_consensus.cc:740] T 75b97d7e848241cf8d894b0ae92a8f8f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.501149 1182 tablet_replica.cc:333] T 80d443bdb66d4cc68ae283e08c410fe6 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:22.501185 1173 consensus_queue.cc:260] T 75b97d7e848241cf8d894b0ae92a8f8f P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.501456 1182 raft_consensus.cc:2243] T 80d443bdb66d4cc68ae283e08c410fe6 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:22.501655 1182 raft_consensus.cc:2272] T 80d443bdb66d4cc68ae283e08c410fe6 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:22.501708 1173 ts_tablet_manager.cc:1434] T 75b97d7e848241cf8d894b0ae92a8f8f P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:22.502007 1173 tablet_bootstrap.cc:492] T 70306ed9dec54286885d965f2c012da4 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:22.502363 1182 ts_tablet_manager.cc:1916] T 80d443bdb66d4cc68ae283e08c410fe6 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:22.503625 1173 tablet_bootstrap.cc:654] T 70306ed9dec54286885d965f2c012da4 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:22.504287 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 75f7824681344d8d88df9402694fff1b (table default.Table_ae49654353a64141b6d7f0e6afe13d9d [id=431440e879c04f0f9ad63c64d8741b56]) successfully deleted
I20260202 02:52:22.505538 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1c8f7e1dc9644a6b6dd2bc264cce8ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.509172 1779 tablet.cc:1717] T d06b77ff1a6f4f78b73082b00145fcf6 P 1f442ef85029461091de847118c34c0d: Skipping requested alter to schema version 3, tablet already version 3
W20260202 02:52:22.511044 1800 log.cc:927] Time spent T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d: Append to log took a long time: real 0.148s user 0.000s sys 0.000s
I20260202 02:52:22.512117 1182 ts_tablet_manager.cc:1929] T 80d443bdb66d4cc68ae283e08c410fe6 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:22.513757 759 tablet_service.cc:1505] Processing CreateTablet for tablet eea6bb04ddf24276b46270d009ed78b4 (DEFAULT_TABLE table=default.Table_ec2d6011c0c6475d9cd5a11c6f94d5d6 [id=b7139f5062824d66af2dbd62469a9c03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.514304 1795 tablet.cc:1722] T b76d1837813c46e8a0925b49939d8884 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:22.516479 1182 log.cc:1199] T 80d443bdb66d4cc68ae283e08c410fe6 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/80d443bdb66d4cc68ae283e08c410fe6
I20260202 02:52:22.517087 1182 ts_tablet_manager.cc:1950] T 80d443bdb66d4cc68ae283e08c410fe6 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:22.522830 918 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "70306ed9dec54286885d965f2c012da4" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:22.523244 918 raft_consensus.cc:3060] T 70306ed9dec54286885d965f2c012da4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:22.523953 1798 tablet.cc:1722] T 5986676f5484498da927b4c6da3f47ff P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:22.523080 922 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8095e0c6ff0041fc8b8c8b0778a20b00" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:22.526978 918 raft_consensus.cc:2468] T 70306ed9dec54286885d965f2c012da4 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:22.527109 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25a407001f964da6b51f42375136c79a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.527432 922 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8095e0c6ff0041fc8b8c8b0778a20b00" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:22.527973 1798 tablet.cc:1717] T d06b77ff1a6f4f78b73082b00145fcf6 P b1f05820b6034228bdad884cc348716d: Skipping requested alter to schema version 3, tablet already version 3
I20260202 02:52:22.528529 1798 tablet.cc:1717] T d06b77ff1a6f4f78b73082b00145fcf6 P b1f05820b6034228bdad884cc348716d: Skipping requested alter to schema version 3, tablet already version 3
I20260202 02:52:22.528690 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 267) took 2563 ms. Trace:
I20260202 02:52:22.528852 1182 rpcz_store.cc:276] 0202 02:52:19.965287 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.965336 (+ 49us) service_pool.cc:225] Handling call
0202 02:52:22.501075 (+2535739us) ts_tablet_manager.cc:1151] Deleting tablet 80d443bdb66d4cc68ae283e08c410fe6
0202 02:52:22.501088 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.508915 (+ 7827us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.511956 (+ 3041us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.528640 (+ 16684us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2465175}
I20260202 02:52:22.529392 1173 tablet_bootstrap.cc:492] T 70306ed9dec54286885d965f2c012da4 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:22.529419 1182 tablet_replica.cc:333] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:22.529564 1173 ts_tablet_manager.cc:1403] T 70306ed9dec54286885d965f2c012da4 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.028s user 0.003s sys 0.000s
I20260202 02:52:22.529760 1182 raft_consensus.cc:2243] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:22.529654 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 80d443bdb66d4cc68ae283e08c410fe6 (table default.Table_78e75899f83c46f09c66dc84192f5438 [id=0bd47d48c02944a19ce210fc83794879]) successfully deleted
I20260202 02:52:22.530020 1182 raft_consensus.cc:2272] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:22.530360 1173 raft_consensus.cc:359] T 70306ed9dec54286885d965f2c012da4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.530567 1173 raft_consensus.cc:385] T 70306ed9dec54286885d965f2c012da4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.530647 1173 raft_consensus.cc:740] T 70306ed9dec54286885d965f2c012da4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.530889 1173 consensus_queue.cc:260] T 70306ed9dec54286885d965f2c012da4 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.531242 1182 ts_tablet_manager.cc:1916] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:22.531513 1173 ts_tablet_manager.cc:1434] T 70306ed9dec54286885d965f2c012da4 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260202 02:52:22.531857 1173 tablet_bootstrap.cc:492] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:22.538856 1182 ts_tablet_manager.cc:1929] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:22.539645 1173 tablet_bootstrap.cc:654] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:22.539724 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5727f9b9bbb4e77abfef065c51606a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.540807 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 723921dfe54c4e66af5cab76a25fee0a (table default.Table_bdfbf5f796b949e196b1c1609fbb3655 [id=c30593053ed7490588ff567e30984807]) successfully deleted
I20260202 02:52:22.543504 748 tablet_service.cc:1505] Processing CreateTablet for tablet 5e20842de8cb4355b732685605c88834 (DEFAULT_TABLE table=default.Table_6e6ab095d9a6497d8a89bcc865b6f42f [id=7d8f7ac4c2ee4b72b2ff25aff060010f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.543475 743 tablet_service.cc:1505] Processing CreateTablet for tablet ee04563d3b8e4cb2b989bb8d6ee9b20e (DEFAULT_TABLE table=default.Table_c9f4a79d03a04c2e8135ba9950104fc8 [id=78feba5e47064271ac41a928580d1f02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.546535 751 tablet_service.cc:1505] Processing CreateTablet for tablet 6a3dac5cd1ce47488a50b4c32eca67c7 (DEFAULT_TABLE table=default.Table_e984fe23a3ab488c9344620087da422d [id=e24ffad12f514d60907b6cadfd09dbe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.548107 775 raft_consensus.cc:2468] T 0186572e7bb9499a932a636ab280f544 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:22.549064 750 tablet_service.cc:1505] Processing CreateTablet for tablet 528d4269f91a44679acfad16e9fbbe32 (DEFAULT_TABLE table=default.Table_c6515f88263042feadae47be333ed860 [id=c2c454eb645044839fa81b33b6460558]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.553485 1173 tablet_bootstrap.cc:492] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:22.553668 1173 ts_tablet_manager.cc:1403] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.002s
I20260202 02:52:22.554554 1173 raft_consensus.cc:359] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.554751 1173 raft_consensus.cc:385] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.554836 1173 raft_consensus.cc:740] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.555497 998 leader_election.cc:304] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:22.555549 1173 consensus_queue.cc:260] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.556162 1173 ts_tablet_manager.cc:1434] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260202 02:52:22.556298 1764 raft_consensus.cc:2804] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:22.556531 1173 tablet_bootstrap.cc:492] T 78f122342dd144b48de89902faa0f94f P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:22.556740 1764 raft_consensus.cc:697] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:22.557314 1764 consensus_queue.cc:237] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.558612 1173 tablet_bootstrap.cc:654] T 78f122342dd144b48de89902faa0f94f P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:22.545174 1175 tablet_bootstrap.cc:492] T 622a113af04a41c6852c29fc8a8acd78 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:22.559069 1182 log.cc:1199] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/c12ff20ca6b44ddba0c936a28cfb2eb9
I20260202 02:52:22.559186 1175 ts_tablet_manager.cc:1403] T 622a113af04a41c6852c29fc8a8acd78 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.143s user 0.006s sys 0.023s
I20260202 02:52:22.560168 1175 raft_consensus.cc:359] T 622a113af04a41c6852c29fc8a8acd78 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:22.560467 1175 raft_consensus.cc:385] T 622a113af04a41c6852c29fc8a8acd78 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.560595 1175 raft_consensus.cc:740] T 622a113af04a41c6852c29fc8a8acd78 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.560935 1175 consensus_queue.cc:260] T 622a113af04a41c6852c29fc8a8acd78 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:22.561563 1175 ts_tablet_manager.cc:1434] T 622a113af04a41c6852c29fc8a8acd78 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260202 02:52:22.561937 1175 tablet_bootstrap.cc:492] T d67e91a022f1401ea724dff898c3b81b P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:22.563716 1175 tablet_bootstrap.cc:654] T d67e91a022f1401ea724dff898c3b81b P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:22.566291 1182 ts_tablet_manager.cc:1950] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:22.569166 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eea6bb04ddf24276b46270d009ed78b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.570003 743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee04563d3b8e4cb2b989bb8d6ee9b20e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.570226 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e20842de8cb4355b732685605c88834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.570393 1175 tablet_bootstrap.cc:492] T d67e91a022f1401ea724dff898c3b81b P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:22.570446 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 264) took 2618 ms. Trace:
I20260202 02:52:22.570572 1175 ts_tablet_manager.cc:1403] T d67e91a022f1401ea724dff898c3b81b P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.001s
I20260202 02:52:22.571278 1175 raft_consensus.cc:359] T d67e91a022f1401ea724dff898c3b81b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:22.571566 1175 raft_consensus.cc:385] T d67e91a022f1401ea724dff898c3b81b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.571666 1175 raft_consensus.cc:740] T d67e91a022f1401ea724dff898c3b81b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.571998 1175 consensus_queue.cc:260] T d67e91a022f1401ea724dff898c3b81b P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
W20260202 02:52:22.572647 418 external_mini_cluster.cc:1715] Unable to check leaks on /tmp/dist-test-taskYXxklX/build/asan/bin/kudu pid 426: Timed out: CheckLeaks RPC to 127.0.104.190:33981 timed out after 1.000s (SENT)
I20260202 02:52:22.572638 1182 rpcz_store.cc:276] 0202 02:52:19.951856 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.952599 (+ 743us) service_pool.cc:225] Handling call
0202 02:52:22.529338 (+2576739us) ts_tablet_manager.cc:1151] Deleting tablet c12ff20ca6b44ddba0c936a28cfb2eb9
0202 02:52:22.529347 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.535449 (+ 6102us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.538699 (+ 3250us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.570390 (+ 31691us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2488732}
I20260202 02:52:22.572846 418 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskYXxklX/build/asan/bin/kudu with pid 426
I20260202 02:52:22.573168 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 276) took 2530 ms. Trace:
I20260202 02:52:22.573276 1182 rpcz_store.cc:276] 0202 02:52:20.042777 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:20.042840 (+ 63us) service_pool.cc:225] Handling call
0202 02:52:22.573001 (+2530161us) ts_tablet_manager.cc:1151] Deleting tablet 9c1271ce0f194021b588b82ac9d3db79
0202 02:52:22.573013 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.573137 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2529518}
W20260202 02:52:22.573655 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 9c1271ce0f194021b588b82ac9d3db79 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9c1271ce0f194021b588b82ac9d3db79 already in progress: creating tablet
I20260202 02:52:22.574072 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a3dac5cd1ce47488a50b4c32eca67c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.574618 1182 tablet_replica.cc:333] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:22.575111 1182 raft_consensus.cc:2243] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:22.575685 1182 raft_consensus.cc:2272] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:22.578153 1173 tablet_bootstrap.cc:492] T 78f122342dd144b48de89902faa0f94f P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:22.578359 1173 ts_tablet_manager.cc:1403] T 78f122342dd144b48de89902faa0f94f P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.005s
I20260202 02:52:22.578173 1175 ts_tablet_manager.cc:1434] T d67e91a022f1401ea724dff898c3b81b P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260202 02:52:22.579073 1175 tablet_bootstrap.cc:492] T 929eb48881b141718873d0e359e768a8 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:22.579173 1173 raft_consensus.cc:359] T 78f122342dd144b48de89902faa0f94f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:22.579392 1173 raft_consensus.cc:385] T 78f122342dd144b48de89902faa0f94f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.579466 1173 raft_consensus.cc:740] T 78f122342dd144b48de89902faa0f94f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.579723 1173 consensus_queue.cc:260] T 78f122342dd144b48de89902faa0f94f P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:22.579105 1182 ts_tablet_manager.cc:1916] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:22.580276 1173 ts_tablet_manager.cc:1434] T 78f122342dd144b48de89902faa0f94f P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260202 02:52:22.580615 1173 tablet_bootstrap.cc:492] T 757492288d7c44fb91b25b7e6e21e01b P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:22.582877 1173 tablet_bootstrap.cc:654] T 757492288d7c44fb91b25b7e6e21e01b P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:22.583581 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 528d4269f91a44679acfad16e9fbbe32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.586812 1175 tablet_bootstrap.cc:654] T 929eb48881b141718873d0e359e768a8 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:22.592635 1173 tablet_bootstrap.cc:492] T 757492288d7c44fb91b25b7e6e21e01b P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:22.592811 1173 ts_tablet_manager.cc:1403] T 757492288d7c44fb91b25b7e6e21e01b P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.001s
I20260202 02:52:22.593575 1173 raft_consensus.cc:359] T 757492288d7c44fb91b25b7e6e21e01b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.593773 1173 raft_consensus.cc:385] T 757492288d7c44fb91b25b7e6e21e01b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.593869 1173 raft_consensus.cc:740] T 757492288d7c44fb91b25b7e6e21e01b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.594090 1173 consensus_queue.cc:260] T 757492288d7c44fb91b25b7e6e21e01b P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.594594 1173 ts_tablet_manager.cc:1434] T 757492288d7c44fb91b25b7e6e21e01b P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260202 02:52:22.594892 1173 tablet_bootstrap.cc:492] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:22.596873 1173 tablet_bootstrap.cc:654] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:22.596890 1182 ts_tablet_manager.cc:1929] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:22.604135 1277 raft_consensus.cc:493] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:22.604430 1277 raft_consensus.cc:515] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:22.605810 1182 log.cc:1199] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/eb0ac55fbb534b29a86d6b3e8aa72ebc
I20260202 02:52:22.605950 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "526f2c7b95184a0987e7aa94efc8475a" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:22.606233 1063 raft_consensus.cc:2468] T 526f2c7b95184a0987e7aa94efc8475a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:22.606544 1182 ts_tablet_manager.cc:1950] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:22.606952 1173 tablet_bootstrap.cc:492] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:22.606940 715 leader_election.cc:304] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:22.607131 1173 ts_tablet_manager.cc:1403] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.000s
I20260202 02:52:22.607215 1277 leader_election.cc:290] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:22.607419 1277 raft_consensus.cc:2804] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:22.607590 1277 raft_consensus.cc:493] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:22.607722 1277 raft_consensus.cc:3060] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:22.607856 1173 raft_consensus.cc:359] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.608079 1173 raft_consensus.cc:385] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.608157 1173 raft_consensus.cc:740] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.608410 1173 consensus_queue.cc:260] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.608896 1173 ts_tablet_manager.cc:1434] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:22.608992 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 277) took 2561 ms. Trace:
I20260202 02:52:22.609189 1173 tablet_bootstrap.cc:492] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:22.609453 1182 rpcz_store.cc:276] 0202 02:52:20.047575 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:20.047662 (+ 87us) service_pool.cc:225] Handling call
0202 02:52:22.574543 (+2526881us) ts_tablet_manager.cc:1151] Deleting tablet eb0ac55fbb534b29a86d6b3e8aa72ebc
0202 02:52:22.574555 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.593725 (+ 19170us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.596768 (+ 3043us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.608932 (+ 12164us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2524853}
I20260202 02:52:22.610337 1182 tablet_replica.cc:333] T b428356fba0c4fb5a6033c5d39740cc5 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:22.610761 1173 tablet_bootstrap.cc:654] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:22.610807 1182 raft_consensus.cc:2243] T b428356fba0c4fb5a6033c5d39740cc5 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:22.611325 1182 raft_consensus.cc:2272] T b428356fba0c4fb5a6033c5d39740cc5 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:22.612121 1182 ts_tablet_manager.cc:1916] T b428356fba0c4fb5a6033c5d39740cc5 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:22.612833 1175 tablet_bootstrap.cc:492] T 929eb48881b141718873d0e359e768a8 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:22.613039 1175 ts_tablet_manager.cc:1403] T 929eb48881b141718873d0e359e768a8 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.034s user 0.006s sys 0.000s
I20260202 02:52:22.613957 1175 raft_consensus.cc:359] T 929eb48881b141718873d0e359e768a8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.614248 1175 raft_consensus.cc:385] T 929eb48881b141718873d0e359e768a8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.614380 1175 raft_consensus.cc:740] T 929eb48881b141718873d0e359e768a8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.614773 1175 consensus_queue.cc:260] T 929eb48881b141718873d0e359e768a8 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.615374 1175 ts_tablet_manager.cc:1434] T 929eb48881b141718873d0e359e768a8 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:22.615705 1175 tablet_bootstrap.cc:492] T 94bada52db6a47179415d302b541d01f P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:22.617439 1175 tablet_bootstrap.cc:654] T 94bada52db6a47179415d302b541d01f P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:22.619845 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet eb0ac55fbb534b29a86d6b3e8aa72ebc (table default.Table_8f23e3356aa14793a1aad274d2b845eb [id=21c284c2be45496ca760633f24f090d2]) successfully deleted
I20260202 02:52:22.621572 1173 tablet_bootstrap.cc:492] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:22.621738 1173 ts_tablet_manager.cc:1403] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.000s
I20260202 02:52:22.622581 1173 raft_consensus.cc:359] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.622767 1173 raft_consensus.cc:385] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.622869 1173 raft_consensus.cc:740] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.623126 1173 consensus_queue.cc:260] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.623744 1173 ts_tablet_manager.cc:1434] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260202 02:52:22.624195 1173 tablet_bootstrap.cc:492] T d1e0e74f32a24729b250c5751677cfb0 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:22.626227 1173 tablet_bootstrap.cc:654] T d1e0e74f32a24729b250c5751677cfb0 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:22.627633 1178 raft_consensus.cc:493] T 80d443bdb66d4cc68ae283e08c410fe6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:22.627910 1178 raft_consensus.cc:515] T 80d443bdb66d4cc68ae283e08c410fe6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.628360 1175 tablet_bootstrap.cc:492] T 94bada52db6a47179415d302b541d01f P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:22.628578 1175 ts_tablet_manager.cc:1403] T 94bada52db6a47179415d302b541d01f P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.001s
I20260202 02:52:22.629356 1175 raft_consensus.cc:359] T 94bada52db6a47179415d302b541d01f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.629626 1175 raft_consensus.cc:385] T 94bada52db6a47179415d302b541d01f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.629742 1175 raft_consensus.cc:740] T 94bada52db6a47179415d302b541d01f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.630095 1175 consensus_queue.cc:260] T 94bada52db6a47179415d302b541d01f P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.630656 1175 ts_tablet_manager.cc:1434] T 94bada52db6a47179415d302b541d01f P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260202 02:52:22.631014 1175 tablet_bootstrap.cc:492] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:22.631091 1178 leader_election.cc:290] T 80d443bdb66d4cc68ae283e08c410fe6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:22.633354 1182 ts_tablet_manager.cc:1929] T b428356fba0c4fb5a6033c5d39740cc5 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:22.633531 1173 tablet_bootstrap.cc:492] T d1e0e74f32a24729b250c5751677cfb0 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:22.633708 1173 ts_tablet_manager.cc:1403] T d1e0e74f32a24729b250c5751677cfb0 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20260202 02:52:22.634481 1173 raft_consensus.cc:359] T d1e0e74f32a24729b250c5751677cfb0 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.634686 1173 raft_consensus.cc:385] T d1e0e74f32a24729b250c5751677cfb0 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.634769 1173 raft_consensus.cc:740] T d1e0e74f32a24729b250c5751677cfb0 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.635011 1173 consensus_queue.cc:260] T d1e0e74f32a24729b250c5751677cfb0 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.635352 1178 raft_consensus.cc:493] T d1e0e74f32a24729b250c5751677cfb0 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:22.635533 1178 raft_consensus.cc:515] T d1e0e74f32a24729b250c5751677cfb0 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.635880 1173 ts_tablet_manager.cc:1434] T d1e0e74f32a24729b250c5751677cfb0 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260202 02:52:22.636229 1173 tablet_bootstrap.cc:492] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
W20260202 02:52:22.632200 999 leader_election.cc:343] T 80d443bdb66d4cc68ae283e08c410fe6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 80d443bdb66d4cc68ae283e08c410fe6
I20260202 02:52:22.635099 1182 log.cc:1199] T b428356fba0c4fb5a6033c5d39740cc5 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/b428356fba0c4fb5a6033c5d39740cc5
I20260202 02:52:22.638070 1173 tablet_bootstrap.cc:654] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:22.638203 1777 raft_consensus.cc:493] T d64f42d187bc4c9f9381c931449ec0cc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:22.638447 1777 raft_consensus.cc:515] T d64f42d187bc4c9f9381c931449ec0cc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:22.638659 922 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d1e0e74f32a24729b250c5751677cfb0" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:22.639104 1777 leader_election.cc:290] T d64f42d187bc4c9f9381c931449ec0cc P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:22.639614 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d64f42d187bc4c9f9381c931449ec0cc" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:22.639971 1182 ts_tablet_manager.cc:1950] T b428356fba0c4fb5a6033c5d39740cc5 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:22.636662 1178 leader_election.cc:290] T d1e0e74f32a24729b250c5751677cfb0 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:22.640244 859 leader_election.cc:343] T d64f42d187bc4c9f9381c931449ec0cc P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:22.633543 1175 tablet_bootstrap.cc:654] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:22.640877 999 leader_election.cc:343] T d1e0e74f32a24729b250c5751677cfb0 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:22.641606 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 281) took 1725 ms. Trace:
I20260202 02:52:22.643277 1182 rpcz_store.cc:276] 0202 02:52:20.915758 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:20.915804 (+ 46us) service_pool.cc:225] Handling call
0202 02:52:22.610250 (+1694446us) ts_tablet_manager.cc:1151] Deleting tablet b428356fba0c4fb5a6033c5d39740cc5
0202 02:52:22.610261 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.622051 (+ 11790us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.633191 (+ 11140us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.641568 (+ 8377us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1694161}
I20260202 02:52:22.649292 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 283) took 1732 ms. Trace:
I20260202 02:52:22.649803 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet b428356fba0c4fb5a6033c5d39740cc5 (table default.Table_c37ce6d5cf70417c83a68c31d02881ef [id=e87761e6b64a4476a05fcbcf73f0a2f1]) successfully deleted
I20260202 02:52:22.649976 1182 rpcz_store.cc:276] 0202 02:52:20.916736 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:20.916787 (+ 51us) service_pool.cc:225] Handling call
0202 02:52:22.649083 (+1732296us) ts_tablet_manager.cc:1151] Deleting tablet 38301e13e17c41fcae912c4b38c26a45
0202 02:52:22.649100 (+ 17us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.649245 (+ 145us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1732095}
I20260202 02:52:22.650894 1182 tablet_replica.cc:333] T c0937b6910534ddc8a2217f0cbd539c5 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:22.651422 1182 raft_consensus.cc:2243] T c0937b6910534ddc8a2217f0cbd539c5 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:22.651961 1182 raft_consensus.cc:2272] T c0937b6910534ddc8a2217f0cbd539c5 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:22.652946 1182 ts_tablet_manager.cc:1916] T c0937b6910534ddc8a2217f0cbd539c5 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:22.654239 1175 tablet_bootstrap.cc:492] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:22.654500 1175 ts_tablet_manager.cc:1403] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.000s
I20260202 02:52:22.655519 1175 raft_consensus.cc:359] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.655867 1175 raft_consensus.cc:385] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.656020 1175 raft_consensus.cc:740] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.656363 1175 consensus_queue.cc:260] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.658998 1175 ts_tablet_manager.cc:1434] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260202 02:52:22.659482 1175 tablet_bootstrap.cc:492] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:22.660887 1173 tablet_bootstrap.cc:492] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:22.661067 1173 ts_tablet_manager.cc:1403] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.025s user 0.013s sys 0.004s
I20260202 02:52:22.661322 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d64f42d187bc4c9f9381c931449ec0cc" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:22.661936 857 leader_election.cc:343] T d64f42d187bc4c9f9381c931449ec0cc P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:22.662135 857 leader_election.cc:304] T d64f42d187bc4c9f9381c931449ec0cc P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:22.662407 1777 raft_consensus.cc:2749] T d64f42d187bc4c9f9381c931449ec0cc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:22.662358 1173 raft_consensus.cc:359] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.662591 1173 raft_consensus.cc:385] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.662681 1173 raft_consensus.cc:740] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.662881 1173 consensus_queue.cc:260] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.663156 1721 raft_consensus.cc:493] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:22.663316 1721 raft_consensus.cc:515] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.663414 1173 ts_tablet_manager.cc:1434] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:22.663703 1173 tablet_bootstrap.cc:492] T ffd80badce1048f3ac95366cad05ae53 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:22.663975 1721 leader_election.cc:290] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:22.665045 922 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4e79cd48fefb4ce29ef48beacfc4d26f" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:22.665275 922 raft_consensus.cc:2468] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:22.665491 1173 tablet_bootstrap.cc:654] T ffd80badce1048f3ac95366cad05ae53 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:22.666600 1175 tablet_bootstrap.cc:654] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:22.668653 999 leader_election.cc:304] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:22.671092 1721 raft_consensus.cc:2804] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:22.671243 1721 raft_consensus.cc:493] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:22.671342 1721 raft_consensus.cc:3060] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:22.674249 1721 raft_consensus.cc:515] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.675442 1721 leader_election.cc:290] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:22.674448 1764 raft_consensus.cc:493] T 70306ed9dec54286885d965f2c012da4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:22.675786 1764 raft_consensus.cc:515] T 70306ed9dec54286885d965f2c012da4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.675859 922 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4e79cd48fefb4ce29ef48beacfc4d26f" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:22.676122 922 raft_consensus.cc:3060] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:22.676828 1764 leader_election.cc:290] T 70306ed9dec54286885d965f2c012da4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:22.679391 922 raft_consensus.cc:2468] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:22.679883 918 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "70306ed9dec54286885d965f2c012da4" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:22.679958 999 leader_election.cc:304] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:22.680141 918 raft_consensus.cc:2393] T 70306ed9dec54286885d965f2c012da4 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1f05820b6034228bdad884cc348716d in current term 1: Already voted for candidate 1f442ef85029461091de847118c34c0d in this term.
I20260202 02:52:22.682108 1764 raft_consensus.cc:2804] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:22.682368 1764 raft_consensus.cc:697] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:22.682746 1764 consensus_queue.cc:237] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.683084 1182 ts_tablet_manager.cc:1929] T c0937b6910534ddc8a2217f0cbd539c5 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:22.683236 1182 log.cc:1199] T c0937b6910534ddc8a2217f0cbd539c5 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/c0937b6910534ddc8a2217f0cbd539c5
I20260202 02:52:22.683646 1182 ts_tablet_manager.cc:1950] T c0937b6910534ddc8a2217f0cbd539c5 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:22.689281 1175 tablet_bootstrap.cc:492] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:22.689564 1175 ts_tablet_manager.cc:1403] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.030s user 0.004s sys 0.000s
I20260202 02:52:22.690433 1175 raft_consensus.cc:359] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.690718 1175 raft_consensus.cc:385] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.690824 1175 raft_consensus.cc:740] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.691128 1175 consensus_queue.cc:260] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.691728 1175 ts_tablet_manager.cc:1434] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:22.692155 1175 tablet_bootstrap.cc:492] T 3711910e050c4515bed4294d1271d5a9 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:22.693974 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 284) took 1776 ms. Trace:
I20260202 02:52:22.694033 1175 tablet_bootstrap.cc:654] T 3711910e050c4515bed4294d1271d5a9 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:22.694126 1182 rpcz_store.cc:276] 0202 02:52:20.917128 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:20.917169 (+ 41us) service_pool.cc:225] Handling call
0202 02:52:22.650816 (+1733647us) ts_tablet_manager.cc:1151] Deleting tablet c0937b6910534ddc8a2217f0cbd539c5
0202 02:52:22.650830 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.672991 (+ 22161us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.682928 (+ 9937us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.693899 (+ 10971us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1733478}
I20260202 02:52:22.694922 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet c0937b6910534ddc8a2217f0cbd539c5 (table default.Table_668e8ec9913f43fa8499eff32be8db0a [id=46b5d47377474ff9bac2acdb26b08c65]) successfully deleted
I20260202 02:52:22.695529 1182 tablet_replica.cc:333] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:22.696120 1182 raft_consensus.cc:2243] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:22.696653 1182 raft_consensus.cc:2272] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:22.697520 1182 ts_tablet_manager.cc:1916] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:22.698196 1173 tablet_bootstrap.cc:492] T ffd80badce1048f3ac95366cad05ae53 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:22.698390 1173 ts_tablet_manager.cc:1403] T ffd80badce1048f3ac95366cad05ae53 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.035s user 0.019s sys 0.006s
I20260202 02:52:22.699285 1173 raft_consensus.cc:359] T ffd80badce1048f3ac95366cad05ae53 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.699510 1173 raft_consensus.cc:385] T ffd80badce1048f3ac95366cad05ae53 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.700001 1173 raft_consensus.cc:740] T ffd80badce1048f3ac95366cad05ae53 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.700402 1173 consensus_queue.cc:260] T ffd80badce1048f3ac95366cad05ae53 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.701115 1173 ts_tablet_manager.cc:1434] T ffd80badce1048f3ac95366cad05ae53 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260202 02:52:22.701509 1173 tablet_bootstrap.cc:492] T 622a113af04a41c6852c29fc8a8acd78 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:22.703773 1173 tablet_bootstrap.cc:654] T 622a113af04a41c6852c29fc8a8acd78 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:22.708035 1000 connection.cc:537] client connection to 127.0.104.190:33981 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260202 02:52:22.708837 515 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.104.190:33981 (error 108) [suppressed 5 similar messages]
W20260202 02:52:22.709051 859 connection.cc:537] client connection to 127.0.104.190:33981 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260202 02:52:22.716804 1175 tablet_bootstrap.cc:492] T 3711910e050c4515bed4294d1271d5a9 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:22.717113 1175 ts_tablet_manager.cc:1403] T 3711910e050c4515bed4294d1271d5a9 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.025s user 0.010s sys 0.014s
W20260202 02:52:22.717283 714 leader_election.cc:343] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 9884c90cf1d245b2a7eaaf00d71b1c3e
I20260202 02:52:22.717543 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "80d443bdb66d4cc68ae283e08c410fe6" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:22.718107 1175 raft_consensus.cc:359] T 3711910e050c4515bed4294d1271d5a9 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:22.718355 1175 raft_consensus.cc:385] T 3711910e050c4515bed4294d1271d5a9 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.718474 1175 raft_consensus.cc:740] T 3711910e050c4515bed4294d1271d5a9 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.718796 1175 consensus_queue.cc:260] T 3711910e050c4515bed4294d1271d5a9 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:22.718899 525 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_7950bdb0cab248acb118e0ebdf92d83d [id=316b8a276d9b415fafe8511aa7c32fc5] alter complete (version 3)
I20260202 02:52:22.719456 1175 ts_tablet_manager.cc:1434] T 3711910e050c4515bed4294d1271d5a9 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
W20260202 02:52:22.719830 525 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.104.130:36691 (request call id 43) took 1144 ms (client timeout 1000 ms). Trace:
I20260202 02:52:22.720499 1175 tablet_bootstrap.cc:492] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
W20260202 02:52:22.719940 525 rpcz_store.cc:269] 0202 02:52:21.574973 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:21.600316 (+ 25343us) service_pool.cc:225] Handling call
0202 02:52:21.733471 (+133155us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x616000041254. stack: 00007fc023f2da86 00007fc023f2d840 00007fc022f083ad 0000000000984f94 0000000000984f5d 00007fc0332576bd 00007fc0332577b3 00007fc02fe1e521 00007fc037ae96fd 00007fc037c705d8 00007fc02e771ca6 00007fc02e7719d5 00007fc02aef055e 00007fc02aeef43e 00007fc02aef5675 00007fc02aef5f9f
0202 02:52:22.719773 (+986302us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:21.742584 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:21.742662 (+ 78us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:21.742666 (+ 4us) write_op.cc:435] Acquired schema lock
0202 02:52:21.742668 (+ 2us) tablet.cc:660] Decoding operations
0202 02:52:21.742742 (+ 74us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:21.742750 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:21.742754 (+ 4us) tablet.cc:683] Acquiring locks for 1 operations
0202 02:52:21.742797 (+ 43us) tablet.cc:699] Row locks acquired
0202 02:52:21.742800 (+ 3us) write_op.cc:260] PREPARE: finished
0202 02:52:21.742825 (+ 25us) write_op.cc:270] Start()
0202 02:52:21.742848 (+ 23us) write_op.cc:276] Timestamp: P: 1770000741742789 usec, L: 0
0202 02:52:21.742851 (+ 3us) op_driver.cc:348] REPLICATION: starting
0202 02:52:21.742995 (+ 144us) log.cc:844] Serialized 356 byte log entry
0202 02:52:21.755645 (+ 12650us) op_driver.cc:464] REPLICATION: finished
0202 02:52:21.758600 (+ 2955us) write_op.cc:301] APPLY: starting
0202 02:52:21.758610 (+ 10us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:21.780309 (+ 21699us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:21.780454 (+ 145us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:21.780526 (+ 72us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:21.780562 (+ 36us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0202 02:52:21.780569 (+ 7us) write_op.cc:312] APPLY: finished
0202 02:52:21.780651 (+ 82us) log.cc:844] Serialized 23 byte log entry
0202 02:52:21.780746 (+ 95us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:21.780767 (+ 21us) write_op.cc:454] Released schema lock
0202 02:52:21.780775 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0202 02:52:21.780863 (+ 88us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:21.780867 (+ 4us) write_op.cc:454] Released schema lock
Related trace 'op':
0202 02:52:21.855739 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:21.855854 (+ 115us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:21.855858 (+ 4us) write_op.cc:435] Acquired schema lock
0202 02:52:21.856129 (+ 271us) tablet.cc:660] Decoding operations
0202 02:52:21.856240 (+ 111us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:21.856253 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:21.856258 (+ 5us) tablet.cc:683] Acquiring locks for 1 operations
0202 02:52:21.856318 (+ 60us) tablet.cc:699] Row locks acquired
0202 02:52:21.856356 (+ 38us) write_op.cc:260] PREPARE: finished
0202 02:52:21.856397 (+ 41us) write_op.cc:270] Start()
0202 02:52:21.856425 (+ 28us) write_op.cc:276] Timestamp: P: 1770000741856345 usec, L: 0
0202 02:52:21.856428 (+ 3us) op_driver.cc:348] REPLICATION: starting
0202 02:52:21.856610 (+ 182us) log.cc:844] Serialized 356 byte log entry
0202 02:52:22.662834 (+806224us) op_driver.cc:464] REPLICATION: finished
0202 02:52:22.718290 (+ 55456us) write_op.cc:301] APPLY: starting
0202 02:52:22.718311 (+ 21us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:22.718352 (+ 41us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:22.718382 (+ 30us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:22.718451 (+ 69us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:22.718483 (+ 32us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0202 02:52:22.718490 (+ 7us) write_op.cc:312] APPLY: finished
0202 02:52:22.718593 (+ 103us) log.cc:844] Serialized 23 byte log entry
0202 02:52:22.718666 (+ 73us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:22.718686 (+ 20us) write_op.cc:454] Released schema lock
0202 02:52:22.718695 (+ 9us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":11,"spinlock_wait_cycles":304342272,"child_traces":[["op",{"apply.queue_time_us":2927,"apply.run_cpu_time_us":599,"apply.run_wall_time_us":22247,"num_ops":1,"prepare.queue_time_us":2351,"prepare.run_cpu_time_us":463,"prepare.run_wall_time_us":462,"replication_time_us":12782,"spinlock_wait_cycles":768}],["op",{"apply.queue_time_us":55377,"apply.run_cpu_time_us":492,"apply.run_wall_time_us":493,"num_ops":1,"prepare.queue_time_us":557,"prepare.run_cpu_time_us":668,"prepare.run_wall_time_us":941,"replication_time_us":806364,"thread_start_us":55931,"threads_started":1}]]}
I20260202 02:52:22.722859 1175 tablet_bootstrap.cc:654] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:22.723503 520 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 68823eff1bc04108ba83487605aec0ea
W20260202 02:52:22.724020 527 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.104.131:43527 (request call id 76) took 792 ms (client timeout 1000 ms). Trace:
W20260202 02:52:22.724146 527 rpcz_store.cc:269] 0202 02:52:21.931508 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:21.931579 (+ 71us) service_pool.cc:225] Handling call
0202 02:52:22.723932 (+792353us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":14}
I20260202 02:52:22.724511 893 tablet_service.cc:1552] Processing DeleteTablet for tablet 75b97d7e848241cf8d894b0ae92a8f8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:22.724776 899 tablet_service.cc:1552] Processing DeleteTablet for tablet 723921dfe54c4e66af5cab76a25fee0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:22.724959 886 tablet_service.cc:1552] Processing DeleteTablet for tablet d2fa4c3d6ec44047abf5986e502c1590 with delete_type TABLET_DATA_DELETED (replaced by tablet a0b5d82e5fb94b60b4aa324b978a5378) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:22.725118 894 tablet_service.cc:1552] Processing DeleteTablet for tablet 61a9203035f2495f8f99924303ebf120 with delete_type TABLET_DATA_DELETED (replaced by tablet 4c63192b273b462a9c1a0e530e938024) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:22.728159 528 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 79f694b740954d429145fba9157338ca
I20260202 02:52:22.729243 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 68823eff1bc04108ba83487605aec0ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:21 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:22.730660 894 tablet_service.cc:1552] Processing DeleteTablet for tablet 79f694b740954d429145fba9157338ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:21 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:22.732725 1173 tablet_bootstrap.cc:492] T 622a113af04a41c6852c29fc8a8acd78 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:22.732970 1173 ts_tablet_manager.cc:1403] T 622a113af04a41c6852c29fc8a8acd78 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.032s user 0.006s sys 0.000s
I20260202 02:52:22.733207 526 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 59ad016e15ec4a488921eb564029b689
I20260202 02:52:22.733922 526 master_service.cc:954] ReplaceTablet: received request to replace tablet 08d8d781bb4e4acf915f11afb4fb2e22 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:22.733961 1173 raft_consensus.cc:359] T 622a113af04a41c6852c29fc8a8acd78 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:22.734323 1173 raft_consensus.cc:385] T 622a113af04a41c6852c29fc8a8acd78 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.734457 1173 raft_consensus.cc:740] T 622a113af04a41c6852c29fc8a8acd78 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.734665 894 tablet_service.cc:1552] Processing DeleteTablet for tablet 59ad016e15ec4a488921eb564029b689 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:21 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:22.735162 1173 consensus_queue.cc:260] T 622a113af04a41c6852c29fc8a8acd78 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:22.735673 1175 tablet_bootstrap.cc:492] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:22.735800 1173 ts_tablet_manager.cc:1434] T 622a113af04a41c6852c29fc8a8acd78 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260202 02:52:22.735937 1175 ts_tablet_manager.cc:1403] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.007s
I20260202 02:52:22.736104 1173 tablet_bootstrap.cc:492] T d64f42d187bc4c9f9381c931449ec0cc P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:22.736268 1182 ts_tablet_manager.cc:1929] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:22.736467 1182 log.cc:1199] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/583e547d0a1943f09f4efe82070fc777
I20260202 02:52:22.736960 1175 raft_consensus.cc:359] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.737251 1175 raft_consensus.cc:385] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.737370 1175 raft_consensus.cc:740] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.737663 1175 consensus_queue.cc:260] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.738049 1173 tablet_bootstrap.cc:654] T d64f42d187bc4c9f9381c931449ec0cc P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:22.738296 1175 ts_tablet_manager.cc:1434] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:22.738670 1175 tablet_bootstrap.cc:492] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:22.740494 1175 tablet_bootstrap.cc:654] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:22.736969 1182 ts_tablet_manager.cc:1950] T 583e547d0a1943f09f4efe82070fc777 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:22.741871 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "70306ed9dec54286885d965f2c012da4" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:22.742399 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 282) took 1826 ms. Trace:
I20260202 02:52:22.742524 1182 rpcz_store.cc:276] 0202 02:52:20.916249 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:20.916310 (+ 61us) service_pool.cc:225] Handling call
0202 02:52:22.695427 (+1779117us) ts_tablet_manager.cc:1151] Deleting tablet 583e547d0a1943f09f4efe82070fc777
0202 02:52:22.695440 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.702214 (+ 6774us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.736109 (+ 33895us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.742349 (+ 6240us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1772076}
I20260202 02:52:22.743041 1182 tablet_replica.cc:333] T ff6f5710de4b4c48948aacdf5be29e8b P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:22.743264 1182 raft_consensus.cc:2243] T ff6f5710de4b4c48948aacdf5be29e8b P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:22.743427 1182 pending_rounds.cc:70] T ff6f5710de4b4c48948aacdf5be29e8b P 1521a6d262b7451bbef8debe970dd6b5: Trying to abort 1 pending ops.
I20260202 02:52:22.743490 1182 pending_rounds.cc:77] T ff6f5710de4b4c48948aacdf5be29e8b P 1521a6d262b7451bbef8debe970dd6b5: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7249923035851276288 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "ff6f5710de4b4c48948aacdf5be29e8b" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 2 new_table_name: "default.Table_ad048ed673a5401b8ffa739c0f03b545" dest_uuid: "1f442ef85029461091de847118c34c0d" new_extra_config { } }
I20260202 02:52:22.743877 1182 raft_consensus.cc:2272] T ff6f5710de4b4c48948aacdf5be29e8b P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:22.743880 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 583e547d0a1943f09f4efe82070fc777 (table default.Table_2669ca9662b942c08bcbfd7641cfe2e3 [id=fb813e4c5b0849f8a488c7a22d1defae]) successfully deleted
I20260202 02:52:22.744364 521 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet a5622ef925bc46cca76f481ddfd12afd
I20260202 02:52:22.745555 1182 ts_tablet_manager.cc:1916] T ff6f5710de4b4c48948aacdf5be29e8b P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:22.746950 894 tablet_service.cc:1552] Processing DeleteTablet for tablet a5622ef925bc46cca76f481ddfd12afd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:21 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:22.752445 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d1e0e74f32a24729b250c5751677cfb0" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:22.754869 998 leader_election.cc:343] T d1e0e74f32a24729b250c5751677cfb0 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:22.755085 998 leader_election.cc:304] T d1e0e74f32a24729b250c5751677cfb0 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:22.755892 1764 raft_consensus.cc:2749] T d1e0e74f32a24729b250c5751677cfb0 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:22.756126 1173 tablet_bootstrap.cc:492] T d64f42d187bc4c9f9381c931449ec0cc P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:22.756407 1173 ts_tablet_manager.cc:1403] T d64f42d187bc4c9f9381c931449ec0cc P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.002s
I20260202 02:52:22.757314 1173 raft_consensus.cc:359] T d64f42d187bc4c9f9381c931449ec0cc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:22.757532 1173 raft_consensus.cc:385] T d64f42d187bc4c9f9381c931449ec0cc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.757519 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4e79cd48fefb4ce29ef48beacfc4d26f" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:22.757599 1173 raft_consensus.cc:740] T d64f42d187bc4c9f9381c931449ec0cc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.757853 1173 consensus_queue.cc:260] T d64f42d187bc4c9f9381c931449ec0cc P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:22.758417 1173 ts_tablet_manager.cc:1434] T d64f42d187bc4c9f9381c931449ec0cc P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260202 02:52:22.758786 1173 tablet_bootstrap.cc:492] T d67e91a022f1401ea724dff898c3b81b P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:22.758877 775 raft_consensus.cc:2393] T 80d443bdb66d4cc68ae283e08c410fe6 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1f05820b6034228bdad884cc348716d in current term 1: Already voted for candidate 1f442ef85029461091de847118c34c0d in this term.
I20260202 02:52:22.759202 714 leader_election.cc:304] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
W20260202 02:52:22.759568 714 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 89315.4us
I20260202 02:52:22.759970 714 leader_election.cc:304] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
W20260202 02:52:22.760345 714 leader_election.cc:343] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: f58a7b25815c4b5bac568aa11c19610c
I20260202 02:52:22.760710 714 leader_election.cc:304] T 80d443bdb66d4cc68ae283e08c410fe6 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:22.761166 714 leader_election.cc:304] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:22.761523 1738 raft_consensus.cc:2749] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:22.761900 1738 raft_consensus.cc:2804] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:22.762001 1738 raft_consensus.cc:493] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:22.762085 1738 raft_consensus.cc:3060] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:22.763094 1723 raft_consensus.cc:2749] T 80d443bdb66d4cc68ae283e08c410fe6 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260202 02:52:22.763311 1723 raft_consensus.cc:2804] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:22.763412 1723 raft_consensus.cc:697] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:22.756970 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4e79cd48fefb4ce29ef48beacfc4d26f" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:22.765323 1182 ts_tablet_manager.cc:1929] T ff6f5710de4b4c48948aacdf5be29e8b P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260202 02:52:22.765502 1182 log.cc:1199] T ff6f5710de4b4c48948aacdf5be29e8b P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/ff6f5710de4b4c48948aacdf5be29e8b
I20260202 02:52:22.765991 1182 ts_tablet_manager.cc:1950] T ff6f5710de4b4c48948aacdf5be29e8b P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:22.766973 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 286) took 1801 ms. Trace:
I20260202 02:52:22.768489 1175 tablet_bootstrap.cc:492] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:22.768630 1175 ts_tablet_manager.cc:1403] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.030s user 0.004s sys 0.022s
I20260202 02:52:22.769519 1175 raft_consensus.cc:359] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.769706 1175 raft_consensus.cc:385] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.769799 1175 raft_consensus.cc:740] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.770040 1175 consensus_queue.cc:260] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.770551 1175 ts_tablet_manager.cc:1434] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:22.770841 1175 tablet_bootstrap.cc:492] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
W20260202 02:52:22.772193 998 leader_election.cc:343] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:22.772866 1175 tablet_bootstrap.cc:654] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:22.775404 1173 tablet_bootstrap.cc:654] T d67e91a022f1401ea724dff898c3b81b P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:22.771147 715 connection.cc:537] client connection to 127.0.104.190:33981 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260202 02:52:22.767102 1182 rpcz_store.cc:276] 0202 02:52:20.965451 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:20.967779 (+ 2328us) service_pool.cc:225] Handling call
0202 02:52:22.742963 (+1775184us) ts_tablet_manager.cc:1151] Deleting tablet ff6f5710de4b4c48948aacdf5be29e8b
0202 02:52:22.742973 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.761601 (+ 18628us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.765159 (+ 3558us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.766900 (+ 1741us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1774971}
W20260202 02:52:22.779331 998 leader_election.cc:343] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:22.779992 998 leader_election.cc:304] T 80d443bdb66d4cc68ae283e08c410fe6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:22.780506 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet ff6f5710de4b4c48948aacdf5be29e8b (table default.Table_ad048ed673a5401b8ffa739c0f03b545 [id=f2276e1c2ca04a70804053da61dd3447]) successfully deleted
I20260202 02:52:22.780542 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 288) took 1793 ms. Trace:
I20260202 02:52:22.780668 1182 rpcz_store.cc:276] 0202 02:52:20.987411 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:20.992198 (+ 4787us) service_pool.cc:225] Handling call
0202 02:52:22.780274 (+1788076us) ts_tablet_manager.cc:1151] Deleting tablet 06b80a1c9251420cbc16fc95ae98aca9
0202 02:52:22.780287 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.780473 (+ 186us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1787743}
I20260202 02:52:22.782316 1178 raft_consensus.cc:3060] T 80d443bdb66d4cc68ae283e08c410fe6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:22.784713 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 291) took 1784 ms. Trace:
I20260202 02:52:22.784832 1182 rpcz_store.cc:276] 0202 02:52:21.000663 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:21.013941 (+ 13278us) service_pool.cc:225] Handling call
0202 02:52:22.784478 (+1770537us) ts_tablet_manager.cc:1151] Deleting tablet 072bb7f0e4ab4935be49461bc8a3c163
0202 02:52:22.784496 (+ 18us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.784639 (+ 143us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1770187}
I20260202 02:52:22.785068 1178 raft_consensus.cc:2749] T 80d443bdb66d4cc68ae283e08c410fe6 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:22.785640 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 072bb7f0e4ab4935be49461bc8a3c163 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 072bb7f0e4ab4935be49461bc8a3c163 already in progress: creating tablet
I20260202 02:52:22.790247 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_bb45d0e2e2a14038aec12db59746877a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:22.790910 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bb45d0e2e2a14038aec12db59746877a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:22.790910 519 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_80e6b91d5978450dbfd9bb2b4b410d3c" } new_table_name: "default.Table_37b8a9e1666a4ddbaf5a13d828e9702c" modify_external_catalogs: true
I20260202 02:52:22.792517 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_24d7cc37a3884cda9a1e269515d3877d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:22.792613 528 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_408b8f21ff294ce9880d0b9a4d681f93" } modify_external_catalogs: true
I20260202 02:52:22.792846 528 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_408b8f21ff294ce9880d0b9a4d681f93" } modify_external_catalogs: true
W20260202 02:52:22.793071 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_24d7cc37a3884cda9a1e269515d3877d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:22.793453 525 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet b76d1837813c46e8a0925b49939d8884 (table default.Table_3919d090261e4b90b0672f2b863588b2 [id=fe7b88b74c864c2f98365c0956117bf1]). Expected version 1 got 0
W20260202 02:52:22.794071 524 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2c31b45dc8004eae993e972e9faa3ff8 consensus state has no leader
I20260202 02:52:22.794559 524 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_5ed512e7fc01454392b327c54420ec55" } new_table_name: "default.Table_96e21d4e03c941129f0581aa558481c0" modify_external_catalogs: true
I20260202 02:52:22.788246 1723 consensus_queue.cc:237] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.798255 525 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet d06b77ff1a6f4f78b73082b00145fcf6 (table default.Table_63e36ff851f849849172c3c8e473214e [id=915b211258ad498286448a5b2598a289]). Expected version 3 got 0
I20260202 02:52:22.798180 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_3a2daddab50e4578898828f3a305b8e7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:22.798928 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3a2daddab50e4578898828f3a305b8e7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:22.804600 779 raft_consensus.cc:2393] T 70306ed9dec54286885d965f2c012da4 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1f05820b6034228bdad884cc348716d in current term 1: Already voted for candidate 1f442ef85029461091de847118c34c0d in this term.
I20260202 02:52:22.804697 525 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_35c5c99502d0489081751bc7403f6ad7" } new_table_name: "default.Table_6cf38c67711747acb1b5be35632b7a68" modify_external_catalogs: true
I20260202 02:52:22.806527 1182 tablet_replica.cc:333] T 3711910e050c4515bed4294d1271d5a9 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:22.806975 1182 raft_consensus.cc:2243] T 3711910e050c4515bed4294d1271d5a9 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:22.807525 1182 raft_consensus.cc:2272] T 3711910e050c4515bed4294d1271d5a9 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:22.807837 1174 tablet_bootstrap.cc:654] T 75b97d7e848241cf8d894b0ae92a8f8f P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:22.808813 1182 ts_tablet_manager.cc:1916] T 3711910e050c4515bed4294d1271d5a9 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:22.810026 998 leader_election.cc:304] T 70306ed9dec54286885d965f2c012da4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
W20260202 02:52:22.832675 513 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 2c31b45dc8004eae993e972e9faa3ff8 (table default.Table_5db99844c1bb484a863a9e289a19da3e [id=f1f6c97fc88340da8bab0347ad6430ab], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 2c31b45dc8004eae993e972e9faa3ff8 consensus state has no leader
I20260202 02:52:22.833252 1178 raft_consensus.cc:3060] T 70306ed9dec54286885d965f2c012da4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:22.833554 520 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 5986676f5484498da927b4c6da3f47ff (table default.Table_755b6ce0796b45b8be9df04e59dfe565 [id=d8485f23f63b493781bea557dbe8a0c1]). Expected version 2 got 1
I20260202 02:52:22.834002 520 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 1f54c1dfe0a942b3bafd85522953a032 (table default.Table_a22f252ee8d44be5a9d7202afb726cfc [id=891d9355855640a28939b84b2dd30537]). Expected version 1 got 0
I20260202 02:52:22.834228 1182 ts_tablet_manager.cc:1929] T 3711910e050c4515bed4294d1271d5a9 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:22.834383 1182 log.cc:1199] T 3711910e050c4515bed4294d1271d5a9 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/3711910e050c4515bed4294d1271d5a9
I20260202 02:52:22.835326 520 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet ef073880898049e2be93d2cb6dc0c0bb (table default.Table_e8e9b4cdaf434b6cbbfed1a65bff1fc0 [id=78a7fb3a2eed4565b55d5dacb8a3d7df]). Expected version 1 got 0
I20260202 02:52:22.835654 520 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet d64f42d187bc4c9f9381c931449ec0cc (table default.Table_0c83932cab13463f9cca28129f18b576 [id=7c76848b98614fbf8b2950e5d55cc609]). Expected version 2 got 0
I20260202 02:52:22.836069 1178 raft_consensus.cc:2749] T 70306ed9dec54286885d965f2c012da4 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:22.839545 523 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 78f122342dd144b48de89902faa0f94f (table default.Table_5eefa56618e84f8d92f8d87132ccd7d4 [id=b4ab2ba86e1a49338a98d855ded99f5d]). Expected version 1 got 0
I20260202 02:52:22.839865 523 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet ef073880898049e2be93d2cb6dc0c0bb (table default.Table_e8e9b4cdaf434b6cbbfed1a65bff1fc0 [id=78a7fb3a2eed4565b55d5dacb8a3d7df]). Expected version 1 got 0
I20260202 02:52:22.840143 523 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 70306ed9dec54286885d965f2c012da4 (table default.Table_d0e51ff8e83f4da19ce3c119dbec1ea7 [id=31abb0facd164caaa15591259eb951c2]). Expected version 1 got 0
I20260202 02:52:22.840583 523 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 1f54c1dfe0a942b3bafd85522953a032 (table default.Table_a22f252ee8d44be5a9d7202afb726cfc [id=891d9355855640a28939b84b2dd30537]). Expected version 1 got 0
I20260202 02:52:22.842053 523 catalog_manager.cc:5654] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d reported cstate change: term changed from 0 to 1, leader changed from <none> to b1f05820b6034228bdad884cc348716d (127.0.104.131). New cstate: current_term: 1 leader_uuid: "b1f05820b6034228bdad884cc348716d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } }
W20260202 02:52:22.843160 519 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f3eebefa74584c1791d52377f5ba1290 consensus state has no leader
I20260202 02:52:22.841404 894 tablet_service.cc:1505] Processing CreateTablet for tablet c17c85759e964f0ba3abcaf5fe938fba (DEFAULT_TABLE table=default.Table_67001aa310ae44c0bb9e2b24bb0b57b8 [id=c17d62a148094547af24de85773a1b8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.843982 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c17c85759e964f0ba3abcaf5fe938fba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.847378 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_ed899eb11436405fa74f3b38d08552f7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:22.847965 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ed899eb11436405fa74f3b38d08552f7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:22.849126 886 tablet_service.cc:1505] Processing CreateTablet for tablet 4a518e8ad2e34ec98a0425c7ed3a470a (DEFAULT_TABLE table=default.Table_3dcd1dea44ed4e54a89f4413e7f17dbe [id=f868f7fd84f247c89a963da591f16c2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.849344 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_f97ead9efdf047a2a4fc35c017853fb5" } new_table_name: "default.Table_cafbdde19f984e0fa22b1fde486f8e7e" modify_external_catalogs: true
I20260202 02:52:22.849651 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a518e8ad2e34ec98a0425c7ed3a470a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.850666 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_aafa37fc93f045a2b0525cea42a60866"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:22.851279 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aafa37fc93f045a2b0525cea42a60866 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:22.852226 524 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 474df41e4fe24a818fa45a4ce84d5474 consensus state has no leader
I20260202 02:52:22.852736 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_ac6e92f3b1f540a7af72ae905e0481a7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:22.853075 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_44086d9f3e324314af6a63c86ade13c9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:22.853308 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ac6e92f3b1f540a7af72ae905e0481a7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:22.853596 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_44086d9f3e324314af6a63c86ade13c9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:22.854893 899 tablet_service.cc:1505] Processing CreateTablet for tablet 7f9dfac19c04475aadb45e95a6474eb4 (DEFAULT_TABLE table=default.Table_c2f3ac0894bd4f4f9702b6429da397bc [id=67e7356eb4d0432f884247749fb97bac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.855405 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f9dfac19c04475aadb45e95a6474eb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.836534 1721 raft_consensus.cc:493] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:22.866046 1721 raft_consensus.cc:515] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.866555 893 tablet_service.cc:1505] Processing CreateTablet for tablet 7f323175ee254df789bcb04e65047dff (DEFAULT_TABLE table=default.Table_783c68f0f6ad41899a919edefc2e85df [id=0d6674bc4f264a36aad4ea7962a9e0e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.866818 1721 leader_election.cc:290] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:22.867017 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f323175ee254df789bcb04e65047dff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.845556 1764 raft_consensus.cc:493] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:22.867571 1764 raft_consensus.cc:515] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.868382 1764 leader_election.cc:290] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:22.872735 889 tablet_service.cc:1505] Processing CreateTablet for tablet 9a1d7d173c41462c8e32a95b494bb708 (DEFAULT_TABLE table=default.Table_28e7d9aa8131413e9a174756c246d164 [id=ec07c59893cf45c9b2a5eece556d10c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.873178 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a1d7d173c41462c8e32a95b494bb708. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.876751 902 tablet_service.cc:1505] Processing CreateTablet for tablet b9fc33e8efc04734a951f7fcf9048142 (DEFAULT_TABLE table=default.Table_fbe3212129904307af7133ead4053a59 [id=0aeb16c903f642d8bfa236a96e203767]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.877271 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9fc33e8efc04734a951f7fcf9048142. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:22.882042 999 leader_election.cc:343] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: f93e8d3c4faf48eaa98c3cbf88d585e2
W20260202 02:52:22.882364 525 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ac2ab12f5040488aa306cb69b107b485 consensus state has no leader
W20260202 02:52:22.882356 999 leader_election.cc:343] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 9884c90cf1d245b2a7eaaf00d71b1c3e
I20260202 02:52:22.882699 525 master_service.cc:954] ReplaceTablet: received request to replace tablet c12ff20ca6b44ddba0c936a28cfb2eb9 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:22.883708 528 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 2b352d9a815e4bccafb7e2251500c5bc
I20260202 02:52:22.884346 528 master_service.cc:954] ReplaceTablet: received request to replace tablet dada209d982d48f3b74a49905f19c5b6 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:22.885468 1173 tablet_bootstrap.cc:492] T d67e91a022f1401ea724dff898c3b81b P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:22.885624 1173 ts_tablet_manager.cc:1403] T d67e91a022f1401ea724dff898c3b81b P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.127s user 0.012s sys 0.004s
I20260202 02:52:22.886011 898 tablet_service.cc:1552] Processing DeleteTablet for tablet 2b352d9a815e4bccafb7e2251500c5bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:22 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:22.886467 1173 raft_consensus.cc:359] T d67e91a022f1401ea724dff898c3b81b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:22.886667 1173 raft_consensus.cc:385] T d67e91a022f1401ea724dff898c3b81b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.886776 1173 raft_consensus.cc:740] T d67e91a022f1401ea724dff898c3b81b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.887043 1173 consensus_queue.cc:260] T d67e91a022f1401ea724dff898c3b81b P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:22.887567 1173 ts_tablet_manager.cc:1434] T d67e91a022f1401ea724dff898c3b81b P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:22.888746 1173 tablet_bootstrap.cc:492] T 929eb48881b141718873d0e359e768a8 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:22.892460 1764 raft_consensus.cc:493] T 75b97d7e848241cf8d894b0ae92a8f8f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:22.892761 1764 raft_consensus.cc:515] T 75b97d7e848241cf8d894b0ae92a8f8f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.893757 1764 leader_election.cc:290] T 75b97d7e848241cf8d894b0ae92a8f8f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:22.894170 1173 tablet_bootstrap.cc:654] T 929eb48881b141718873d0e359e768a8 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:22.894143 922 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "75b97d7e848241cf8d894b0ae92a8f8f" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:22.894349 922 raft_consensus.cc:2468] T 75b97d7e848241cf8d894b0ae92a8f8f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:22.895097 999 leader_election.cc:304] T 75b97d7e848241cf8d894b0ae92a8f8f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:22.898795 1764 raft_consensus.cc:2804] T 75b97d7e848241cf8d894b0ae92a8f8f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:22.899039 1764 raft_consensus.cc:493] T 75b97d7e848241cf8d894b0ae92a8f8f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:22.899220 1764 raft_consensus.cc:3060] T 75b97d7e848241cf8d894b0ae92a8f8f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:22.900110 525 master_service.cc:954] ReplaceTablet: received request to replace tablet d64f42d187bc4c9f9381c931449ec0cc from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:22.902082 526 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 08d8d781bb4e4acf915f11afb4fb2e22
I20260202 02:52:22.902565 526 catalog_manager.cc:6724] ReplaceTablet: tablet 08d8d781bb4e4acf915f11afb4fb2e22 deleted and replaced by tablet 7921daa000f54c8dbed04473e21cf0a5
I20260202 02:52:22.902810 526 master_service.cc:954] ReplaceTablet: received request to replace tablet 26eeef6eebbf4f2cb44f69255fb37594 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:22.903918 1178 raft_consensus.cc:493] T ffd80badce1048f3ac95366cad05ae53 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:22.904211 1178 raft_consensus.cc:515] T ffd80badce1048f3ac95366cad05ae53 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.905251 1178 leader_election.cc:290] T ffd80badce1048f3ac95366cad05ae53 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:22.905573 922 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ffd80badce1048f3ac95366cad05ae53" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:22.905812 922 raft_consensus.cc:2468] T ffd80badce1048f3ac95366cad05ae53 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:22.906289 999 leader_election.cc:304] T ffd80badce1048f3ac95366cad05ae53 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:22.906754 1178 raft_consensus.cc:2804] T ffd80badce1048f3ac95366cad05ae53 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:22.906915 1178 raft_consensus.cc:493] T ffd80badce1048f3ac95366cad05ae53 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:22.907141 1178 raft_consensus.cc:3060] T ffd80badce1048f3ac95366cad05ae53 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:22.907246 898 tablet_service.cc:1505] Processing CreateTablet for tablet c977c146bac34638874b7a990604a0d5 (DEFAULT_TABLE table=default.Table_61741238f8ad47cbb8fe712419268ff0 [id=6b1b33ede4134cbe83d015e4b958f1a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.907732 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c977c146bac34638874b7a990604a0d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.909657 1173 tablet_bootstrap.cc:492] T 929eb48881b141718873d0e359e768a8 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:22.909827 1173 ts_tablet_manager.cc:1403] T 929eb48881b141718873d0e359e768a8 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.001s
I20260202 02:52:22.911331 1175 tablet_bootstrap.cc:492] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:22.911487 1175 ts_tablet_manager.cc:1403] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.141s user 0.048s sys 0.054s
I20260202 02:52:22.912388 1175 raft_consensus.cc:359] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.912590 1175 raft_consensus.cc:385] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:22.912662 1175 raft_consensus.cc:740] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:22.912914 1175 consensus_queue.cc:260] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:22.913476 1175 ts_tablet_manager.cc:1434] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260202 02:52:22.914399 893 tablet_service.cc:1505] Processing CreateTablet for tablet cd1b6662da254889ae4bbe8d0b1ef1bb (DEFAULT_TABLE table=default.Table_b0e344dc1a824cd9b41aa8991b7911dd [id=8e640d1261014706a067f522bda7ea61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.915025 520 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_63e36ff851f849849172c3c8e473214e [id=915b211258ad498286448a5b2598a289] alter complete (version 3)
I20260202 02:52:22.915551 889 tablet_service.cc:1505] Processing CreateTablet for tablet bbcc723a8079443db455ae20fb0d585d (DEFAULT_TABLE table=default.Table_93c8b5e956214e88abc353bfb46c8757 [id=1cb84b11b2f34287bbade3af6cd4df67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:22.915978 894 tablet_service.cc:1552] Processing DeleteTablet for tablet 08d8d781bb4e4acf915f11afb4fb2e22 with delete_type TABLET_DATA_DELETED (replaced by tablet 7921daa000f54c8dbed04473e21cf0a5) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:22.916491 1182 ts_tablet_manager.cc:1950] T 3711910e050c4515bed4294d1271d5a9 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:22.917532 901 tablet_service.cc:1552] Processing DeleteTablet for tablet ff6f5710de4b4c48948aacdf5be29e8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:22.917774 884 tablet_service.cc:1552] Processing DeleteTablet for tablet ffd80badce1048f3ac95366cad05ae53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:22.917950 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 583e547d0a1943f09f4efe82070fc777 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:22.918112 894 tablet_service.cc:1552] Processing DeleteTablet for tablet d67e91a022f1401ea724dff898c3b81b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:22.920838 1175 tablet_bootstrap.cc:492] T 59990791123c4d4eb620090a1e8e31e1 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:22.920974 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd1b6662da254889ae4bbe8d0b1ef1bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.922916 1175 tablet_bootstrap.cc:654] T 59990791123c4d4eb620090a1e8e31e1 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:22.926277 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbcc723a8079443db455ae20fb0d585d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:22.934051 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 292) took 1918 ms. Trace:
I20260202 02:52:22.934237 1182 rpcz_store.cc:276] 0202 02:52:21.015187 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:21.015246 (+ 59us) service_pool.cc:225] Handling call
0202 02:52:22.806089 (+1790843us) ts_tablet_manager.cc:1151] Deleting tablet 3711910e050c4515bed4294d1271d5a9
0202 02:52:22.806105 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.813133 (+ 7028us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.834035 (+ 20902us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.934007 (+ 99972us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1790598}
I20260202 02:52:22.934793 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 294) took 1917 ms. Trace:
I20260202 02:52:22.934854 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 3711910e050c4515bed4294d1271d5a9 (table default.Table_ab29f752f1e140da9038d9e11ee61f39 [id=9ef504467a544f299b48499e49adf27f]) successfully deleted
I20260202 02:52:22.934937 1182 rpcz_store.cc:276] 0202 02:52:21.016875 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:21.016926 (+ 51us) service_pool.cc:225] Handling call
0202 02:52:22.934649 (+1917723us) ts_tablet_manager.cc:1151] Deleting tablet 38fdbc6df7ec40b9afe4e0dfe0119e3e
0202 02:52:22.934660 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.934757 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1917506}
I20260202 02:52:22.935354 1182 tablet_replica.cc:333] T c63a670d5c5e489897bc6dbb142b844d P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
W20260202 02:52:22.935346 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 38fdbc6df7ec40b9afe4e0dfe0119e3e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 38fdbc6df7ec40b9afe4e0dfe0119e3e already in progress: creating tablet
I20260202 02:52:22.935652 1182 raft_consensus.cc:2243] T c63a670d5c5e489897bc6dbb142b844d P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:22.935870 1182 raft_consensus.cc:2272] T c63a670d5c5e489897bc6dbb142b844d P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:22.936553 1182 ts_tablet_manager.cc:1916] T c63a670d5c5e489897bc6dbb142b844d P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:22.936992 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9884c90cf1d245b2a7eaaf00d71b1c3e" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:22.938117 775 raft_consensus.cc:2468] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:22.939077 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ffd80badce1048f3ac95366cad05ae53" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:22.939477 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f93e8d3c4faf48eaa98c3cbf88d585e2" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:22.940111 779 raft_consensus.cc:2468] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:22.945398 1180 ts_tablet_manager.cc:1929] T c064ea0dade840b4840a5715c2d83f69 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:22.946743 1182 ts_tablet_manager.cc:1929] T c63a670d5c5e489897bc6dbb142b844d P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:22.945551 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "75b97d7e848241cf8d894b0ae92a8f8f" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:22.947094 1182 log.cc:1199] T c63a670d5c5e489897bc6dbb142b844d P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/c63a670d5c5e489897bc6dbb142b844d
I20260202 02:52:22.947538 1182 ts_tablet_manager.cc:1950] T c63a670d5c5e489897bc6dbb142b844d P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:22.948442 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 296) took 1930 ms. Trace:
I20260202 02:52:22.948657 1182 rpcz_store.cc:276] 0202 02:52:21.018062 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:21.018116 (+ 54us) service_pool.cc:225] Handling call
0202 02:52:22.935288 (+1917172us) ts_tablet_manager.cc:1151] Deleting tablet c63a670d5c5e489897bc6dbb142b844d
0202 02:52:22.935298 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.942362 (+ 7064us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.946618 (+ 4256us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.948372 (+ 1754us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1916985}
I20260202 02:52:22.949322 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet c63a670d5c5e489897bc6dbb142b844d (table default.Table_12d68a317c8e4d809dbbe77c095a1f6e [id=7219522ffc3849faab94d754a42a8511]) successfully deleted
I20260202 02:52:22.949424 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 297) took 1930 ms. Trace:
I20260202 02:52:22.949505 1277 raft_consensus.cc:515] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:22.949633 1182 rpcz_store.cc:276] 0202 02:52:21.018535 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:21.018584 (+ 49us) service_pool.cc:225] Handling call
0202 02:52:22.949211 (+1930627us) ts_tablet_manager.cc:1151] Deleting tablet d0671586e45e45ffb7fd9e1be6e2bbf8
0202 02:52:22.949224 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.949363 (+ 139us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1930461}
I20260202 02:52:22.950320 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 302) took 1887 ms. Trace:
W20260202 02:52:22.950532 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet d0671586e45e45ffb7fd9e1be6e2bbf8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d0671586e45e45ffb7fd9e1be6e2bbf8 already in progress: creating tablet
I20260202 02:52:22.950470 1182 rpcz_store.cc:276] 0202 02:52:21.063478 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:21.064010 (+ 532us) service_pool.cc:225] Handling call
0202 02:52:22.950138 (+1886128us) ts_tablet_manager.cc:1151] Deleting tablet 026e6ee0b3ae4d938fdf1550044ed1fa
0202 02:52:22.950150 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.950261 (+ 111us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1885760}
I20260202 02:52:22.950812 1738 raft_consensus.cc:515] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
W20260202 02:52:22.950937 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 026e6ee0b3ae4d938fdf1550044ed1fa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 026e6ee0b3ae4d938fdf1550044ed1fa already in progress: creating tablet
I20260202 02:52:22.951094 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 313) took 1784 ms. Trace:
I20260202 02:52:22.951184 1182 rpcz_store.cc:276] 0202 02:52:21.167082 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:21.168751 (+ 1669us) service_pool.cc:225] Handling call
0202 02:52:22.950935 (+1782184us) ts_tablet_manager.cc:1151] Deleting tablet d30557fbe9084cd3860820f69dda8752
0202 02:52:22.950946 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.951024 (+ 78us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1768221}
W20260202 02:52:22.951519 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet d30557fbe9084cd3860820f69dda8752 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d30557fbe9084cd3860820f69dda8752 already in progress: creating tablet
I20260202 02:52:22.951642 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 330) took 1248 ms. Trace:
I20260202 02:52:22.951843 1182 rpcz_store.cc:276] 0202 02:52:21.702972 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:21.708387 (+ 5415us) service_pool.cc:225] Handling call
0202 02:52:22.951508 (+1243121us) ts_tablet_manager.cc:1151] Deleting tablet 415d14533fd54f20adb00455a7bcad5f
0202 02:52:22.951518 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.951587 (+ 69us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1242845}
W20260202 02:52:22.952286 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 415d14533fd54f20adb00455a7bcad5f because the tablet was not found. No further retry: Not found: Tablet not found: 415d14533fd54f20adb00455a7bcad5f
I20260202 02:52:22.952466 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 336) took 1240 ms. Trace:
I20260202 02:52:22.952561 1182 rpcz_store.cc:276] 0202 02:52:21.711866 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:21.711919 (+ 53us) service_pool.cc:225] Handling call
0202 02:52:22.952282 (+1240363us) ts_tablet_manager.cc:1151] Deleting tablet 2f5aae089fcc492f9ce55f84674a452c
0202 02:52:22.952292 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.952401 (+ 109us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1240008}
I20260202 02:52:22.952845 1277 leader_election.cc:290] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
W20260202 02:52:22.953015 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 2f5aae089fcc492f9ce55f84674a452c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2f5aae089fcc492f9ce55f84674a452c already in progress: creating tablet
I20260202 02:52:22.953111 1738 leader_election.cc:290] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:22.953300 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 337) took 1241 ms. Trace:
I20260202 02:52:22.953433 1182 rpcz_store.cc:276] 0202 02:52:21.712071 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:21.712142 (+ 71us) service_pool.cc:225] Handling call
0202 02:52:22.953066 (+1240924us) ts_tablet_manager.cc:1151] Deleting tablet 6154460838eb4e508f3646d3e3e4cf21
0202 02:52:22.953075 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.953237 (+ 162us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1240755}
W20260202 02:52:22.953810 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 6154460838eb4e508f3646d3e3e4cf21 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6154460838eb4e508f3646d3e3e4cf21 already in progress: creating tablet
I20260202 02:52:22.953948 1182 tablet_replica.cc:333] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:22.954173 1182 raft_consensus.cc:2243] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:22.954404 1182 raft_consensus.cc:2272] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:22.954967 1182 ts_tablet_manager.cc:1916] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:22.960438 1180 log.cc:1199] T c064ea0dade840b4840a5715c2d83f69 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/c064ea0dade840b4840a5715c2d83f69
I20260202 02:52:22.961018 1180 ts_tablet_manager.cc:1950] T c064ea0dade840b4840a5715c2d83f69 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:22.961439 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 269) took 3277 ms. Trace:
I20260202 02:52:22.961666 1180 rpcz_store.cc:276] 0202 02:52:19.689032 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.691921 (+ 2889us) service_pool.cc:225] Handling call
0202 02:52:22.287562 (+2595641us) ts_tablet_manager.cc:1151] Deleting tablet c064ea0dade840b4840a5715c2d83f69
0202 02:52:22.304507 (+ 16945us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.453809 (+149302us) spinlock_profiling.cc:243] Waited 52.1 ms on lock 0x6160000b2f54. stack: 00007f1e37b5aa86 00007f1e37b5a840 00007f1e36b353ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f1e43a481e0 00007f1e44d960b3 00007f1e47956757 00007f1e47989744 00007f1e4796a240 00007f1e47969fcd 0000000000a8970c 00007f1e37ba0daf 00007f1e37ba40cb 00007f1e37ba3f48
0202 02:52:22.777102 (+323293us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.945259 (+168157us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.961388 (+ 16129us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":68184,"spinlock_wait_cycles":119908736,"tablet-delete.queue_time_us":2560873}
I20260202 02:52:22.961890 1182 ts_tablet_manager.cc:1929] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:22.962072 1182 log.cc:1199] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/0d3633f5f9f54115bc78b9c1b78aba67
I20260202 02:52:22.962505 1182 ts_tablet_manager.cc:1950] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:22.963320 1180 tablet_replica.cc:333] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:22.963348 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 338) took 1250 ms. Trace:
I20260202 02:52:22.963557 1182 rpcz_store.cc:276] 0202 02:52:21.712513 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:21.712554 (+ 41us) service_pool.cc:225] Handling call
0202 02:52:22.953793 (+1241239us) ts_tablet_manager.cc:1151] Deleting tablet 0d3633f5f9f54115bc78b9c1b78aba67
0202 02:52:22.953802 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.958366 (+ 4564us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.961742 (+ 3376us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.963297 (+ 1555us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1232957}
I20260202 02:52:22.963752 1180 raft_consensus.cc:2243] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:22.964017 1180 pending_rounds.cc:70] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d: Trying to abort 1 pending ops.
I20260202 02:52:22.964079 1180 pending_rounds.cc:77] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923040011849728 op_type: NO_OP noop_request { }
I20260202 02:52:22.964190 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 0d3633f5f9f54115bc78b9c1b78aba67 (table default.Table_63bb1b19c34a430e94ad0d7bd9bbe795 [id=af3058593c2747bf99db3b8da1aaedc7]) successfully deleted
I20260202 02:52:22.964227 1180 raft_consensus.cc:2889] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:22.964383 1180 raft_consensus.cc:2272] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:22.964555 1182 tablet_replica.cc:333] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:22.964830 1182 raft_consensus.cc:2243] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:22.965142 1182 pending_rounds.cc:70] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5: Trying to abort 1 pending ops.
I20260202 02:52:22.965230 1777 raft_consensus.cc:493] T 622a113af04a41c6852c29fc8a8acd78 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:22.965287 1182 pending_rounds.cc:77] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923040045858816 op_type: NO_OP noop_request { }
I20260202 02:52:22.965515 1182 raft_consensus.cc:2889] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:22.965440 1777 raft_consensus.cc:515] T 622a113af04a41c6852c29fc8a8acd78 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:22.965764 1182 raft_consensus.cc:2272] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:22.966307 1777 leader_election.cc:290] T 622a113af04a41c6852c29fc8a8acd78 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:22.966709 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "622a113af04a41c6852c29fc8a8acd78" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:22.966781 1182 ts_tablet_manager.cc:1916] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:22.967208 857 leader_election.cc:343] T 622a113af04a41c6852c29fc8a8acd78 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:22.968783 1180 ts_tablet_manager.cc:1916] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:22.983994 528 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet dada209d982d48f3b74a49905f19c5b6
I20260202 02:52:22.984611 528 catalog_manager.cc:6724] ReplaceTablet: tablet dada209d982d48f3b74a49905f19c5b6 deleted and replaced by tablet 67a8c9a1998b47b3b214a06ce62381e9
I20260202 02:52:22.985080 525 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet d64f42d187bc4c9f9381c931449ec0cc
I20260202 02:52:22.985126 526 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 26eeef6eebbf4f2cb44f69255fb37594
I20260202 02:52:22.985554 526 catalog_manager.cc:6724] ReplaceTablet: tablet 26eeef6eebbf4f2cb44f69255fb37594 deleted and replaced by tablet dc4c3f52f4f44a84b0d0e3c9a4a49f09
I20260202 02:52:22.985553 525 catalog_manager.cc:6724] ReplaceTablet: tablet d64f42d187bc4c9f9381c931449ec0cc deleted and replaced by tablet e95153ee8f2d40de955e3389d3a58644
I20260202 02:52:22.986723 898 tablet_service.cc:1552] Processing DeleteTablet for tablet d64f42d187bc4c9f9381c931449ec0cc with delete_type TABLET_DATA_DELETED (replaced by tablet e95153ee8f2d40de955e3389d3a58644) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:22.987094 889 tablet_service.cc:1552] Processing DeleteTablet for tablet dada209d982d48f3b74a49905f19c5b6 with delete_type TABLET_DATA_DELETED (replaced by tablet 67a8c9a1998b47b3b214a06ce62381e9) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:22.990391 893 tablet_service.cc:1552] Processing DeleteTablet for tablet 26eeef6eebbf4f2cb44f69255fb37594 with delete_type TABLET_DATA_DELETED (replaced by tablet dc4c3f52f4f44a84b0d0e3c9a4a49f09) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:22.991586 1182 ts_tablet_manager.cc:1929] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:22.991791 1182 log.cc:1199] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/8095e0c6ff0041fc8b8c8b0778a20b00
I20260202 02:52:22.992408 1182 ts_tablet_manager.cc:1950] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:22.993352 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 349) took 1168 ms. Trace:
I20260202 02:52:22.993552 1182 rpcz_store.cc:276] 0202 02:52:21.824777 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:21.824845 (+ 68us) service_pool.cc:225] Handling call
0202 02:52:22.964415 (+1139570us) ts_tablet_manager.cc:1151] Deleting tablet 8095e0c6ff0041fc8b8c8b0778a20b00
0202 02:52:22.964429 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.980804 (+ 16375us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.991437 (+ 10633us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:22.993303 (+ 1866us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1132023}
I20260202 02:52:22.994159 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 8095e0c6ff0041fc8b8c8b0778a20b00 (table default.Table_4e6c3886f58b49698a00d45b957f4c9a [id=b5cc03b77f414304b29c5fb4bb7dc3ee]) successfully deleted
I20260202 02:52:22.994331 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 347) took 1198 ms. Trace:
I20260202 02:52:22.994603 1182 rpcz_store.cc:276] 0202 02:52:21.798419 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:21.799383 (+ 964us) service_pool.cc:225] Handling call
0202 02:52:22.994158 (+1194775us) ts_tablet_manager.cc:1151] Deleting tablet 6df3d11cf578404ca41807f0777b6c53
0202 02:52:22.994171 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.994293 (+ 122us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1150376}
I20260202 02:52:22.995182 1182 tablet_replica.cc:333] T 929eb48881b141718873d0e359e768a8 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
W20260202 02:52:22.995154 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 6df3d11cf578404ca41807f0777b6c53 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6df3d11cf578404ca41807f0777b6c53 already in progress: creating tablet
I20260202 02:52:22.995515 1182 raft_consensus.cc:2243] T 929eb48881b141718873d0e359e768a8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:22.995744 1182 raft_consensus.cc:2272] T 929eb48881b141718873d0e359e768a8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:22.996696 1182 ts_tablet_manager.cc:1916] T 929eb48881b141718873d0e359e768a8 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:23.002873 714 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 241691us
I20260202 02:52:23.004009 1795 tablet.cc:1717] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d: Skipping requested alter to schema version 2, tablet already version 2
I20260202 02:52:23.005134 523 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_35fb6ec04ef245a88e633fedb3d9c186 [id=5f8923fc0fb84598a1fb7c3a8abbb8cb] alter complete (version 1)
I20260202 02:52:23.012945 1182 ts_tablet_manager.cc:1929] T 929eb48881b141718873d0e359e768a8 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260202 02:52:23.014254 515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.190:33981, user_credentials={real_user=slave}} blocked reactor thread for 305412us
I20260202 02:52:23.016477 1182 log.cc:1199] T 929eb48881b141718873d0e359e768a8 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/929eb48881b141718873d0e359e768a8
I20260202 02:52:23.017841 1182 ts_tablet_manager.cc:1950] T 929eb48881b141718873d0e359e768a8 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:23.018863 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 351) took 1190 ms. Trace:
W20260202 02:52:23.019986 714 leader_election.cc:343] T eaa47f3388144e8b852804c167f232ce P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: eaa47f3388144e8b852804c167f232ce
I20260202 02:52:23.019912 1182 rpcz_store.cc:276] 0202 02:52:21.828891 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:21.828965 (+ 74us) service_pool.cc:225] Handling call
0202 02:52:22.995115 (+1166150us) ts_tablet_manager.cc:1151] Deleting tablet 929eb48881b141718873d0e359e768a8
0202 02:52:22.995126 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:23.000278 (+ 5152us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:23.012778 (+ 12500us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:23.018795 (+ 6017us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1148900}
I20260202 02:52:23.020439 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 929eb48881b141718873d0e359e768a8 (table default.Table_b0e344dc1a824cd9b41aa8991b7911dd [id=8e640d1261014706a067f522bda7ea61]) successfully deleted
I20260202 02:52:23.020884 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 348) took 1202 ms. Trace:
I20260202 02:52:23.021420 1182 rpcz_store.cc:276] 0202 02:52:21.818263 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:21.822623 (+ 4360us) service_pool.cc:225] Handling call
0202 02:52:23.020698 (+1198075us) ts_tablet_manager.cc:1151] Deleting tablet 79badde1001b432b95f58b5161ace15c
0202 02:52:23.020712 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:23.020823 (+ 111us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1169405}
W20260202 02:52:23.022310 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 79badde1001b432b95f58b5161ace15c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 79badde1001b432b95f58b5161ace15c already in progress: creating tablet
I20260202 02:52:23.022394 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 352) took 1191 ms. Trace:
I20260202 02:52:23.022711 1721 raft_consensus.cc:493] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:23.022888 1182 rpcz_store.cc:276] 0202 02:52:21.830842 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:21.830908 (+ 66us) service_pool.cc:225] Handling call
0202 02:52:23.022225 (+1191317us) ts_tablet_manager.cc:1151] Deleting tablet fe53a8bdbd13417f9ce195c54a52a93c
0202 02:52:23.022239 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:23.022346 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1159511}
I20260202 02:52:23.022737 1173 raft_consensus.cc:359] T 929eb48881b141718873d0e359e768a8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:23.024370 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet fe53a8bdbd13417f9ce195c54a52a93c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fe53a8bdbd13417f9ce195c54a52a93c already in progress: creating tablet
I20260202 02:52:23.024443 1182 tablet_replica.cc:333] T 75b97d7e848241cf8d894b0ae92a8f8f P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:23.024638 1182 raft_consensus.cc:2243] T 75b97d7e848241cf8d894b0ae92a8f8f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:23.025337 1182 raft_consensus.cc:2272] T 75b97d7e848241cf8d894b0ae92a8f8f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:23.026194 1182 ts_tablet_manager.cc:1916] T 75b97d7e848241cf8d894b0ae92a8f8f P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:23.027606 999 consensus_peers.cc:597] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bc1539e5c8aa4d6e98f687728b891530. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:23.036425 998 leader_election.cc:304] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:23.037393 1721 raft_consensus.cc:515] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.037914 1173 raft_consensus.cc:385] T 929eb48881b141718873d0e359e768a8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:23.038020 1173 raft_consensus.cc:740] T 929eb48881b141718873d0e359e768a8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:23.039050 1182 ts_tablet_manager.cc:1929] T 75b97d7e848241cf8d894b0ae92a8f8f P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:23.041114 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "526f2c7b95184a0987e7aa94efc8475a" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:23.041364 1063 raft_consensus.cc:3060] T 526f2c7b95184a0987e7aa94efc8475a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:23.041426 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "78f122342dd144b48de89902faa0f94f" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:23.041877 1059 raft_consensus.cc:3060] T 78f122342dd144b48de89902faa0f94f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:23.042246 998 leader_election.cc:343] T ffd80badce1048f3ac95366cad05ae53 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:23.044134 1868 tablet.cc:1722] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260202 02:52:23.044500 1182 log.cc:1199] T 75b97d7e848241cf8d894b0ae92a8f8f P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/75b97d7e848241cf8d894b0ae92a8f8f
I20260202 02:52:23.045850 1182 ts_tablet_manager.cc:1950] T 75b97d7e848241cf8d894b0ae92a8f8f P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:23.047676 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 75b97d7e848241cf8d894b0ae92a8f8f (table default.Table_5e366ca4b9744597add5ca93f7fe7850 [id=f2e865926cf5453292c34daa4af8be2a]) successfully deleted
I20260202 02:52:23.048002 714 leader_election.cc:304] T eaa47f3388144e8b852804c167f232ce P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
W20260202 02:52:23.048658 714 leader_election.cc:343] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: f58a7b25815c4b5bac568aa11c19610c
W20260202 02:52:23.048983 714 leader_election.cc:343] T 0186572e7bb9499a932a636ab280f544 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 0186572e7bb9499a932a636ab280f544
I20260202 02:52:23.049094 714 leader_election.cc:304] T 0186572e7bb9499a932a636ab280f544 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:23.049970 523 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_755b6ce0796b45b8be9df04e59dfe565 [id=d8485f23f63b493781bea557dbe8a0c1] alter complete (version 2)
I20260202 02:52:23.052184 1173 consensus_queue.cc:260] T 929eb48881b141718873d0e359e768a8 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.051918 1721 leader_election.cc:290] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:23.053321 1764 raft_consensus.cc:515] T 75b97d7e848241cf8d894b0ae92a8f8f P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.053629 1721 raft_consensus.cc:2804] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:23.053741 1721 raft_consensus.cc:493] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:23.053846 1721 raft_consensus.cc:3060] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:23.054111 1764 leader_election.cc:290] T 75b97d7e848241cf8d894b0ae92a8f8f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:23.055226 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "622a113af04a41c6852c29fc8a8acd78" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:23.055444 1062 raft_consensus.cc:2468] T 622a113af04a41c6852c29fc8a8acd78 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
I20260202 02:52:23.056790 1721 raft_consensus.cc:515] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.057528 1721 leader_election.cc:290] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:23.057782 999 leader_election.cc:343] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: eaa47f3388144e8b852804c167f232ce
I20260202 02:52:23.058439 998 leader_election.cc:304] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
W20260202 02:52:23.058949 999 leader_election.cc:343] T 75b97d7e848241cf8d894b0ae92a8f8f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 75b97d7e848241cf8d894b0ae92a8f8f
W20260202 02:52:23.059196 999 leader_election.cc:343] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 9884c90cf1d245b2a7eaaf00d71b1c3e
W20260202 02:52:23.059445 998 leader_election.cc:343] T 75b97d7e848241cf8d894b0ae92a8f8f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:23.060184 1721 raft_consensus.cc:2804] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:23.060384 1721 raft_consensus.cc:493] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:23.060559 1721 raft_consensus.cc:3060] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:23.060778 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "eaa47f3388144e8b852804c167f232ce" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:23.061013 774 raft_consensus.cc:2468] T eaa47f3388144e8b852804c167f232ce P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:23.061309 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "75b97d7e848241cf8d894b0ae92a8f8f" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:23.061617 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9884c90cf1d245b2a7eaaf00d71b1c3e" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:23.061784 772 raft_consensus.cc:3060] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:23.063548 1721 raft_consensus.cc:515] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.064029 1182 tablet_replica.cc:333] T ffd80badce1048f3ac95366cad05ae53 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:23.064465 1721 leader_election.cc:290] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:23.065074 999 leader_election.cc:343] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: f93e8d3c4faf48eaa98c3cbf88d585e2
I20260202 02:52:23.065325 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f93e8d3c4faf48eaa98c3cbf88d585e2" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:23.064292 1182 raft_consensus.cc:2243] T ffd80badce1048f3ac95366cad05ae53 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:23.065423 1178 raft_consensus.cc:515] T ffd80badce1048f3ac95366cad05ae53 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.065511 779 raft_consensus.cc:3060] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:23.065622 1182 raft_consensus.cc:2272] T ffd80badce1048f3ac95366cad05ae53 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:23.066175 1178 leader_election.cc:290] T ffd80badce1048f3ac95366cad05ae53 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:23.066401 918 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ffd80badce1048f3ac95366cad05ae53" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:23.067476 1182 ts_tablet_manager.cc:1916] T ffd80badce1048f3ac95366cad05ae53 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:23.068225 779 raft_consensus.cc:2468] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:23.070667 772 raft_consensus.cc:2468] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
W20260202 02:52:23.072474 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 723921dfe54c4e66af5cab76a25fee0a because the tablet was not found. No further retry: Not found: Tablet not found: 723921dfe54c4e66af5cab76a25fee0a
W20260202 02:52:23.072803 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet d2fa4c3d6ec44047abf5986e502c1590 because the tablet was not found. No further retry: Not found: Tablet not found: d2fa4c3d6ec44047abf5986e502c1590
W20260202 02:52:23.073028 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 61a9203035f2495f8f99924303ebf120 because the tablet was not found. No further retry: Not found: Tablet not found: 61a9203035f2495f8f99924303ebf120
W20260202 02:52:23.073220 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 68823eff1bc04108ba83487605aec0ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 68823eff1bc04108ba83487605aec0ea already in progress: creating tablet
W20260202 02:52:23.073429 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 79f694b740954d429145fba9157338ca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 79f694b740954d429145fba9157338ca already in progress: creating tablet
I20260202 02:52:23.074657 1804 tablet.cc:1722] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260202 02:52:23.074797 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ffd80badce1048f3ac95366cad05ae53" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
W20260202 02:52:23.075318 999 leader_election.cc:343] T ffd80badce1048f3ac95366cad05ae53 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:23.073634 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 59ad016e15ec4a488921eb564029b689 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 59ad016e15ec4a488921eb564029b689 already in progress: creating tablet
I20260202 02:52:23.079644 1863 raft_consensus.cc:493] T d21fca9a5b234c26879717f51c8ea0f0 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:23.079954 1863 raft_consensus.cc:515] T d21fca9a5b234c26879717f51c8ea0f0 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.080296 859 leader_election.cc:304] T 622a113af04a41c6852c29fc8a8acd78 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
W20260202 02:52:23.079780 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet a5622ef925bc46cca76f481ddfd12afd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a5622ef925bc46cca76f481ddfd12afd already in progress: creating tablet
W20260202 02:52:23.081048 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 2b352d9a815e4bccafb7e2251500c5bc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2b352d9a815e4bccafb7e2251500c5bc already in progress: creating tablet
I20260202 02:52:23.081240 1865 raft_consensus.cc:2804] T 622a113af04a41c6852c29fc8a8acd78 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260202 02:52:23.081385 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 08d8d781bb4e4acf915f11afb4fb2e22 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 08d8d781bb4e4acf915f11afb4fb2e22 already in progress: creating tablet
I20260202 02:52:23.081395 1865 raft_consensus.cc:493] T 622a113af04a41c6852c29fc8a8acd78 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260202 02:52:23.081588 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet ff6f5710de4b4c48948aacdf5be29e8b because the tablet was not found. No further retry: Not found: Tablet not found: ff6f5710de4b4c48948aacdf5be29e8b
I20260202 02:52:23.081724 1865 raft_consensus.cc:3060] T 622a113af04a41c6852c29fc8a8acd78 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:23.084836 1174 tablet_bootstrap.cc:492] T 75b97d7e848241cf8d894b0ae92a8f8f P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:23.085035 1174 ts_tablet_manager.cc:1403] T 75b97d7e848241cf8d894b0ae92a8f8f P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.679s user 0.000s sys 0.015s
I20260202 02:52:23.085608 1182 ts_tablet_manager.cc:1929] T ffd80badce1048f3ac95366cad05ae53 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:23.085772 1600 raft_consensus.cc:2749] T eaa47f3388144e8b852804c167f232ce P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:23.085959 1600 raft_consensus.cc:2749] T 0186572e7bb9499a932a636ab280f544 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:23.085944 1182 log.cc:1199] T ffd80badce1048f3ac95366cad05ae53 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/ffd80badce1048f3ac95366cad05ae53
I20260202 02:52:23.086413 714 leader_election.cc:304] T 70306ed9dec54286885d965f2c012da4 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:23.086726 1182 ts_tablet_manager.cc:1950] T ffd80badce1048f3ac95366cad05ae53 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:23.087198 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d21fca9a5b234c26879717f51c8ea0f0" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:23.087683 857 leader_election.cc:343] T d21fca9a5b234c26879717f51c8ea0f0 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:23.088042 1600 raft_consensus.cc:2804] T 70306ed9dec54286885d965f2c012da4 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:23.088189 1600 raft_consensus.cc:697] T 70306ed9dec54286885d965f2c012da4 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:23.088477 1600 consensus_queue.cc:237] T 70306ed9dec54286885d965f2c012da4 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.088711 1865 raft_consensus.cc:515] T 622a113af04a41c6852c29fc8a8acd78 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:23.089380 1865 leader_election.cc:290] T 622a113af04a41c6852c29fc8a8acd78 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:23.089771 1277 consensus_queue.cc:1048] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:23.090303 1182 tablet_replica.cc:333] T d67e91a022f1401ea724dff898c3b81b P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:23.086802 1863 leader_election.cc:290] T d21fca9a5b234c26879717f51c8ea0f0 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:23.090667 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "622a113af04a41c6852c29fc8a8acd78" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:23.090554 1182 raft_consensus.cc:2243] T d67e91a022f1401ea724dff898c3b81b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:23.090942 1182 raft_consensus.cc:2272] T d67e91a022f1401ea724dff898c3b81b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:23.091218 857 leader_election.cc:343] T 622a113af04a41c6852c29fc8a8acd78 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:23.091861 1182 ts_tablet_manager.cc:1916] T d67e91a022f1401ea724dff898c3b81b P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:23.094242 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet ffd80badce1048f3ac95366cad05ae53 (table default.Table_38c5bca682e343c1be1ebdc253c9e848 [id=6f44bec7f8ea4d2abed8c0fd603c6569]) successfully deleted
W20260202 02:52:23.094640 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 583e547d0a1943f09f4efe82070fc777 because the tablet was not found. No further retry: Not found: Tablet not found: 583e547d0a1943f09f4efe82070fc777
I20260202 02:52:23.095110 1795 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:42950 (request call id 326) took 2182 ms. Trace:
I20260202 02:52:23.095273 1795 rpcz_store.cc:276] 0202 02:52:20.912395 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:20.912461 (+ 66us) service_pool.cc:225] Handling call
0202 02:52:23.095049 (+2182588us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:20.929217 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0202 02:52:20.929296 (+ 79us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0202 02:52:21.173328 (+244032us) alter_schema_op.cc:66] Acquired schema lock
0202 02:52:21.173341 (+ 13us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0202 02:52:21.174699 (+ 1358us) alter_schema_op.cc:128] START. Timestamp: 1770000741174628
0202 02:52:21.174703 (+ 4us) op_driver.cc:348] REPLICATION: starting
0202 02:52:21.228758 (+ 54055us) log.cc:844] Serialized 173 byte log entry
0202 02:52:23.003875 (+1775117us) op_driver.cc:464] REPLICATION: finished
0202 02:52:23.003955 (+ 80us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0202 02:52:23.004230 (+ 275us) log.cc:844] Serialized 94 byte log entry
0202 02:52:23.094988 (+ 90758us) alter_schema_op.cc:73] Released schema lock
0202 02:52:23.094994 (+ 6us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":33,"mutex_wait_us":977,"prepare.queue_time_us":1075,"prepare.run_cpu_time_us":25917,"prepare.run_wall_time_us":299653,"raft.queue_time_us":458,"raft.run_cpu_time_us":356,"raft.run_wall_time_us":3039,"replication_time_us":1829137,"thread_start_us":90669,"threads_started":1,"wal-append.queue_time_us":74123}]]}
I20260202 02:52:23.103685 1174 raft_consensus.cc:359] T 75b97d7e848241cf8d894b0ae92a8f8f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.103982 893 tablet_service.cc:1505] Processing CreateTablet for tablet 09504070c9ec4bcfbb59e3fb4c189775 (DEFAULT_TABLE table=default.Table_3a2daddab50e4578898828f3a305b8e7 [id=4c17da59a0554cee93ce4e626838a2c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.104390 1174 raft_consensus.cc:385] T 75b97d7e848241cf8d894b0ae92a8f8f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:23.104475 1174 raft_consensus.cc:740] T 75b97d7e848241cf8d894b0ae92a8f8f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:23.104511 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09504070c9ec4bcfbb59e3fb4c189775. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.104818 1174 consensus_queue.cc:260] T 75b97d7e848241cf8d894b0ae92a8f8f P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.106139 1182 ts_tablet_manager.cc:1929] T d67e91a022f1401ea724dff898c3b81b P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:23.106288 1182 log.cc:1199] T d67e91a022f1401ea724dff898c3b81b P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/d67e91a022f1401ea724dff898c3b81b
I20260202 02:52:23.106699 1182 ts_tablet_manager.cc:1950] T d67e91a022f1401ea724dff898c3b81b P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:23.108048 1182 tablet_replica.cc:333] T d64f42d187bc4c9f9381c931449ec0cc P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:23.108361 1182 raft_consensus.cc:2243] T d64f42d187bc4c9f9381c931449ec0cc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:23.108604 1182 raft_consensus.cc:2272] T d64f42d187bc4c9f9381c931449ec0cc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:23.108530 1277 consensus_queue.cc:1048] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:23.108771 998 leader_election.cc:304] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:23.109294 1182 ts_tablet_manager.cc:1916] T d64f42d187bc4c9f9381c931449ec0cc P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:23.111104 918 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "78f122342dd144b48de89902faa0f94f" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:23.111395 918 raft_consensus.cc:3060] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:23.113987 1868 tablet.cc:1722] T 5986676f5484498da927b4c6da3f47ff P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:23.115847 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet d67e91a022f1401ea724dff898c3b81b (table default.Table_c245b83c1987454fabbe8646d14252d6 [id=08498a654b6a4a9ca3f968d70daad9b6]) successfully deleted
I20260202 02:52:23.118072 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet c064ea0dade840b4840a5715c2d83f69 (table default.Table_5fa327d665484811934aa3b11188a76d [id=f7c5b24c065443d89a2b1f9c34409c9d]) successfully deleted
I20260202 02:52:23.122146 918 raft_consensus.cc:2468] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:23.126333 1057 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d21fca9a5b234c26879717f51c8ea0f0" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:23.126528 998 leader_election.cc:343] T 75b97d7e848241cf8d894b0ae92a8f8f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:23.126683 1178 raft_consensus.cc:2804] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:23.127012 1180 ts_tablet_manager.cc:1929] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:23.127044 893 tablet_service.cc:1505] Processing CreateTablet for tablet 3dcf4f794ec14b2f9d181d015fc437fb (DEFAULT_TABLE table=default.Table_bb45d0e2e2a14038aec12db59746877a [id=2a786b768e5e436f86e56c725c9c90a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.127184 1180 log.cc:1199] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/f58a7b25815c4b5bac568aa11c19610c
I20260202 02:52:23.127485 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dcf4f794ec14b2f9d181d015fc437fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.127619 1180 ts_tablet_manager.cc:1950] T f58a7b25815c4b5bac568aa11c19610c P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
W20260202 02:52:23.127656 859 leader_election.cc:343] T d21fca9a5b234c26879717f51c8ea0f0 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:23.127990 859 leader_election.cc:304] T d21fca9a5b234c26879717f51c8ea0f0 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:23.129207 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 270) took 3412 ms. Trace:
I20260202 02:52:23.129467 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "622a113af04a41c6852c29fc8a8acd78" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:23.130676 1060 raft_consensus.cc:3060] T 622a113af04a41c6852c29fc8a8acd78 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:23.131392 1174 ts_tablet_manager.cc:1434] T 75b97d7e848241cf8d894b0ae92a8f8f P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.046s user 0.002s sys 0.001s
I20260202 02:52:23.132071 1174 tablet_bootstrap.cc:492] T ef073880898049e2be93d2cb6dc0c0bb P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:23.130676 998 leader_election.cc:304] T 75b97d7e848241cf8d894b0ae92a8f8f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:23.134033 1174 tablet_bootstrap.cc:654] T ef073880898049e2be93d2cb6dc0c0bb P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:23.134007 1867 raft_consensus.cc:2749] T d21fca9a5b234c26879717f51c8ea0f0 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:23.134303 1058 raft_consensus.cc:3060] T c12ff20ca6b44ddba0c936a28cfb2eb9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:23.136986 998 leader_election.cc:304] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:23.136377 1721 raft_consensus.cc:493] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:23.138185 1182 ts_tablet_manager.cc:1929] T d64f42d187bc4c9f9381c931449ec0cc P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:23.138592 1178 raft_consensus.cc:493] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:23.139879 1723 raft_consensus.cc:493] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:23.140156 1182 log.cc:1199] T d64f42d187bc4c9f9381c931449ec0cc P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/d64f42d187bc4c9f9381c931449ec0cc
I20260202 02:52:23.140128 1723 raft_consensus.cc:515] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.140786 1182 ts_tablet_manager.cc:1950] T d64f42d187bc4c9f9381c931449ec0cc P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:23.141058 1723 leader_election.cc:290] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:23.141345 1764 raft_consensus.cc:2804] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:23.141497 1764 raft_consensus.cc:697] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:23.142014 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet d64f42d187bc4c9f9381c931449ec0cc (table default.Table_0c83932cab13463f9cca28129f18b576 [id=7c76848b98614fbf8b2950e5d55cc609]) successfully deleted
I20260202 02:52:23.142201 1059 raft_consensus.cc:2468] T 78f122342dd144b48de89902faa0f94f P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:23.142366 1721 raft_consensus.cc:515] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.142577 1869 raft_consensus.cc:2749] T 75b97d7e848241cf8d894b0ae92a8f8f P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260202 02:52:23.144476 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet dada209d982d48f3b74a49905f19c5b6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dada209d982d48f3b74a49905f19c5b6 already in progress: creating tablet
W20260202 02:52:23.145241 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 26eeef6eebbf4f2cb44f69255fb37594 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 26eeef6eebbf4f2cb44f69255fb37594 already in progress: creating tablet
I20260202 02:52:23.146183 742 tablet_service.cc:1552] Processing DeleteTablet for tablet 68823eff1bc04108ba83487605aec0ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:21 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:23.146303 752 tablet_service.cc:1552] Processing DeleteTablet for tablet 59ad016e15ec4a488921eb564029b689 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:21 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:23.146517 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 79f694b740954d429145fba9157338ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:21 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:23.146636 752 tablet_service.cc:1552] Processing DeleteTablet for tablet a5622ef925bc46cca76f481ddfd12afd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:21 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:23.147056 752 tablet_service.cc:1552] Processing DeleteTablet for tablet 723921dfe54c4e66af5cab76a25fee0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:23.147290 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 0186572e7bb9499a932a636ab280f544 with delete_type TABLET_DATA_DELETED (replaced by tablet 79c73f4a92044bafb5b8384229cb3a4d) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:23.147490 742 tablet_service.cc:1552] Processing DeleteTablet for tablet eaa47f3388144e8b852804c167f232ce with delete_type TABLET_DATA_DELETED (replaced by tablet 38fdbc6df7ec40b9afe4e0dfe0119e3e) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:23.147552 752 tablet_service.cc:1505] Processing CreateTablet for tablet c17c85759e964f0ba3abcaf5fe938fba (DEFAULT_TABLE table=default.Table_67001aa310ae44c0bb9e2b24bb0b57b8 [id=c17d62a148094547af24de85773a1b8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.148025 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c17c85759e964f0ba3abcaf5fe938fba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.149132 1173 ts_tablet_manager.cc:1434] T 929eb48881b141718873d0e359e768a8 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.230s user 0.002s sys 0.018s
I20260202 02:52:23.149734 1876 tablet.cc:1722] T c050a2882f764dda8df2a7df6547f149 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:23.150341 998 leader_election.cc:304] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:23.150570 1063 raft_consensus.cc:2468] T 526f2c7b95184a0987e7aa94efc8475a P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:23.152436 893 tablet_service.cc:1505] Processing CreateTablet for tablet 604b3b863c9540d3b55381dbfe1afdc3 (DEFAULT_TABLE table=default.Table_89983dadd3fc4966bf26418544c1729a [id=ce752170017a4e119a8c42c73fd95302]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.156421 1764 consensus_queue.cc:237] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.156865 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 604b3b863c9540d3b55381dbfe1afdc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.158788 1178 raft_consensus.cc:3060] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:23.160884 715 leader_election.cc:304] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:23.162285 889 tablet_service.cc:1505] Processing CreateTablet for tablet 2f664f6119ba4d28b5ef1c212e611866 (DEFAULT_TABLE table=default.Table_cc8d0231630743f180159c4d3a4efab8 [id=0dc5dc53fc014dd89bc66ac6260de9e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.162741 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f664f6119ba4d28b5ef1c212e611866. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.163676 715 leader_election.cc:304] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:23.164060 1723 raft_consensus.cc:2804] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:23.164542 1850 raft_consensus.cc:2804] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:23.165047 1850 raft_consensus.cc:697] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:23.165439 1850 consensus_queue.cc:237] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:23.166905 1723 raft_consensus.cc:697] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:23.167274 1723 consensus_queue.cc:237] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:23.168548 899 tablet_service.cc:1505] Processing CreateTablet for tablet 1959438c70c544e69422f7354082cf89 (DEFAULT_TABLE table=default.Table_eeb507a1a49d407196bff72270b0834f [id=a18136bbb87a47d69600f926a474da24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.169299 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1959438c70c544e69422f7354082cf89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.171197 917 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "94bada52db6a47179415d302b541d01f" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:23.171456 917 raft_consensus.cc:2468] T 94bada52db6a47179415d302b541d01f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
W20260202 02:52:23.172904 515 consensus_peers.cc:597] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b -> Peer 7dab2abef14445d688ce1335b1f9b2f8 (127.0.104.190:33981): Couldn't send request to peer 7dab2abef14445d688ce1335b1f9b2f8. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.190:33981: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:23.174472 898 tablet_service.cc:1505] Processing CreateTablet for tablet 7921daa000f54c8dbed04473e21cf0a5 (DEFAULT_TABLE table=default.Table_2acd399bc33146fe9f1b6e2a15e0b101 [id=487a204c7845401ab7391ae8464fdf1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.174988 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7921daa000f54c8dbed04473e21cf0a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.177052 523 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 25765fb687294e9db1289f76530dd877 (table default.Table_8c47663f416b467cad37110bd09d85ad [id=ea0c52a72da8422ba14345fbfc242d27]). Expected version 1 got 0
I20260202 02:52:23.179687 1863 raft_consensus.cc:493] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:23.179913 1863 raft_consensus.cc:515] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.180564 1863 leader_election.cc:290] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:23.180533 752 tablet_service.cc:1505] Processing CreateTablet for tablet 4a518e8ad2e34ec98a0425c7ed3a470a (DEFAULT_TABLE table=default.Table_3dcd1dea44ed4e54a89f4413e7f17dbe [id=f868f7fd84f247c89a963da591f16c2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.181003 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a518e8ad2e34ec98a0425c7ed3a470a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.181145 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d1e0e74f32a24729b250c5751677cfb0" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:23.181808 857 leader_election.cc:343] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:23.186412 523 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_cafbdde19f984e0fa22b1fde486f8e7e [id=1229c6d37b994a669d3363fdf6f8c6c8] alter complete (version 3)
I20260202 02:52:23.189498 894 tablet_service.cc:1552] Processing DeleteTablet for tablet 94bada52db6a47179415d302b541d01f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:23.189551 902 tablet_service.cc:1552] Processing DeleteTablet for tablet 2f0e8c5e2fd44370ae150a206f4e6cbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:23.189719 886 tablet_service.cc:1552] Processing DeleteTablet for tablet d67e91a022f1401ea724dff898c3b81b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:23.189973 1182 tablet_replica.cc:333] T 94bada52db6a47179415d302b541d01f P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:23.129472 1180 rpcz_store.cc:276] 0202 02:52:19.716111 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.716202 (+ 91us) service_pool.cc:225] Handling call
0202 02:52:22.963229 (+3247027us) ts_tablet_manager.cc:1151] Deleting tablet f58a7b25815c4b5bac568aa11c19610c
0202 02:52:22.963242 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:22.971796 (+ 8554us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:23.126858 (+155062us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:23.129012 (+ 2154us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3235253}
I20260202 02:52:23.190271 1182 raft_consensus.cc:2243] T 94bada52db6a47179415d302b541d01f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:23.190501 1182 raft_consensus.cc:2272] T 94bada52db6a47179415d302b541d01f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:23.190691 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 273) took 3461 ms. Trace:
I20260202 02:52:23.190936 1180 rpcz_store.cc:276] 0202 02:52:19.729040 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.729108 (+ 68us) service_pool.cc:225] Handling call
0202 02:52:23.190435 (+3461327us) ts_tablet_manager.cc:1151] Deleting tablet 94bada52db6a47179415d302b541d01f
0202 02:52:23.190449 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:23.190632 (+ 183us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3460249}
I20260202 02:52:23.191172 1182 ts_tablet_manager.cc:1916] T 94bada52db6a47179415d302b541d01f P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:23.194808 1828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } } }
I20260202 02:52:23.195094 1828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [sys.catalog]: This master's current role is: LEADER
I20260202 02:52:23.196599 889 tablet_service.cc:1552] Processing DeleteTablet for tablet d1e0e74f32a24729b250c5751677cfb0 with delete_type TABLET_DATA_DELETED (replaced by tablet 2c31b45dc8004eae993e972e9faa3ff8) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:23.197088 998 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 59001.6us
I20260202 02:52:23.200122 1182 ts_tablet_manager.cc:1929] T 94bada52db6a47179415d302b541d01f P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:23.200122 999 leader_election.cc:304] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:23.192126 742 tablet_service.cc:1505] Processing CreateTablet for tablet 7f9dfac19c04475aadb45e95a6474eb4 (DEFAULT_TABLE table=default.Table_c2f3ac0894bd4f4f9702b6429da397bc [id=67e7356eb4d0432f884247749fb97bac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.192981 741 tablet_service.cc:1505] Processing CreateTablet for tablet 7f323175ee254df789bcb04e65047dff (DEFAULT_TABLE table=default.Table_783c68f0f6ad41899a919edefc2e85df [id=0d6674bc4f264a36aad4ea7962a9e0e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.201193 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f9dfac19c04475aadb45e95a6474eb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.201498 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f323175ee254df789bcb04e65047dff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.203135 1721 leader_election.cc:290] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:23.203302 1173 tablet_bootstrap.cc:492] T d21fca9a5b234c26879717f51c8ea0f0 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:23.203409 1037 tablet_service.cc:1552] Processing DeleteTablet for tablet fe7b6004e5094c0dbe5847144a2489b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.205833 1860 raft_consensus.cc:2804] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:23.206023 1860 raft_consensus.cc:697] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:23.206346 1860 consensus_queue.cc:237] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.207886 1042 tablet_service.cc:1552] Processing DeleteTablet for tablet c12ff20ca6b44ddba0c936a28cfb2eb9 with delete_type TABLET_DATA_DELETED (replaced by tablet 072bb7f0e4ab4935be49461bc8a3c163) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.209237 1182 log.cc:1199] T 94bada52db6a47179415d302b541d01f P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/94bada52db6a47179415d302b541d01f
I20260202 02:52:23.209618 1182 ts_tablet_manager.cc:1950] T 94bada52db6a47179415d302b541d01f P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:23.209918 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet f58a7b25815c4b5bac568aa11c19610c with delete_type TABLET_DATA_DELETED (replaced by tablet cb634d4bae554ef3b39eeaa9958adc6c) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.210150 1039 tablet_service.cc:1552] Processing DeleteTablet for tablet bbf31e5619ed4f60b4d7c8b19ec8a549 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:16 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.208755 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bbf31e5619ed4f60b4d7c8b19ec8a549" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:23.210398 1180 tablet_replica.cc:333] T c184d44d7f434b39a997a7387e5d3de0 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:23.210621 1180 raft_consensus.cc:2243] T c184d44d7f434b39a997a7387e5d3de0 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:23.211051 1180 raft_consensus.cc:2272] T c184d44d7f434b39a997a7387e5d3de0 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:23.211073 1032 tablet_service.cc:1552] Processing DeleteTablet for tablet bc1539e5c8aa4d6e98f687728b891530 with delete_type TABLET_DATA_DELETED (replaced by tablet 84689b18ad274ba0a1522441980b62ce) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.211750 1180 ts_tablet_manager.cc:1916] T c184d44d7f434b39a997a7387e5d3de0 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:23.210261 899 tablet_service.cc:1505] Processing CreateTablet for tablet 842197c689064660b791b40e521d346b (DEFAULT_TABLE table=default.Table_24d7cc37a3884cda9a1e269515d3877d [id=61117090be4347b6a7e5a88aaa765ffc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.213567 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet a5622ef925bc46cca76f481ddfd12afd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:21 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.213768 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 842197c689064660b791b40e521d346b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.214124 1038 tablet_service.cc:1505] Processing CreateTablet for tablet c17c85759e964f0ba3abcaf5fe938fba (DEFAULT_TABLE table=default.Table_67001aa310ae44c0bb9e2b24bb0b57b8 [id=c17d62a148094547af24de85773a1b8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.215656 1032 tablet_service.cc:1552] Processing DeleteTablet for tablet 68823eff1bc04108ba83487605aec0ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:21 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.216178 1039 tablet_service.cc:1552] Processing DeleteTablet for tablet 583e547d0a1943f09f4efe82070fc777 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.216403 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet c184d44d7f434b39a997a7387e5d3de0 with delete_type TABLET_DATA_DELETED (replaced by tablet 3711910e050c4515bed4294d1271d5a9) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.216356 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d1e0e74f32a24729b250c5751677cfb0" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:23.216998 494 master_runner.cc:186] Error getting master registration for 127.0.104.190:33981: Network error: Client connection negotiation failed: client connection to 127.0.104.190:33981: connect: Connection refused (error 111),
I20260202 02:52:23.217173 494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:52:23.217927 1030 tablet_service.cc:1505] Processing CreateTablet for tablet b9fc33e8efc04734a951f7fcf9048142 (DEFAULT_TABLE table=default.Table_fbe3212129904307af7133ead4053a59 [id=0aeb16c903f642d8bfa236a96e203767]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.218345 1063 raft_consensus.cc:2468] T bbf31e5619ed4f60b4d7c8b19ec8a549 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:23.218711 1040 tablet_service.cc:1552] Processing DeleteTablet for tablet 2b352d9a815e4bccafb7e2251500c5bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:22 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.219247 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet 79f694b740954d429145fba9157338ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:21 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.219414 1034 tablet_service.cc:1552] Processing DeleteTablet for tablet 59ad016e15ec4a488921eb564029b689 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:21 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.219449 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 4a518e8ad2e34ec98a0425c7ed3a470a (DEFAULT_TABLE table=default.Table_3dcd1dea44ed4e54a89f4413e7f17dbe [id=f868f7fd84f247c89a963da591f16c2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.219563 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c17c85759e964f0ba3abcaf5fe938fba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.219825 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a518e8ad2e34ec98a0425c7ed3a470a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.219903 1042 tablet_service.cc:1552] Processing DeleteTablet for tablet 723921dfe54c4e66af5cab76a25fee0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.221572 1882 tablet_replica.cc:333] T fe7b6004e5094c0dbe5847144a2489b9 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:23.221819 1882 raft_consensus.cc:2243] T fe7b6004e5094c0dbe5847144a2489b9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:23.222113 1882 raft_consensus.cc:2272] T fe7b6004e5094c0dbe5847144a2489b9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:23.222683 1882 ts_tablet_manager.cc:1916] T fe7b6004e5094c0dbe5847144a2489b9 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:23.223670 1041 tablet_service.cc:1505] Processing CreateTablet for tablet 9a1d7d173c41462c8e32a95b494bb708 (DEFAULT_TABLE table=default.Table_28e7d9aa8131413e9a174756c246d164 [id=ec07c59893cf45c9b2a5eece556d10c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.224102 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a1d7d173c41462c8e32a95b494bb708. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.224566 1031 tablet_service.cc:1552] Processing DeleteTablet for tablet 267bd4752c5f4b99a3fe778ba8f38a2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.224754 749 tablet_service.cc:1505] Processing CreateTablet for tablet 9a1d7d173c41462c8e32a95b494bb708 (DEFAULT_TABLE table=default.Table_28e7d9aa8131413e9a174756c246d164 [id=ec07c59893cf45c9b2a5eece556d10c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.225178 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a1d7d173c41462c8e32a95b494bb708. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.225188 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet c064ea0dade840b4840a5715c2d83f69 with delete_type TABLET_DATA_DELETED (replaced by tablet 622a113af04a41c6852c29fc8a8acd78) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.226583 1059 raft_consensus.cc:2468] T d1e0e74f32a24729b250c5751677cfb0 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
I20260202 02:52:23.226917 1043 tablet_service.cc:1505] Processing CreateTablet for tablet bbcc723a8079443db455ae20fb0d585d (DEFAULT_TABLE table=default.Table_93c8b5e956214e88abc353bfb46c8757 [id=1cb84b11b2f34287bbade3af6cd4df67]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:23.227131 998 leader_election.cc:343] T ffd80badce1048f3ac95366cad05ae53 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:23.227833 859 leader_election.cc:304] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:23.228309 1863 raft_consensus.cc:2804] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:23.228439 1863 raft_consensus.cc:493] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:23.228543 1863 raft_consensus.cc:3060] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:23.231240 1863 raft_consensus.cc:515] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.231597 715 leader_election.cc:304] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:23.231915 1850 raft_consensus.cc:2804] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260202 02:52:23.231976 742 tablet_service.cc:1552] Processing DeleteTablet for tablet 2b352d9a815e4bccafb7e2251500c5bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:22 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:23.232044 1850 raft_consensus.cc:493] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:23.232012 1040 tablet_service.cc:1552] Processing DeleteTablet for tablet 08d8d781bb4e4acf915f11afb4fb2e22 with delete_type TABLET_DATA_DELETED (replaced by tablet 7921daa000f54c8dbed04473e21cf0a5) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.232304 1173 tablet_bootstrap.cc:654] T d21fca9a5b234c26879717f51c8ea0f0 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:23.232414 742 tablet_service.cc:1552] Processing DeleteTablet for tablet 08d8d781bb4e4acf915f11afb4fb2e22 with delete_type TABLET_DATA_DELETED (replaced by tablet 7921daa000f54c8dbed04473e21cf0a5) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:23.232133 1850 raft_consensus.cc:3060] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:23.232637 1863 leader_election.cc:290] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:23.232944 749 tablet_service.cc:1505] Processing CreateTablet for tablet b9fc33e8efc04734a951f7fcf9048142 (DEFAULT_TABLE table=default.Table_fbe3212129904307af7133ead4053a59 [id=0aeb16c903f642d8bfa236a96e203767]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.233405 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9fc33e8efc04734a951f7fcf9048142. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.233521 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d1e0e74f32a24729b250c5751677cfb0" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:23.233651 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 94bada52db6a47179415d302b541d01f (table default.Table_086abcdd67e94b38ac7f5d98777f5c6b [id=32fe1d8734de41cfae07a4662814987e]) successfully deleted
I20260202 02:52:23.236166 1850 raft_consensus.cc:515] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.233704 1182 tablet_replica.cc:333] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
W20260202 02:52:23.234056 857 leader_election.cc:343] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:23.236658 1182 raft_consensus.cc:2243] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:23.236850 1182 raft_consensus.cc:2272] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:23.236958 1850 leader_election.cc:290] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:23.237711 1174 tablet_bootstrap.cc:492] T ef073880898049e2be93d2cb6dc0c0bb P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:23.237861 1174 ts_tablet_manager.cc:1403] T ef073880898049e2be93d2cb6dc0c0bb P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.106s user 0.003s sys 0.000s
I20260202 02:52:23.238637 1174 raft_consensus.cc:359] T ef073880898049e2be93d2cb6dc0c0bb P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.238812 1174 raft_consensus.cc:385] T ef073880898049e2be93d2cb6dc0c0bb P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:23.238899 1174 raft_consensus.cc:740] T ef073880898049e2be93d2cb6dc0c0bb P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:23.239084 1174 consensus_queue.cc:260] T ef073880898049e2be93d2cb6dc0c0bb P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.239789 1174 ts_tablet_manager.cc:1434] T ef073880898049e2be93d2cb6dc0c0bb P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260202 02:52:23.240809 1174 tablet_bootstrap.cc:492] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:23.243633 1877 raft_consensus.cc:2804] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:23.243757 1877 raft_consensus.cc:493] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:23.243938 998 leader_election.cc:304] T ffd80badce1048f3ac95366cad05ae53 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:23.242017 1182 ts_tablet_manager.cc:1916] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:23.247704 1044 tablet_service.cc:1505] Processing CreateTablet for tablet c977c146bac34638874b7a990604a0d5 (DEFAULT_TABLE table=default.Table_61741238f8ad47cbb8fe712419268ff0 [id=6b1b33ede4134cbe83d015e4b958f1a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.247920 1039 tablet_service.cc:1505] Processing CreateTablet for tablet 7f9dfac19c04475aadb45e95a6474eb4 (DEFAULT_TABLE table=default.Table_c2f3ac0894bd4f4f9702b6429da397bc [id=67e7356eb4d0432f884247749fb97bac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.248926 1032 tablet_service.cc:1505] Processing CreateTablet for tablet 7f323175ee254df789bcb04e65047dff (DEFAULT_TABLE table=default.Table_783c68f0f6ad41899a919edefc2e85df [id=0d6674bc4f264a36aad4ea7962a9e0e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.249087 1860 raft_consensus.cc:2749] T ffd80badce1048f3ac95366cad05ae53 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260202 02:52:23.249773 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9fc33e8efc04734a951f7fcf9048142. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.256912 1031 tablet_service.cc:1552] Processing DeleteTablet for tablet 26eeef6eebbf4f2cb44f69255fb37594 with delete_type TABLET_DATA_DELETED (replaced by tablet dc4c3f52f4f44a84b0d0e3c9a4a49f09) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.257879 893 tablet_service.cc:1505] Processing CreateTablet for tablet c9ef08b5830147a3a55049285a40535b (DEFAULT_TABLE table=default.Table_ac6e92f3b1f540a7af72ae905e0481a7 [id=63269345f7e842dd9d4d250191e12c6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.258356 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9ef08b5830147a3a55049285a40535b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.260273 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet f58a7b25815c4b5bac568aa11c19610c (table default.Table_cc069a0949364216b8f08d86a0d9c776 [id=c97e407e519d4a538860f66931f6aefd]) successfully deleted
W20260202 02:52:23.261109 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 94bada52db6a47179415d302b541d01f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 94bada52db6a47179415d302b541d01f already in progress: creating tablet
I20260202 02:52:23.264809 521 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_444485558ccc4c6ba771ab686bc9e1c1" } modify_external_catalogs: true
I20260202 02:52:23.265084 521 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_444485558ccc4c6ba771ab686bc9e1c1" } modify_external_catalogs: true
I20260202 02:52:23.268004 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_17dd200ab5824a4a90f05d78d31c6887"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:23.268576 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_17dd200ab5824a4a90f05d78d31c6887 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:23.268716 523 catalog_manager.cc:5654] T 70306ed9dec54286885d965f2c012da4 P 1f442ef85029461091de847118c34c0d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f442ef85029461091de847118c34c0d (127.0.104.129). New cstate: current_term: 1 leader_uuid: "1f442ef85029461091de847118c34c0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } }
I20260202 02:52:23.269161 523 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet 70306ed9dec54286885d965f2c012da4 (table default.Table_d0e51ff8e83f4da19ce3c119dbec1ea7 [id=31abb0facd164caaa15591259eb951c2]). Expected version 1 got 0
I20260202 02:52:23.270128 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_aa12e8c2bff14a968a174b8879bba516"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:23.270659 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aa12e8c2bff14a968a174b8879bba516 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:23.273000 528 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_cc069a0949364216b8f08d86a0d9c776" } new_table_name: "default.Table_e5f4ac82d1ef4b85999695997eb9df2a" modify_external_catalogs: true
I20260202 02:52:23.273183 519 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_2be7fcc0f92846059283a7691fcda407" } new_table_name: "default.Table_698a93b0be2e4203836e0522ac44acc3" modify_external_catalogs: true
I20260202 02:52:23.273768 527 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_67c55c53ca254b03b0c326672a41b559" } modify_external_catalogs: true
I20260202 02:52:23.273948 527 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_67c55c53ca254b03b0c326672a41b559" } modify_external_catalogs: true
I20260202 02:52:23.274762 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_ae54fc1a1cfc4475823c8c3b5cf86e5f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:23.275362 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ae54fc1a1cfc4475823c8c3b5cf86e5f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:23.275573 521 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 07580ad1d049475a9645acfad0b1d219
I20260202 02:52:23.277024 521 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_9913ba1a0b0c4f30bc3dd302ee61b92d" } modify_external_catalogs: true
I20260202 02:52:23.277356 521 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_9913ba1a0b0c4f30bc3dd302ee61b92d" } modify_external_catalogs: true
I20260202 02:52:23.279139 899 tablet_service.cc:1552] Processing DeleteTablet for tablet 07580ad1d049475a9645acfad0b1d219 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:23.253530 742 tablet_service.cc:1505] Processing CreateTablet for tablet bbcc723a8079443db455ae20fb0d585d (DEFAULT_TABLE table=default.Table_93c8b5e956214e88abc353bfb46c8757 [id=1cb84b11b2f34287bbade3af6cd4df67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.288821 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbcc723a8079443db455ae20fb0d585d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.290884 1034 tablet_service.cc:1552] Processing DeleteTablet for tablet 526f2c7b95184a0987e7aa94efc8475a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.291153 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet eb0ac55fbb534b29a86d6b3e8aa72ebc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.292595 1060 raft_consensus.cc:2468] T 622a113af04a41c6852c29fc8a8acd78 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
I20260202 02:52:23.293385 859 leader_election.cc:304] T 622a113af04a41c6852c29fc8a8acd78 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:23.254420 741 tablet_service.cc:1505] Processing CreateTablet for tablet cd1b6662da254889ae4bbe8d0b1ef1bb (DEFAULT_TABLE table=default.Table_b0e344dc1a824cd9b41aa8991b7911dd [id=8e640d1261014706a067f522bda7ea61]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:23.294065 528 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cb634d4bae554ef3b39eeaa9958adc6c consensus state has no leader
I20260202 02:52:23.294152 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd1b6662da254889ae4bbe8d0b1ef1bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.295536 1863 raft_consensus.cc:2804] T 622a113af04a41c6852c29fc8a8acd78 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:23.295749 1863 raft_consensus.cc:697] T 622a113af04a41c6852c29fc8a8acd78 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:23.296036 1863 consensus_queue.cc:237] T 622a113af04a41c6852c29fc8a8acd78 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:23.255005 745 tablet_service.cc:1505] Processing CreateTablet for tablet c977c146bac34638874b7a990604a0d5 (DEFAULT_TABLE table=default.Table_61741238f8ad47cbb8fe712419268ff0 [id=6b1b33ede4134cbe83d015e4b958f1a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.296655 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c977c146bac34638874b7a990604a0d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.255375 746 tablet_service.cc:1552] Processing DeleteTablet for tablet dada209d982d48f3b74a49905f19c5b6 with delete_type TABLET_DATA_DELETED (replaced by tablet 67a8c9a1998b47b3b214a06ce62381e9) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:23.255679 744 tablet_service.cc:1552] Processing DeleteTablet for tablet d64f42d187bc4c9f9381c931449ec0cc with delete_type TABLET_DATA_DELETED (replaced by tablet e95153ee8f2d40de955e3389d3a58644) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:23.255971 755 tablet_service.cc:1552] Processing DeleteTablet for tablet 26eeef6eebbf4f2cb44f69255fb37594 with delete_type TABLET_DATA_DELETED (replaced by tablet dc4c3f52f4f44a84b0d0e3c9a4a49f09) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:23.256491 758 tablet_service.cc:1505] Processing CreateTablet for tablet 09504070c9ec4bcfbb59e3fb4c189775 (DEFAULT_TABLE table=default.Table_3a2daddab50e4578898828f3a305b8e7 [id=4c17da59a0554cee93ce4e626838a2c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.301416 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09504070c9ec4bcfbb59e3fb4c189775. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.256956 759 tablet_service.cc:1505] Processing CreateTablet for tablet 3dcf4f794ec14b2f9d181d015fc437fb (DEFAULT_TABLE table=default.Table_bb45d0e2e2a14038aec12db59746877a [id=2a786b768e5e436f86e56c725c9c90a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.304701 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dcf4f794ec14b2f9d181d015fc437fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.304844 893 tablet_service.cc:1505] Processing CreateTablet for tablet a81e1d10ca49459989260fbd6a2b8645 (DEFAULT_TABLE table=default.Table_aafa37fc93f045a2b0525cea42a60866 [id=75cfcae38a6344b190bfee547469cab7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.305272 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a81e1d10ca49459989260fbd6a2b8645. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.305738 1877 raft_consensus.cc:3060] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:23.306253 998 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 48661.4us
I20260202 02:52:23.307735 1721 raft_consensus.cc:493] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:23.307901 1721 raft_consensus.cc:515] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.308614 1721 leader_election.cc:290] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:23.309119 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "94bada52db6a47179415d302b541d01f" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:23.310137 1764 raft_consensus.cc:493] T d64f42d187bc4c9f9381c931449ec0cc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:23.257429 750 tablet_service.cc:1505] Processing CreateTablet for tablet 604b3b863c9540d3b55381dbfe1afdc3 (DEFAULT_TABLE table=default.Table_89983dadd3fc4966bf26418544c1729a [id=ce752170017a4e119a8c42c73fd95302]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.310308 1764 raft_consensus.cc:515] T d64f42d187bc4c9f9381c931449ec0cc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:23.310544 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 604b3b863c9540d3b55381dbfe1afdc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.310946 1764 leader_election.cc:290] T d64f42d187bc4c9f9381c931449ec0cc P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:23.311174 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d1e0e74f32a24729b250c5751677cfb0" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:23.311340 1059 raft_consensus.cc:3060] T d1e0e74f32a24729b250c5751677cfb0 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:23.314102 1059 raft_consensus.cc:2468] T d1e0e74f32a24729b250c5751677cfb0 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
I20260202 02:52:23.314373 1037 tablet_service.cc:1552] Processing DeleteTablet for tablet dada209d982d48f3b74a49905f19c5b6 with delete_type TABLET_DATA_DELETED (replaced by tablet 67a8c9a1998b47b3b214a06ce62381e9) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.314572 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c977c146bac34638874b7a990604a0d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.257884 747 tablet_service.cc:1505] Processing CreateTablet for tablet 2f664f6119ba4d28b5ef1c212e611866 (DEFAULT_TABLE table=default.Table_cc8d0231630743f180159c4d3a4efab8 [id=0dc5dc53fc014dd89bc66ac6260de9e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.316545 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f323175ee254df789bcb04e65047dff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.316900 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f664f6119ba4d28b5ef1c212e611866. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.294241 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bbf31e5619ed4f60b4d7c8b19ec8a549" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:23.319187 1040 tablet_service.cc:1505] Processing CreateTablet for tablet cd1b6662da254889ae4bbe8d0b1ef1bb (DEFAULT_TABLE table=default.Table_b0e344dc1a824cd9b41aa8991b7911dd [id=8e640d1261014706a067f522bda7ea61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.258356 754 tablet_service.cc:1505] Processing CreateTablet for tablet 7921daa000f54c8dbed04473e21cf0a5 (DEFAULT_TABLE table=default.Table_2acd399bc33146fe9f1b6e2a15e0b101 [id=487a204c7845401ab7391ae8464fdf1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.320650 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7921daa000f54c8dbed04473e21cf0a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.321007 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d64f42d187bc4c9f9381c931449ec0cc" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:23.321700 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ef073880898049e2be93d2cb6dc0c0bb" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:23.321923 779 raft_consensus.cc:2468] T ef073880898049e2be93d2cb6dc0c0bb P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:23.323107 859 leader_election.cc:304] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:23.323653 1865 raft_consensus.cc:2804] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:23.323824 1865 raft_consensus.cc:697] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:23.324085 1865 consensus_queue.cc:237] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.258802 757 tablet_service.cc:1505] Processing CreateTablet for tablet 1959438c70c544e69422f7354082cf89 (DEFAULT_TABLE table=default.Table_eeb507a1a49d407196bff72270b0834f [id=a18136bbb87a47d69600f926a474da24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.324642 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1959438c70c544e69422f7354082cf89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.259248 751 tablet_service.cc:1505] Processing CreateTablet for tablet 842197c689064660b791b40e521d346b (DEFAULT_TABLE table=default.Table_24d7cc37a3884cda9a1e269515d3877d [id=61117090be4347b6a7e5a88aaa765ffc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.328593 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f9dfac19c04475aadb45e95a6474eb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.328752 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 842197c689064660b791b40e521d346b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.330796 1042 tablet_service.cc:1552] Processing DeleteTablet for tablet 75b97d7e848241cf8d894b0ae92a8f8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:23.331708 998 leader_election.cc:343] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:23.332146 1175 tablet_bootstrap.cc:492] T 59990791123c4d4eb620090a1e8e31e1 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:23.332304 1175 ts_tablet_manager.cc:1403] T 59990791123c4d4eb620090a1e8e31e1 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.412s user 0.021s sys 0.040s
I20260202 02:52:23.333287 917 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ef073880898049e2be93d2cb6dc0c0bb" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:23.333541 917 raft_consensus.cc:2468] T ef073880898049e2be93d2cb6dc0c0bb P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:23.333781 899 tablet_service.cc:1505] Processing CreateTablet for tablet f6c2bc531d194a4b8f3062c9b40e3da6 (DEFAULT_TABLE table=default.Table_44086d9f3e324314af6a63c86ade13c9 [id=cc2b6a4c360a48cebe72c981b47d708a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.334290 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6c2bc531d194a4b8f3062c9b40e3da6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.335403 1175 raft_consensus.cc:359] T 59990791123c4d4eb620090a1e8e31e1 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.335613 1175 raft_consensus.cc:385] T 59990791123c4d4eb620090a1e8e31e1 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:23.335729 1175 raft_consensus.cc:740] T 59990791123c4d4eb620090a1e8e31e1 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:23.333781 898 tablet_service.cc:1505] Processing CreateTablet for tablet dc4c3f52f4f44a84b0d0e3c9a4a49f09 (DEFAULT_TABLE table=default.Table_82c14c20a7bf41b487efaff52ebc15a9 [id=7fbb33c105f149ef98505dcf622930c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.336220 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbcc723a8079443db455ae20fb0d585d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.336496 527 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 1f379a29e2454f689d68f5951837512b
I20260202 02:52:23.338687 893 tablet_service.cc:1505] Processing CreateTablet for tablet e95153ee8f2d40de955e3389d3a58644 (DEFAULT_TABLE table=default.Table_0c83932cab13463f9cca28129f18b576 [id=7c76848b98614fbf8b2950e5d55cc609]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.340494 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc4c3f52f4f44a84b0d0e3c9a4a49f09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.341677 1035 tablet_service.cc:1552] Processing DeleteTablet for tablet ffd80badce1048f3ac95366cad05ae53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.341908 1028 tablet_service.cc:1552] Processing DeleteTablet for tablet eaa47f3388144e8b852804c167f232ce with delete_type TABLET_DATA_DELETED (replaced by tablet 38fdbc6df7ec40b9afe4e0dfe0119e3e) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.342139 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet 80d443bdb66d4cc68ae283e08c410fe6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.342298 1064 raft_consensus.cc:1275] T 8095e0c6ff0041fc8b8c8b0778a20b00 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Refusing update from remote peer 1f442ef85029461091de847118c34c0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260202 02:52:23.342592 1031 tablet_service.cc:1552] Processing DeleteTablet for tablet d64f42d187bc4c9f9381c931449ec0cc with delete_type TABLET_DATA_DELETED (replaced by tablet e95153ee8f2d40de955e3389d3a58644) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.343261 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 1f379a29e2454f689d68f5951837512b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:23.343683 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd1b6662da254889ae4bbe8d0b1ef1bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.344429 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet d1e0e74f32a24729b250c5751677cfb0 with delete_type TABLET_DATA_DELETED (replaced by tablet 2c31b45dc8004eae993e972e9faa3ff8) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:23.345319 1875 log.cc:927] Time spent T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d: Append to log took a long time: real 0.066s user 0.000s sys 0.000s
I20260202 02:52:23.345690 1063 raft_consensus.cc:3060] T bbf31e5619ed4f60b4d7c8b19ec8a549 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:23.340502 1175 consensus_queue.cc:260] T 59990791123c4d4eb620090a1e8e31e1 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.353158 902 tablet_service.cc:1505] Processing CreateTablet for tablet bb1005103c25423783736ace1866c328 (DEFAULT_TABLE table=default.Table_ed899eb11436405fa74f3b38d08552f7 [id=15a0348866d34a209fb8332e17c91ae0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.357167 1867 raft_consensus.cc:493] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:23.357388 1867 raft_consensus.cc:515] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.358117 1867 leader_election.cc:290] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:23.358744 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f6746ccbf37e477196a9706396d1e3fd" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:23.359488 857 leader_election.cc:343] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:23.359454 1178 raft_consensus.cc:515] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:23.351900 998 leader_election.cc:343] T d64f42d187bc4c9f9381c931449ec0cc P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:23.362448 1860 raft_consensus.cc:493] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:23.362653 1860 raft_consensus.cc:515] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.362807 1178 leader_election.cc:290] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:23.372185 998 consensus_peers.cc:597] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c064ea0dade840b4840a5715c2d83f69. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:23.368402 1860 leader_election.cc:290] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:23.373446 1175 ts_tablet_manager.cc:1434] T 59990791123c4d4eb620090a1e8e31e1 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.041s user 0.003s sys 0.000s
I20260202 02:52:23.374337 1175 tablet_bootstrap.cc:492] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:23.374601 1723 consensus_queue.cc:1048] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:23.374787 889 tablet_service.cc:1505] Processing CreateTablet for tablet 67a8c9a1998b47b3b214a06ce62381e9 (DEFAULT_TABLE table=default.Table_7a967fd89a5243c4a02abdaf16f6d62e [id=116247212fca4c0a95ad1bdbbac4787f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.375344 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e95153ee8f2d40de955e3389d3a58644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.375869 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67a8c9a1998b47b3b214a06ce62381e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.375828 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f6746ccbf37e477196a9706396d1e3fd" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:23.377849 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb1005103c25423783736ace1866c328. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.382395 1850 raft_consensus.cc:493] T 75b97d7e848241cf8d894b0ae92a8f8f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:23.382725 1850 raft_consensus.cc:515] T 75b97d7e848241cf8d894b0ae92a8f8f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.383751 1850 leader_election.cc:290] T 75b97d7e848241cf8d894b0ae92a8f8f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:23.384244 1042 tablet_service.cc:1552] Processing DeleteTablet for tablet 9884c90cf1d245b2a7eaaf00d71b1c3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.384513 999 leader_election.cc:304] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
W20260202 02:52:23.384902 999 consensus_peers.cc:597] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 267bd4752c5f4b99a3fe778ba8f38a2f. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:23.385080 999 leader_election.cc:343] T d64f42d187bc4c9f9381c931449ec0cc P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: d64f42d187bc4c9f9381c931449ec0cc
I20260202 02:52:23.385182 999 leader_election.cc:304] T d64f42d187bc4c9f9381c931449ec0cc P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:23.385449 1178 raft_consensus.cc:2804] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:23.385535 1178 raft_consensus.cc:493] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:23.385634 1178 raft_consensus.cc:3060] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:23.386750 1860 raft_consensus.cc:2749] T d64f42d187bc4c9f9381c931449ec0cc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:23.386991 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet 8095e0c6ff0041fc8b8c8b0778a20b00 with delete_type TABLET_DATA_DELETED (replaced by tablet f5727f9b9bbb4e77abfef065c51606a1) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.387220 1764 consensus_queue.cc:1048] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.055s
I20260202 02:52:23.389660 918 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f54c1dfe0a942b3bafd85522953a032" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:23.389922 918 raft_consensus.cc:2468] T 1f54c1dfe0a942b3bafd85522953a032 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:23.390385 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f54c1dfe0a942b3bafd85522953a032" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:23.390830 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "eaa47f3388144e8b852804c167f232ce" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:23.391012 772 raft_consensus.cc:3060] T eaa47f3388144e8b852804c167f232ce P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:23.262020 714 leader_election.cc:343] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 526f2c7b95184a0987e7aa94efc8475a
I20260202 02:52:23.392197 1182 ts_tablet_manager.cc:1929] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260202 02:52:23.393021 999 leader_election.cc:343] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: eaa47f3388144e8b852804c167f232ce
I20260202 02:52:23.393247 999 leader_election.cc:304] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:23.394040 1764 raft_consensus.cc:2804] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:23.394161 1764 raft_consensus.cc:493] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:23.394148 772 raft_consensus.cc:2468] T eaa47f3388144e8b852804c167f232ce P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:23.394238 1764 raft_consensus.cc:3060] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:23.395658 999 consensus_peers.cc:597] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c064ea0dade840b4840a5715c2d83f69. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:23.395951 1182 log.cc:1199] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/2f0e8c5e2fd44370ae150a206f4e6cbc
W20260202 02:52:23.395973 998 leader_election.cc:343] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:23.396438 998 leader_election.cc:304] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:23.396806 1182 ts_tablet_manager.cc:1950] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:23.396907 1721 consensus_queue.cc:1048] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:23.397362 1721 raft_consensus.cc:2804] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:23.397511 1721 raft_consensus.cc:697] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:23.397773 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 2f0e8c5e2fd44370ae150a206f4e6cbc (table default.Table_04337834a3ce43ae8f6012b818d70d9f [id=3d1bb64023a145bb9fdcb2378a844a37]) successfully deleted
I20260202 02:52:23.397770 1721 consensus_queue.cc:237] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:23.398689 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet d67e91a022f1401ea724dff898c3b81b because the tablet was not found. No further retry: Not found: Tablet not found: d67e91a022f1401ea724dff898c3b81b
I20260202 02:52:23.398990 1182 tablet_replica.cc:333] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:23.401726 1175 tablet_bootstrap.cc:654] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:23.405499 527 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_cafbdde19f984e0fa22b1fde486f8e7e" } modify_external_catalogs: true
I20260202 02:52:23.413607 527 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_cafbdde19f984e0fa22b1fde486f8e7e" } modify_external_catalogs: true
W20260202 02:52:23.413759 519 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 587cfd0ae62f4cfd937faf6890e78f4a consensus state has no leader
I20260202 02:52:23.415637 521 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 874ca4ba4ebc4721bb4b60cba8e01841
I20260202 02:52:23.417508 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_dc83c58402214d3eb5cd548d2bc9a1a6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:23.417918 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_e7dff4eb18124743848a9f61ba603f6f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:23.418221 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dc83c58402214d3eb5cd548d2bc9a1a6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:23.418646 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e7dff4eb18124743848a9f61ba603f6f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:23.419572 523 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_12db441a8c934f9780599755191c58f5" } new_table_name: "default.Table_6a546caebf9e4075816236ca1bd8d73f" modify_external_catalogs: true
I20260202 02:52:23.420025 521 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_7d0cce2fe6094a54a4b51acf6b1d42da" } new_table_name: "default.Table_6b9de049c34449509645f14959ae1200" modify_external_catalogs: true
I20260202 02:52:23.421046 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_b9ea6590fd0b48608fc1ebe68e7eae9b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:23.421449 527 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 938813ee0a8a40b6bf0e537e3ec1a0d4
W20260202 02:52:23.421643 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b9ea6590fd0b48608fc1ebe68e7eae9b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:23.422497 714 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 160457us
W20260202 02:52:23.422787 714 leader_election.cc:343] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 526f2c7b95184a0987e7aa94efc8475a
W20260202 02:52:23.423135 714 consensus_peers.cc:597] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ff6f5710de4b4c48948aacdf5be29e8b. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:23.423321 714 consensus_peers.cc:597] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c12ff20ca6b44ddba0c936a28cfb2eb9. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:23.423583 714 consensus_peers.cc:597] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8095e0c6ff0041fc8b8c8b0778a20b00. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:23.423806 714 consensus_peers.cc:597] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c63a670d5c5e489897bc6dbb142b844d. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:23.424011 714 consensus_peers.cc:597] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d3633f5f9f54115bc78b9c1b78aba67. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:23.424635 1033 tablet_service.cc:1552] Processing DeleteTablet for tablet 75f7824681344d8d88df9402694fff1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.425866 526 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_0c83932cab13463f9cca28129f18b576" } modify_external_catalogs: true
I20260202 02:52:23.426084 526 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_0c83932cab13463f9cca28129f18b576" } modify_external_catalogs: true
I20260202 02:52:23.427649 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_8fcc3828b43444159f0df6023ec1ba0f" } new_table_name: "default.Table_bde65e061a274734a7df7a428bd8ce53" modify_external_catalogs: true
I20260202 02:52:23.429590 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_2c56436bf39647f2b5af982d04f990dc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:23.430250 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2c56436bf39647f2b5af982d04f990dc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:23.434077 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_b9aa847125ed4116b6fde1cfc1cafa27"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:23.434340 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_aee0b3d530954ca3ba6492f8263dd7ff"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:23.434857 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aee0b3d530954ca3ba6492f8263dd7ff in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:23.435103 1884 tablet.cc:1722] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
W20260202 02:52:23.434871 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b9aa847125ed4116b6fde1cfc1cafa27 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:23.437809 1178 raft_consensus.cc:515] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.439024 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ef073880898049e2be93d2cb6dc0c0bb" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:23.439231 775 raft_consensus.cc:3060] T ef073880898049e2be93d2cb6dc0c0bb P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:23.440097 1180 ts_tablet_manager.cc:1929] T c184d44d7f434b39a997a7387e5d3de0 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:23.440245 1180 log.cc:1199] T c184d44d7f434b39a997a7387e5d3de0 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/c184d44d7f434b39a997a7387e5d3de0
I20260202 02:52:23.440697 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_077acfd01c1648ef96128a866323ec3e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:23.441322 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_077acfd01c1648ef96128a866323ec3e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:23.441995 775 raft_consensus.cc:2468] T ef073880898049e2be93d2cb6dc0c0bb P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:23.443814 918 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ef073880898049e2be93d2cb6dc0c0bb" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:23.444049 918 raft_consensus.cc:3060] T ef073880898049e2be93d2cb6dc0c0bb P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:23.444618 1174 tablet_bootstrap.cc:654] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:23.445773 714 leader_election.cc:343] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: bbf31e5619ed4f60b4d7c8b19ec8a549
W20260202 02:52:23.446046 714 leader_election.cc:343] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: bbf31e5619ed4f60b4d7c8b19ec8a549
I20260202 02:52:23.446623 1033 tablet_service.cc:1552] Processing DeleteTablet for tablet f93e8d3c4faf48eaa98c3cbf88d585e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.446695 918 raft_consensus.cc:2468] T ef073880898049e2be93d2cb6dc0c0bb P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:23.449005 1178 leader_election.cc:290] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:23.450126 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 874ca4ba4ebc4721bb4b60cba8e01841 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:23.450480 523 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 550895b4010f4088b2cc34acff50e03f consensus state has no leader
I20260202 02:52:23.450816 523 master_service.cc:954] ReplaceTablet: received request to replace tablet 33150e1456144b8e9c64cc2a2ac6dcca from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:23.451084 1042 tablet_service.cc:1552] Processing DeleteTablet for tablet 0186572e7bb9499a932a636ab280f544 with delete_type TABLET_DATA_DELETED (replaced by tablet 79c73f4a92044bafb5b8384229cb3a4d) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.451874 1033 tablet_service.cc:1552] Processing DeleteTablet for tablet d67e91a022f1401ea724dff898c3b81b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.452080 998 leader_election.cc:304] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
W20260202 02:52:23.452636 859 leader_election.cc:343] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:23.452726 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet b428356fba0c4fb5a6033c5d39740cc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.452809 859 leader_election.cc:304] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:23.452857 1867 raft_consensus.cc:493] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:23.453018 1867 raft_consensus.cc:515] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.453568 1777 raft_consensus.cc:2749] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:23.453698 1867 leader_election.cc:290] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:23.454744 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e1284fde67bc4d65b067a54c416ccc33" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:23.455341 857 leader_election.cc:343] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:23.457085 519 master_service.cc:954] ReplaceTablet: received request to replace tablet bd7b0a0fcb7645fdb9edfeccb3d18268 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:23.458137 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "75b97d7e848241cf8d894b0ae92a8f8f" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:23.460305 521 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b427c7e897c64977b924e7d586420e69 consensus state has no leader
I20260202 02:52:23.465647 1064 raft_consensus.cc:2393] T 75b97d7e848241cf8d894b0ae92a8f8f P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f442ef85029461091de847118c34c0d in current term 1: Already voted for candidate b1f05820b6034228bdad884cc348716d in this term.
I20260202 02:52:23.467784 902 tablet_service.cc:1552] Processing DeleteTablet for tablet 938813ee0a8a40b6bf0e537e3ec1a0d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:23.470806 1033 tablet_service.cc:1552] Processing DeleteTablet for tablet 929eb48881b141718873d0e359e768a8 with delete_type TABLET_DATA_DELETED (replaced by tablet cd1b6662da254889ae4bbe8d0b1ef1bb) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.471825 1042 tablet_service.cc:1505] Processing CreateTablet for tablet 3dcf4f794ec14b2f9d181d015fc437fb (DEFAULT_TABLE table=default.Table_bb45d0e2e2a14038aec12db59746877a [id=2a786b768e5e436f86e56c725c9c90a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.472290 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dcf4f794ec14b2f9d181d015fc437fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.472990 1884 tablet.cc:1722] T c12ff20ca6b44ddba0c936a28cfb2eb9 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:23.473064 742 tablet_service.cc:1505] Processing CreateTablet for tablet c9ef08b5830147a3a55049285a40535b (DEFAULT_TABLE table=default.Table_ac6e92f3b1f540a7af72ae905e0481a7 [id=63269345f7e842dd9d4d250191e12c6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.473533 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9ef08b5830147a3a55049285a40535b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.477339 526 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet e95153ee8f2d40de955e3389d3a58644
I20260202 02:52:23.487684 902 tablet_service.cc:1552] Processing DeleteTablet for tablet e95153ee8f2d40de955e3389d3a58644 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:23.494179 1180 ts_tablet_manager.cc:1950] T c184d44d7f434b39a997a7387e5d3de0 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:23.495239 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 271) took 3773 ms. Trace:
I20260202 02:52:23.495453 1180 rpcz_store.cc:276] 0202 02:52:19.721384 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.725832 (+ 4448us) service_pool.cc:225] Handling call
0202 02:52:23.210312 (+3484480us) ts_tablet_manager.cc:1151] Deleting tablet c184d44d7f434b39a997a7387e5d3de0
0202 02:52:23.210322 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:23.214799 (+ 4477us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:23.439853 (+225054us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:23.495195 (+ 55342us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3474376}
I20260202 02:52:23.496176 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 274) took 3758 ms. Trace:
I20260202 02:52:23.496456 1180 rpcz_store.cc:276] 0202 02:52:19.737713 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.737896 (+ 183us) service_pool.cc:225] Handling call
0202 02:52:23.495918 (+3758022us) ts_tablet_manager.cc:1151] Deleting tablet ffd80badce1048f3ac95366cad05ae53
0202 02:52:23.495929 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:23.496137 (+ 208us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3738268}
I20260202 02:52:23.499377 1182 raft_consensus.cc:2243] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:23.499989 1182 pending_rounds.cc:70] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5: Trying to abort 1 pending ops.
I20260202 02:52:23.500065 1182 pending_rounds.cc:77] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923044657704960 op_type: NO_OP noop_request { }
I20260202 02:52:23.500200 1182 raft_consensus.cc:2889] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:23.500300 1182 raft_consensus.cc:2272] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:23.507560 1887 consensus_queue.cc:1175] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "1521a6d262b7451bbef8debe970dd6b5" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20260202 02:52:23.508911 1182 ts_tablet_manager.cc:1916] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:23.515193 1182 ts_tablet_manager.cc:1929] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:23.515362 1182 log.cc:1199] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/d1e0e74f32a24729b250c5751677cfb0
I20260202 02:52:23.515770 1182 ts_tablet_manager.cc:1950] T d1e0e74f32a24729b250c5751677cfb0 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:23.516891 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 94bada52db6a47179415d302b541d01f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.519147 1182 tablet_replica.cc:333] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:23.519485 1182 raft_consensus.cc:2243] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:23.520102 1182 raft_consensus.cc:2272] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:23.503449 745 tablet_service.cc:1552] Processing DeleteTablet for tablet 526f2c7b95184a0987e7aa94efc8475a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:23.503806 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 75b97d7e848241cf8d894b0ae92a8f8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:23.504091 757 tablet_service.cc:1552] Processing DeleteTablet for tablet f58a7b25815c4b5bac568aa11c19610c with delete_type TABLET_DATA_DELETED (replaced by tablet cb634d4bae554ef3b39eeaa9958adc6c) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:23.504655 754 tablet_service.cc:1505] Processing CreateTablet for tablet a81e1d10ca49459989260fbd6a2b8645 (DEFAULT_TABLE table=default.Table_aafa37fc93f045a2b0525cea42a60866 [id=75cfcae38a6344b190bfee547469cab7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.521073 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a81e1d10ca49459989260fbd6a2b8645. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.521418 1182 ts_tablet_manager.cc:1916] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:23.505177 759 tablet_service.cc:1505] Processing CreateTablet for tablet e95153ee8f2d40de955e3389d3a58644 (DEFAULT_TABLE table=default.Table_0c83932cab13463f9cca28129f18b576 [id=7c76848b98614fbf8b2950e5d55cc609]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.524854 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e95153ee8f2d40de955e3389d3a58644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.528436 1038 tablet_service.cc:1505] Processing CreateTablet for tablet 09504070c9ec4bcfbb59e3fb4c189775 (DEFAULT_TABLE table=default.Table_3a2daddab50e4578898828f3a305b8e7 [id=4c17da59a0554cee93ce4e626838a2c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.506188 751 tablet_service.cc:1505] Processing CreateTablet for tablet f6c2bc531d194a4b8f3062c9b40e3da6 (DEFAULT_TABLE table=default.Table_44086d9f3e324314af6a63c86ade13c9 [id=cc2b6a4c360a48cebe72c981b47d708a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.528901 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09504070c9ec4bcfbb59e3fb4c189775. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.528951 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6c2bc531d194a4b8f3062c9b40e3da6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.506740 747 tablet_service.cc:1505] Processing CreateTablet for tablet dc4c3f52f4f44a84b0d0e3c9a4a49f09 (DEFAULT_TABLE table=default.Table_82c14c20a7bf41b487efaff52ebc15a9 [id=7fbb33c105f149ef98505dcf622930c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.532714 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc4c3f52f4f44a84b0d0e3c9a4a49f09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.532617 1182 ts_tablet_manager.cc:1929] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20260202 02:52:23.532892 1182 log.cc:1199] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/938813ee0a8a40b6bf0e537e3ec1a0d4
I20260202 02:52:23.533716 1182 ts_tablet_manager.cc:1950] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:23.507267 758 tablet_service.cc:1505] Processing CreateTablet for tablet 67a8c9a1998b47b3b214a06ce62381e9 (DEFAULT_TABLE table=default.Table_7a967fd89a5243c4a02abdaf16f6d62e [id=116247212fca4c0a95ad1bdbbac4787f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.509860 1174 tablet_bootstrap.cc:492] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:23.510370 750 tablet_service.cc:1552] Processing DeleteTablet for tablet 1f379a29e2454f689d68f5951837512b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:23.510964 755 tablet_service.cc:1505] Processing CreateTablet for tablet bb1005103c25423783736ace1866c328 (DEFAULT_TABLE table=default.Table_ed899eb11436405fa74f3b38d08552f7 [id=15a0348866d34a209fb8332e17c91ae0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.511360 744 tablet_service.cc:1552] Processing DeleteTablet for tablet 874ca4ba4ebc4721bb4b60cba8e01841 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:23.511600 746 tablet_service.cc:1552] Processing DeleteTablet for tablet 938813ee0a8a40b6bf0e537e3ec1a0d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:23.537792 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67a8c9a1998b47b3b214a06ce62381e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.538007 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb1005103c25423783736ace1866c328. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.534274 1063 raft_consensus.cc:2468] T bbf31e5619ed4f60b4d7c8b19ec8a549 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 2.
I20260202 02:52:23.540520 1174 ts_tablet_manager.cc:1403] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.297s user 0.003s sys 0.001s
I20260202 02:52:23.541236 1031 tablet_service.cc:1505] Processing CreateTablet for tablet 7921daa000f54c8dbed04473e21cf0a5 (DEFAULT_TABLE table=default.Table_2acd399bc33146fe9f1b6e2a15e0b101 [id=487a204c7845401ab7391ae8464fdf1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.511861 749 tablet_service.cc:1552] Processing DeleteTablet for tablet e95153ee8f2d40de955e3389d3a58644 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:23.542042 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7921daa000f54c8dbed04473e21cf0a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.543622 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 1959438c70c544e69422f7354082cf89 (DEFAULT_TABLE table=default.Table_eeb507a1a49d407196bff72270b0834f [id=a18136bbb87a47d69600f926a474da24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.544530 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1959438c70c544e69422f7354082cf89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.546250 1174 raft_consensus.cc:359] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.508911 1175 tablet_bootstrap.cc:492] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:23.549823 1175 ts_tablet_manager.cc:1403] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.176s user 0.007s sys 0.000s
I20260202 02:52:23.550266 1174 raft_consensus.cc:385] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:23.553593 1877 raft_consensus.cc:515] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.554344 1174 raft_consensus.cc:740] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:23.555356 1035 tablet_service.cc:1505] Processing CreateTablet for tablet c9ef08b5830147a3a55049285a40535b (DEFAULT_TABLE table=default.Table_ac6e92f3b1f540a7af72ae905e0481a7 [id=63269345f7e842dd9d4d250191e12c6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.555775 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9ef08b5830147a3a55049285a40535b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.557139 1037 tablet_service.cc:1552] Processing DeleteTablet for tablet 07580ad1d049475a9645acfad0b1d219 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.557783 1037 tablet_service.cc:1505] Processing CreateTablet for tablet a81e1d10ca49459989260fbd6a2b8645 (DEFAULT_TABLE table=default.Table_aafa37fc93f045a2b0525cea42a60866 [id=75cfcae38a6344b190bfee547469cab7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.558169 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a81e1d10ca49459989260fbd6a2b8645. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.559810 1036 tablet_service.cc:1505] Processing CreateTablet for tablet e95153ee8f2d40de955e3389d3a58644 (DEFAULT_TABLE table=default.Table_0c83932cab13463f9cca28129f18b576 [id=7c76848b98614fbf8b2950e5d55cc609]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.560236 1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e95153ee8f2d40de955e3389d3a58644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.561470 1034 tablet_service.cc:1505] Processing CreateTablet for tablet f6c2bc531d194a4b8f3062c9b40e3da6 (DEFAULT_TABLE table=default.Table_44086d9f3e324314af6a63c86ade13c9 [id=cc2b6a4c360a48cebe72c981b47d708a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.562059 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6c2bc531d194a4b8f3062c9b40e3da6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.562527 1175 raft_consensus.cc:359] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.562739 1175 raft_consensus.cc:385] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:23.562812 1175 raft_consensus.cc:740] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:23.562431 1174 consensus_queue.cc:260] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.563918 1026 tablet_service.cc:1505] Processing CreateTablet for tablet dc4c3f52f4f44a84b0d0e3c9a4a49f09 (DEFAULT_TABLE table=default.Table_82c14c20a7bf41b487efaff52ebc15a9 [id=7fbb33c105f149ef98505dcf622930c9]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:23.514086 714 leader_election.cc:343] T 75b97d7e848241cf8d894b0ae92a8f8f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 75b97d7e848241cf8d894b0ae92a8f8f
W20260202 02:52:23.564237 714 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 50146.3us
I20260202 02:52:23.564486 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc4c3f52f4f44a84b0d0e3c9a4a49f09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.565303 1174 ts_tablet_manager.cc:1434] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.024s user 0.002s sys 0.003s
I20260202 02:52:23.566244 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet 1f379a29e2454f689d68f5951837512b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.566816 1174 tablet_bootstrap.cc:492] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:23.568665 1028 tablet_service.cc:1505] Processing CreateTablet for tablet 842197c689064660b791b40e521d346b (DEFAULT_TABLE table=default.Table_24d7cc37a3884cda9a1e269515d3877d [id=61117090be4347b6a7e5a88aaa765ffc]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:23.571358 714 consensus_peers.cc:597] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0937b6910534ddc8a2217f0cbd539c5. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:23.563055 1175 consensus_queue.cc:260] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.572778 1175 ts_tablet_manager.cc:1434] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.022s user 0.002s sys 0.000s
W20260202 02:52:23.573659 527 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d0ce2121d65b4a228596e0ca9018883d consensus state has no leader
W20260202 02:52:23.574234 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet ffd80badce1048f3ac95366cad05ae53 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ffd80badce1048f3ac95366cad05ae53 already in progress: creating tablet
I20260202 02:52:23.574569 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 67a8c9a1998b47b3b214a06ce62381e9 (DEFAULT_TABLE table=default.Table_7a967fd89a5243c4a02abdaf16f6d62e [id=116247212fca4c0a95ad1bdbbac4787f]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:23.575161 999 leader_election.cc:343] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 94bada52db6a47179415d302b541d01f
I20260202 02:52:23.577354 1033 tablet_service.cc:1505] Processing CreateTablet for tablet 604b3b863c9540d3b55381dbfe1afdc3 (DEFAULT_TABLE table=default.Table_89983dadd3fc4966bf26418544c1729a [id=ce752170017a4e119a8c42c73fd95302]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.577838 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 842197c689064660b791b40e521d346b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.579236 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 604b3b863c9540d3b55381dbfe1afdc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.580310 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67a8c9a1998b47b3b214a06ce62381e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.580850 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet d1e0e74f32a24729b250c5751677cfb0 (table default.Table_5db99844c1bb484a863a9e289a19da3e [id=f1f6c97fc88340da8bab0347ad6430ab]) successfully deleted
I20260202 02:52:23.581425 1877 leader_election.cc:290] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:23.581658 514 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 63631.1us
I20260202 02:52:23.581811 1877 raft_consensus.cc:493] T 757492288d7c44fb91b25b7e6e21e01b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:23.582052 1877 raft_consensus.cc:515] T 757492288d7c44fb91b25b7e6e21e01b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.582808 1764 raft_consensus.cc:515] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.583405 1764 leader_election.cc:290] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:23.584844 1764 raft_consensus.cc:493] T eb0ac55fbb534b29a86d6b3e8aa72ebc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:23.585043 1764 raft_consensus.cc:515] T eb0ac55fbb534b29a86d6b3e8aa72ebc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:23.585539 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 07580ad1d049475a9645acfad0b1d219 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 07580ad1d049475a9645acfad0b1d219 already in progress: creating tablet
I20260202 02:52:23.585764 1764 leader_election.cc:290] T eb0ac55fbb534b29a86d6b3e8aa72ebc P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:23.585911 1764 raft_consensus.cc:493] T 75f7824681344d8d88df9402694fff1b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260202 02:52:23.585882 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 1f379a29e2454f689d68f5951837512b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1f379a29e2454f689d68f5951837512b already in progress: creating tablet
I20260202 02:52:23.586053 1764 raft_consensus.cc:515] T 75f7824681344d8d88df9402694fff1b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:23.586126 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 874ca4ba4ebc4721bb4b60cba8e01841 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 874ca4ba4ebc4721bb4b60cba8e01841 already in progress: creating tablet
I20260202 02:52:23.586308 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 938813ee0a8a40b6bf0e537e3ec1a0d4 (table default.Table_cafbdde19f984e0fa22b1fde486f8e7e [id=1229c6d37b994a669d3363fdf6f8c6c8]) successfully deleted
W20260202 02:52:23.586495 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet e95153ee8f2d40de955e3389d3a58644 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e95153ee8f2d40de955e3389d3a58644 already in progress: creating tablet
I20260202 02:52:23.586589 1764 leader_election.cc:290] T 75f7824681344d8d88df9402694fff1b P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:23.586721 1764 raft_consensus.cc:493] T 929eb48881b141718873d0e359e768a8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:23.586786 527 master_service.cc:954] ReplaceTablet: received request to replace tablet 6df3d11cf578404ca41807f0777b6c53 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:23.586838 1764 raft_consensus.cc:515] T 929eb48881b141718873d0e359e768a8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.587323 1764 leader_election.cc:290] T 929eb48881b141718873d0e359e768a8 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:23.588146 528 catalog_manager.cc:5654] T 622a113af04a41c6852c29fc8a8acd78 P 1521a6d262b7451bbef8debe970dd6b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130). New cstate: current_term: 1 leader_uuid: "1521a6d262b7451bbef8debe970dd6b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } }
I20260202 02:52:23.588296 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 2f664f6119ba4d28b5ef1c212e611866 (DEFAULT_TABLE table=default.Table_cc8d0231630743f180159c4d3a4efab8 [id=0dc5dc53fc014dd89bc66ac6260de9e7]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:23.589727 999 leader_election.cc:343] T eb0ac55fbb534b29a86d6b3e8aa72ebc P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: eb0ac55fbb534b29a86d6b3e8aa72ebc
W20260202 02:52:23.589988 999 leader_election.cc:343] T 75f7824681344d8d88df9402694fff1b P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 75f7824681344d8d88df9402694fff1b
W20260202 02:52:23.590171 999 leader_election.cc:343] T 929eb48881b141718873d0e359e768a8 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 929eb48881b141718873d0e359e768a8
I20260202 02:52:23.590350 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f664f6119ba4d28b5ef1c212e611866. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.596350 1174 tablet_bootstrap.cc:654] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:23.595844 742 tablet_service.cc:1552] Processing DeleteTablet for tablet 07580ad1d049475a9645acfad0b1d219 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:23.595697 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 275) took 3839 ms. Trace:
I20260202 02:52:23.597569 1877 leader_election.cc:290] T 757492288d7c44fb91b25b7e6e21e01b P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:23.598012 1721 raft_consensus.cc:2804] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:23.598337 1721 raft_consensus.cc:697] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:23.598927 1721 consensus_queue.cc:237] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.597918 1180 rpcz_store.cc:276] 0202 02:52:19.741450 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.741562 (+ 112us) service_pool.cc:225] Handling call
0202 02:52:23.556853 (+3815291us) ts_tablet_manager.cc:1151] Deleting tablet 59990791123c4d4eb620090a1e8e31e1
0202 02:52:23.556866 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:23.580663 (+ 23797us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3790038}
W20260202 02:52:23.600816 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 59990791123c4d4eb620090a1e8e31e1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 59990791123c4d4eb620090a1e8e31e1 already in progress: creating tablet
I20260202 02:52:23.601131 1180 tablet_replica.cc:333] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:23.604385 525 master_service.cc:954] ReplaceTablet: received request to replace tablet b4a4c75a906f40beb896bf6fff705ae6 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:23.605151 1175 tablet_bootstrap.cc:492] T 6154460838eb4e508f3646d3e3e4cf21 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:23.605361 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskYXxklX/build/asan/bin/kudu
/tmp/dist-test-taskYXxklX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.190:33981
--webserver_interface=127.0.104.190
--webserver_port=40497
--builtin_ntp_servers=127.0.104.148:45945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.190:33981,127.0.104.189:46271,127.0.104.188:39077
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260202 02:52:23.604741 1180 raft_consensus.cc:2243] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:23.585003 918 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f54c1dfe0a942b3bafd85522953a032" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:23.608480 918 raft_consensus.cc:3060] T 1f54c1dfe0a942b3bafd85522953a032 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:23.610142 522 catalog_manager.cc:5654] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f442ef85029461091de847118c34c0d (127.0.104.129). New cstate: current_term: 1 leader_uuid: "1f442ef85029461091de847118c34c0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } }
I20260202 02:52:23.610610 522 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet 78f122342dd144b48de89902faa0f94f (table default.Table_5eefa56618e84f8d92f8d87132ccd7d4 [id=b4ab2ba86e1a49338a98d855ded99f5d]). Expected version 1 got 0
I20260202 02:52:23.611202 918 raft_consensus.cc:2468] T 1f54c1dfe0a942b3bafd85522953a032 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:23.612038 527 master_service.cc:954] ReplaceTablet: received request to replace tablet 65bd1c1a382e42e4a68077264563c55b from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:23.612238 522 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet ef073880898049e2be93d2cb6dc0c0bb (table default.Table_e8e9b4cdaf434b6cbbfed1a65bff1fc0 [id=78a7fb3a2eed4565b55d5dacb8a3d7df]). Expected version 1 got 0
I20260202 02:52:23.614461 1030 tablet_service.cc:1552] Processing DeleteTablet for tablet 874ca4ba4ebc4721bb4b60cba8e01841 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.614910 1030 tablet_service.cc:1552] Processing DeleteTablet for tablet 938813ee0a8a40b6bf0e537e3ec1a0d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.614413 1180 pending_rounds.cc:70] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d: Trying to abort 1 pending ops.
I20260202 02:52:23.599418 921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "757492288d7c44fb91b25b7e6e21e01b" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:23.615563 921 raft_consensus.cc:2468] T 757492288d7c44fb91b25b7e6e21e01b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:23.605470 1865 raft_consensus.cc:493] T 59990791123c4d4eb620090a1e8e31e1 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:23.615420 1030 tablet_service.cc:1552] Processing DeleteTablet for tablet e95153ee8f2d40de955e3389d3a58644 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.615818 1865 raft_consensus.cc:515] T 59990791123c4d4eb620090a1e8e31e1 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.616313 1042 tablet_service.cc:1505] Processing CreateTablet for tablet bb1005103c25423783736ace1866c328 (DEFAULT_TABLE table=default.Table_ed899eb11436405fa74f3b38d08552f7 [id=15a0348866d34a209fb8332e17c91ae0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.616542 1865 leader_election.cc:290] T 59990791123c4d4eb620090a1e8e31e1 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:23.616808 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb1005103c25423783736ace1866c328. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.618492 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "59990791123c4d4eb620090a1e8e31e1" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:23.619153 857 leader_election.cc:343] T 59990791123c4d4eb620090a1e8e31e1 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:23.621886 1173 tablet_bootstrap.cc:492] T d21fca9a5b234c26879717f51c8ea0f0 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:23.622061 1173 ts_tablet_manager.cc:1403] T d21fca9a5b234c26879717f51c8ea0f0 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.472s user 0.006s sys 0.006s
I20260202 02:52:23.622983 1173 raft_consensus.cc:359] T d21fca9a5b234c26879717f51c8ea0f0 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.623184 1173 raft_consensus.cc:385] T d21fca9a5b234c26879717f51c8ea0f0 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:23.623277 1173 raft_consensus.cc:740] T d21fca9a5b234c26879717f51c8ea0f0 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:23.623497 1173 consensus_queue.cc:260] T d21fca9a5b234c26879717f51c8ea0f0 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.624050 1173 ts_tablet_manager.cc:1434] T d21fca9a5b234c26879717f51c8ea0f0 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:23.624200 1180 pending_rounds.cc:77] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7249923038367436800 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "c12ff20ca6b44ddba0c936a28cfb2eb9" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 2 new_table_name: "default.Table_959d3e3da2814b3a986d29864e47246c" dest_uuid: "1f442ef85029461091de847118c34c0d" new_extra_config { } }
I20260202 02:52:23.624558 1173 tablet_bootstrap.cc:492] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
W20260202 02:52:23.624594 1180 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42950: Aborted: Op aborted
I20260202 02:52:23.624830 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:42950 (request call id 305) took 3138 ms. Trace:
I20260202 02:52:23.624925 1180 rpcz_store.cc:276] 0202 02:52:20.486003 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:20.486068 (+ 65us) service_pool.cc:225] Handling call
0202 02:52:23.624773 (+3138705us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:20.486495 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0202 02:52:20.486589 (+ 94us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0202 02:52:21.788912 (+1302323us) alter_schema_op.cc:66] Acquired schema lock
0202 02:52:21.788925 (+ 13us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0202 02:52:21.788980 (+ 55us) alter_schema_op.cc:128] START. Timestamp: 1770000741788925
0202 02:52:21.788984 (+ 4us) op_driver.cc:348] REPLICATION: starting
0202 02:52:21.806176 (+ 17192us) log.cc:844] Serialized 173 byte log entry
0202 02:52:21.807080 (+ 904us) spinlock_profiling.cc:243] Waited 17 ms on lock 0x616000175c54. stack: 00007f1e37b5aa86 00007f1e37b5a840 00007f1e36b353ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f1e43a1d922 00007f1e44e00875 00007f1e44dff399 00007f1e44e077c0 00007f1e44e075cd 0000000000a8970c 00007f1e37ba0daf 00007f1e37ba40cb 00007f1e37ba3f48 0000000000a8970c
0202 02:52:23.624513 (+1817433us) op_driver.cc:464] REPLICATION: finished
0202 02:52:23.624539 (+ 26us) op_driver.cc:389] HandleFailure(Aborted: Op aborted)
0202 02:52:23.624545 (+ 6us) alter_schema_op.cc:171] AlterSchemaCommitCallback: op aborted
Metrics: {"child_traces":[["op",{"prepare.queue_time_us":65,"prepare.run_cpu_time_us":49581,"prepare.run_wall_time_us":1320649,"raft.queue_time_us":97,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":5,"replication_time_us":1835476,"spinlock_wait_cycles":39083392}]]}
I20260202 02:52:23.625483 1180 raft_consensus.cc:2272] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:23.625680 999 leader_election.cc:304] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:23.626045 999 leader_election.cc:304] T 757492288d7c44fb91b25b7e6e21e01b P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:23.626091 1180 ts_tablet_manager.cc:1916] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:23.626346 1173 tablet_bootstrap.cc:654] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:23.628208 1764 raft_consensus.cc:2804] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:23.628506 1764 raft_consensus.cc:697] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:23.628880 1764 consensus_queue.cc:237] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.629220 1877 raft_consensus.cc:2804] T 757492288d7c44fb91b25b7e6e21e01b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:23.630055 1877 raft_consensus.cc:493] T 757492288d7c44fb91b25b7e6e21e01b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:23.630246 1877 raft_consensus.cc:3060] T 757492288d7c44fb91b25b7e6e21e01b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:23.633316 1877 raft_consensus.cc:515] T 757492288d7c44fb91b25b7e6e21e01b P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.636782 1175 tablet_bootstrap.cc:654] T 6154460838eb4e508f3646d3e3e4cf21 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:23.646347 520 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_a0f1f2d30d064244bec97181c11c3a06" } modify_external_catalogs: true
I20260202 02:52:23.646592 520 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_a0f1f2d30d064244bec97181c11c3a06" } modify_external_catalogs: true
I20260202 02:52:23.646867 526 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_bcf2aa761e104360bb4a1251204a6608" } modify_external_catalogs: true
I20260202 02:52:23.647065 526 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_bcf2aa761e104360bb4a1251204a6608" } modify_external_catalogs: true
I20260202 02:52:23.647121 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_e45bc97d737440b5b6c3ad7bb03d2583"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:23.647742 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e45bc97d737440b5b6c3ad7bb03d2583 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:23.648950 524 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_783c68f0f6ad41899a919edefc2e85df" } new_table_name: "default.Table_c08aceb74c164c06a3a149c454c9c59e" modify_external_catalogs: true
I20260202 02:52:23.650995 1180 ts_tablet_manager.cc:1929] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260202 02:52:23.651135 1180 log.cc:1199] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/c12ff20ca6b44ddba0c936a28cfb2eb9
I20260202 02:52:23.651535 1180 ts_tablet_manager.cc:1950] T c12ff20ca6b44ddba0c936a28cfb2eb9 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:23.655496 523 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 33150e1456144b8e9c64cc2a2ac6dcca
I20260202 02:52:23.655957 523 catalog_manager.cc:6724] ReplaceTablet: tablet 33150e1456144b8e9c64cc2a2ac6dcca deleted and replaced by tablet 1d98c7a46139413b8f0d3697b74b1e52
I20260202 02:52:23.657413 523 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_755b6ce0796b45b8be9df04e59dfe565" } modify_external_catalogs: true
I20260202 02:52:23.657596 523 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_755b6ce0796b45b8be9df04e59dfe565" } modify_external_catalogs: true
I20260202 02:52:23.660706 742 tablet_service.cc:1552] Processing DeleteTablet for tablet 33150e1456144b8e9c64cc2a2ac6dcca with delete_type TABLET_DATA_DELETED (replaced by tablet 1d98c7a46139413b8f0d3697b74b1e52) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:23.664405 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 278) took 3916 ms. Trace:
I20260202 02:52:23.664559 1180 rpcz_store.cc:276] 0202 02:52:19.747988 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.748138 (+ 150us) service_pool.cc:225] Handling call
0202 02:52:23.600596 (+3852458us) ts_tablet_manager.cc:1151] Deleting tablet c12ff20ca6b44ddba0c936a28cfb2eb9
0202 02:52:23.600610 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:23.647841 (+ 47231us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:23.650828 (+ 2987us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:23.664349 (+ 13521us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3820492}
I20260202 02:52:23.664666 519 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet bd7b0a0fcb7645fdb9edfeccb3d18268
I20260202 02:52:23.665169 519 catalog_manager.cc:6724] ReplaceTablet: tablet bd7b0a0fcb7645fdb9edfeccb3d18268 deleted and replaced by tablet 5ffdf97dff3e4dca89507f488c5b0f8e
I20260202 02:52:23.666002 525 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet b4a4c75a906f40beb896bf6fff705ae6
I20260202 02:52:23.666472 525 catalog_manager.cc:6724] ReplaceTablet: tablet b4a4c75a906f40beb896bf6fff705ae6 deleted and replaced by tablet af29c7a2ad9d4797b08052606151a7d5
I20260202 02:52:23.667692 1180 tablet_replica.cc:333] T 61a9203035f2495f8f99924303ebf120 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:23.668375 758 tablet_service.cc:1552] Processing DeleteTablet for tablet b4a4c75a906f40beb896bf6fff705ae6 with delete_type TABLET_DATA_DELETED (replaced by tablet af29c7a2ad9d4797b08052606151a7d5) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:23.668385 742 tablet_service.cc:1552] Processing DeleteTablet for tablet bd7b0a0fcb7645fdb9edfeccb3d18268 with delete_type TABLET_DATA_DELETED (replaced by tablet 5ffdf97dff3e4dca89507f488c5b0f8e) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:23.667937 1180 raft_consensus.cc:2243] T 61a9203035f2495f8f99924303ebf120 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:23.668880 1180 raft_consensus.cc:2272] T 61a9203035f2495f8f99924303ebf120 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:23.672892 1180 ts_tablet_manager.cc:1916] T 61a9203035f2495f8f99924303ebf120 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:23.685716 1180 ts_tablet_manager.cc:1929] T 61a9203035f2495f8f99924303ebf120 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:23.686003 1180 log.cc:1199] T 61a9203035f2495f8f99924303ebf120 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/61a9203035f2495f8f99924303ebf120
I20260202 02:52:23.686575 1180 ts_tablet_manager.cc:1950] T 61a9203035f2495f8f99924303ebf120 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:23.689220 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 279) took 3941 ms. Trace:
I20260202 02:52:23.689415 1180 rpcz_store.cc:276] 0202 02:52:19.748105 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.748191 (+ 86us) service_pool.cc:225] Handling call
0202 02:52:23.666611 (+3918420us) ts_tablet_manager.cc:1151] Deleting tablet 61a9203035f2495f8f99924303ebf120
0202 02:52:23.667621 (+ 1010us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:23.677436 (+ 9815us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:23.685571 (+ 8135us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:23.689161 (+ 3590us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3870555}
I20260202 02:52:23.690236 1180 tablet_replica.cc:333] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:23.690495 1180 raft_consensus.cc:2243] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:23.691941 1180 raft_consensus.cc:2272] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:23.692596 1180 ts_tablet_manager.cc:1916] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:23.698441 1850 raft_consensus.cc:493] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:23.698643 1850 raft_consensus.cc:515] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.699303 1850 leader_election.cc:290] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:23.703181 1174 tablet_bootstrap.cc:492] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:23.703367 1174 ts_tablet_manager.cc:1403] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.137s user 0.005s sys 0.003s
I20260202 02:52:23.704206 1174 raft_consensus.cc:359] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.704430 1174 raft_consensus.cc:385] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:23.704530 1174 raft_consensus.cc:740] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:23.704773 1174 consensus_queue.cc:260] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.709183 1180 ts_tablet_manager.cc:1929] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:23.709367 1180 log.cc:1199] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/bbf31e5619ed4f60b4d7c8b19ec8a549
I20260202 02:52:23.709759 1180 ts_tablet_manager.cc:1950] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:23.656785 893 tablet_service.cc:1552] Processing DeleteTablet for tablet 33150e1456144b8e9c64cc2a2ac6dcca with delete_type TABLET_DATA_DELETED (replaced by tablet 1d98c7a46139413b8f0d3697b74b1e52) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:23.666013 902 tablet_service.cc:1552] Processing DeleteTablet for tablet bd7b0a0fcb7645fdb9edfeccb3d18268 with delete_type TABLET_DATA_DELETED (replaced by tablet 5ffdf97dff3e4dca89507f488c5b0f8e) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:23.710381 1182 tablet_replica.cc:333] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:23.710629 1182 raft_consensus.cc:2243] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:23.710899 1182 raft_consensus.cc:2272] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:23.711521 1182 ts_tablet_manager.cc:1916] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:23.667517 889 tablet_service.cc:1552] Processing DeleteTablet for tablet b4a4c75a906f40beb896bf6fff705ae6 with delete_type TABLET_DATA_DELETED (replaced by tablet af29c7a2ad9d4797b08052606151a7d5) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:23.700194 921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "757492288d7c44fb91b25b7e6e21e01b" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:23.714401 921 raft_consensus.cc:2468] T 757492288d7c44fb91b25b7e6e21e01b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
W20260202 02:52:23.714867 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 33150e1456144b8e9c64cc2a2ac6dcca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 33150e1456144b8e9c64cc2a2ac6dcca already in progress: creating tablet
I20260202 02:52:23.714978 714 leader_election.cc:304] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters:
I20260202 02:52:23.715562 1850 raft_consensus.cc:2804] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:23.715631 1174 ts_tablet_manager.cc:1434] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.012s user 0.003s sys 0.000s
I20260202 02:52:23.715670 1850 raft_consensus.cc:493] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:23.715776 1850 raft_consensus.cc:3060] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:23.718408 1850 raft_consensus.cc:515] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.719662 921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "757492288d7c44fb91b25b7e6e21e01b" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:23.719908 921 raft_consensus.cc:3060] T 757492288d7c44fb91b25b7e6e21e01b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:23.722591 921 raft_consensus.cc:2468] T 757492288d7c44fb91b25b7e6e21e01b P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:23.723133 1174 tablet_bootstrap.cc:492] T d1e0e74f32a24729b250c5751677cfb0 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:23.723552 1865 raft_consensus.cc:493] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:23.723636 714 leader_election.cc:304] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters:
I20260202 02:52:23.723731 1865 raft_consensus.cc:515] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.724097 1723 raft_consensus.cc:2804] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:23.724246 1723 raft_consensus.cc:697] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:23.724391 1865 leader_election.cc:290] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:23.724551 1723 consensus_queue.cc:237] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.725334 1175 tablet_bootstrap.cc:492] T 6154460838eb4e508f3646d3e3e4cf21 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:23.725571 1175 ts_tablet_manager.cc:1403] T 6154460838eb4e508f3646d3e3e4cf21 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.121s user 0.003s sys 0.001s
I20260202 02:52:23.725871 1850 leader_election.cc:290] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:23.727035 1175 raft_consensus.cc:359] T 6154460838eb4e508f3646d3e3e4cf21 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.726122 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "25765fb687294e9db1289f76530dd877" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:23.735083 1850 raft_consensus.cc:493] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:23.735342 1850 raft_consensus.cc:515] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.730602 1182 ts_tablet_manager.cc:1929] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
W20260202 02:52:23.731458 857 leader_election.cc:343] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:23.736181 1182 log.cc:1199] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/bd7b0a0fcb7645fdb9edfeccb3d18268
I20260202 02:52:23.736617 1182 ts_tablet_manager.cc:1950] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
W20260202 02:52:23.737550 714 leader_election.cc:343] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: eb0ac55fbb534b29a86d6b3e8aa72ebc
I20260202 02:52:23.737783 1850 leader_election.cc:290] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:23.738227 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet bd7b0a0fcb7645fdb9edfeccb3d18268 (table default.Table_18fec323d6b04e10a687de6361b643b2 [id=1ffcc11561524fe8a7b09a7ddc92d5b2]) successfully deleted
W20260202 02:52:23.738866 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet b4a4c75a906f40beb896bf6fff705ae6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b4a4c75a906f40beb896bf6fff705ae6 already in progress: creating tablet
I20260202 02:52:23.739058 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 272) took 4012 ms. Trace:
I20260202 02:52:23.739132 1175 raft_consensus.cc:385] T 6154460838eb4e508f3646d3e3e4cf21 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:23.739245 1175 raft_consensus.cc:740] T 6154460838eb4e508f3646d3e3e4cf21 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:23.739389 1180 rpcz_store.cc:276] 0202 02:52:19.726191 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.728433 (+ 2242us) service_pool.cc:225] Handling call
0202 02:52:23.690144 (+3961711us) ts_tablet_manager.cc:1151] Deleting tablet bbf31e5619ed4f60b4d7c8b19ec8a549
0202 02:52:23.690155 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:23.695799 (+ 5644us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:23.708989 (+ 13190us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:23.738608 (+ 29619us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3893983}
I20260202 02:52:23.739506 1175 consensus_queue.cc:260] T 6154460838eb4e508f3646d3e3e4cf21 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.741520 1175 ts_tablet_manager.cc:1434] T 6154460838eb4e508f3646d3e3e4cf21 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.016s user 0.001s sys 0.001s
I20260202 02:52:23.741889 1175 tablet_bootstrap.cc:492] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:23.742128 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet bbf31e5619ed4f60b4d7c8b19ec8a549 (table default.Table_ef11a1ba065d483c9f7862345ab1aedb [id=0e54d0abe11c46fa8dbf0befb46219cb]) successfully deleted
I20260202 02:52:23.742600 1180 tablet_replica.cc:333] T d2fa4c3d6ec44047abf5986e502c1590 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:23.740753 1174 tablet_bootstrap.cc:654] T d1e0e74f32a24729b250c5751677cfb0 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:23.750712 1180 raft_consensus.cc:2243] T d2fa4c3d6ec44047abf5986e502c1590 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:23.751099 1180 raft_consensus.cc:2272] T d2fa4c3d6ec44047abf5986e502c1590 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:23.751819 1180 ts_tablet_manager.cc:1916] T d2fa4c3d6ec44047abf5986e502c1590 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:23.754055 1175 tablet_bootstrap.cc:654] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:23.767815 1175 tablet_bootstrap.cc:492] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:23.768131 1175 ts_tablet_manager.cc:1403] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.026s user 0.005s sys 0.001s
I20260202 02:52:23.769054 1175 raft_consensus.cc:359] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:23.769402 1175 raft_consensus.cc:385] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:23.769519 1175 raft_consensus.cc:740] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:23.771420 1175 consensus_queue.cc:260] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:23.773315 1175 ts_tablet_manager.cc:1434] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260202 02:52:23.773705 1175 tablet_bootstrap.cc:492] T 79f694b740954d429145fba9157338ca P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:23.775493 1175 tablet_bootstrap.cc:654] T 79f694b740954d429145fba9157338ca P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:23.777496 715 leader_election.cc:304] T 75b97d7e848241cf8d894b0ae92a8f8f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:23.778116 1850 raft_consensus.cc:3060] T 75b97d7e848241cf8d894b0ae92a8f8f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:23.781096 1850 raft_consensus.cc:2749] T 75b97d7e848241cf8d894b0ae92a8f8f P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:23.785717 715 leader_election.cc:304] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
W20260202 02:52:23.786005 715 raft_consensus.cc:2686] T bbf31e5619ed4f60b4d7c8b19ec8a549 P 1f442ef85029461091de847118c34c0d: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260202 02:52:23.656013 1173 tablet_bootstrap.cc:492] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:23.666709 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet bd7b0a0fcb7645fdb9edfeccb3d18268 with delete_type TABLET_DATA_DELETED (replaced by tablet 5ffdf97dff3e4dca89507f488c5b0f8e) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.667062 1037 tablet_service.cc:1552] Processing DeleteTablet for tablet 33150e1456144b8e9c64cc2a2ac6dcca with delete_type TABLET_DATA_DELETED (replaced by tablet 1d98c7a46139413b8f0d3697b74b1e52) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.667220 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet b4a4c75a906f40beb896bf6fff705ae6 with delete_type TABLET_DATA_DELETED (replaced by tablet af29c7a2ad9d4797b08052606151a7d5) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.746389 1882 ts_tablet_manager.cc:1929] T fe7b6004e5094c0dbe5847144a2489b9 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:23.803457 1173 ts_tablet_manager.cc:1403] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.179s user 0.004s sys 0.000s
I20260202 02:52:23.803552 1882 log.cc:1199] T fe7b6004e5094c0dbe5847144a2489b9 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/fe7b6004e5094c0dbe5847144a2489b9
I20260202 02:52:23.804021 1882 ts_tablet_manager.cc:1950] T fe7b6004e5094c0dbe5847144a2489b9 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:23.753680 1894 raft_consensus.cc:493] T d67e91a022f1401ea724dff898c3b81b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:23.759420 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "757492288d7c44fb91b25b7e6e21e01b" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:23.804746 1063 raft_consensus.cc:2393] T 757492288d7c44fb91b25b7e6e21e01b P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f442ef85029461091de847118c34c0d in current term 1: Already voted for candidate b1f05820b6034228bdad884cc348716d in this term.
I20260202 02:52:23.759812 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "757492288d7c44fb91b25b7e6e21e01b" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:23.759919 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "eb0ac55fbb534b29a86d6b3e8aa72ebc" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:23.805073 1064 raft_consensus.cc:2468] T eb0ac55fbb534b29a86d6b3e8aa72ebc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:23.805305 1894 raft_consensus.cc:515] T d67e91a022f1401ea724dff898c3b81b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:23.805943 1894 leader_election.cc:290] T d67e91a022f1401ea724dff898c3b81b P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:23.786284 1058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "59990791123c4d4eb620090a1e8e31e1" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:23.786634 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "25765fb687294e9db1289f76530dd877" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:23.787973 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e1284fde67bc4d65b067a54c416ccc33" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:23.790289 999 consensus_peers.cc:597] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0186572e7bb9499a932a636ab280f544. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:23.804576 1173 raft_consensus.cc:359] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:23.761114 998 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 318506us
I20260202 02:52:23.804571 1882 tablet_replica.cc:333] T c12ff20ca6b44ddba0c936a28cfb2eb9 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:23.807212 918 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "757492288d7c44fb91b25b7e6e21e01b" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:23.807484 918 raft_consensus.cc:2393] T 757492288d7c44fb91b25b7e6e21e01b P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b1f05820b6034228bdad884cc348716d in current term 1: Already voted for candidate 1f442ef85029461091de847118c34c0d in this term.
I20260202 02:52:23.807839 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "94bada52db6a47179415d302b541d01f" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:23.812532 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet fe7b6004e5094c0dbe5847144a2489b9 (table default.Table_a6b938c21e164294bd8b5fb205673f75 [id=1abad082d47d4f31be7c697ed72e241c]) successfully deleted
I20260202 02:52:23.814280 527 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 65bd1c1a382e42e4a68077264563c55b
I20260202 02:52:23.814806 527 catalog_manager.cc:6724] ReplaceTablet: tablet 65bd1c1a382e42e4a68077264563c55b deleted and replaced by tablet 3706562a22634eb6b72e36bb6a2d1ae5
I20260202 02:52:23.814793 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f54c1dfe0a942b3bafd85522953a032" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:23.815567 742 tablet_service.cc:1552] Processing DeleteTablet for tablet 65bd1c1a382e42e4a68077264563c55b with delete_type TABLET_DATA_DELETED (replaced by tablet 3706562a22634eb6b72e36bb6a2d1ae5) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:23.815876 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "eb0ac55fbb534b29a86d6b3e8aa72ebc" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:23.816089 775 raft_consensus.cc:2468] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:23.816164 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet 65bd1c1a382e42e4a68077264563c55b with delete_type TABLET_DATA_DELETED (replaced by tablet 3706562a22634eb6b72e36bb6a2d1ae5) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.816272 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 65bd1c1a382e42e4a68077264563c55b with delete_type TABLET_DATA_DELETED (replaced by tablet 3706562a22634eb6b72e36bb6a2d1ae5) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:23.816642 917 raft_consensus.cc:1275] T 70306ed9dec54286885d965f2c012da4 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Refusing update from remote peer 1f442ef85029461091de847118c34c0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260202 02:52:23.817075 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 65bd1c1a382e42e4a68077264563c55b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 65bd1c1a382e42e4a68077264563c55b already in progress: creating tablet
I20260202 02:52:23.817103 1850 consensus_queue.cc:1048] T 70306ed9dec54286885d965f2c012da4 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:23.820233 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_520d85c11ce24a88a0a5f162d14e8a61"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:23.820935 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_520d85c11ce24a88a0a5f162d14e8a61 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:23.821028 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_e11021f60bfe480ab07b2242bb36e94f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:23.821563 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e11021f60bfe480ab07b2242bb36e94f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:23.822253 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "929eb48881b141718873d0e359e768a8" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:23.824736 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "757492288d7c44fb91b25b7e6e21e01b" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:23.826239 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "757492288d7c44fb91b25b7e6e21e01b" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:23.826592 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d67e91a022f1401ea724dff898c3b81b" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:23.829600 519 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_bde65e061a274734a7df7a428bd8ce53" } modify_external_catalogs: true
I20260202 02:52:23.829833 519 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_bde65e061a274734a7df7a428bd8ce53" } modify_external_catalogs: true
I20260202 02:52:23.830890 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 59990791123c4d4eb620090a1e8e31e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:23.831236 1182 tablet_replica.cc:333] T 59990791123c4d4eb620090a1e8e31e1 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:23.831368 889 tablet_service.cc:1552] Processing DeleteTablet for tablet d0671586e45e45ffb7fd9e1be6e2bbf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:23.831456 1182 raft_consensus.cc:2243] T 59990791123c4d4eb620090a1e8e31e1 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:23.831643 1182 raft_consensus.cc:2272] T 59990791123c4d4eb620090a1e8e31e1 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:23.832253 1182 ts_tablet_manager.cc:1916] T 59990791123c4d4eb620090a1e8e31e1 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:23.834946 715 leader_election.cc:304] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:23.835321 1877 leader_election.cc:290] T 757492288d7c44fb91b25b7e6e21e01b P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:23.835534 1850 raft_consensus.cc:2804] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:23.835685 1850 raft_consensus.cc:493] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:23.835800 1173 raft_consensus.cc:385] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260202 02:52:23.835865 999 leader_election.cc:343] T d67e91a022f1401ea724dff898c3b81b P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: d67e91a022f1401ea724dff898c3b81b
I20260202 02:52:23.835935 1173 raft_consensus.cc:740] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:23.836201 1173 consensus_queue.cc:260] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.836437 1882 raft_consensus.cc:2243] T c12ff20ca6b44ddba0c936a28cfb2eb9 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
W20260202 02:52:23.836467 998 leader_election.cc:343] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:23.836594 998 leader_election.cc:304] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:23.836647 1882 pending_rounds.cc:70] T c12ff20ca6b44ddba0c936a28cfb2eb9 P b1f05820b6034228bdad884cc348716d: Trying to abort 1 pending ops.
I20260202 02:52:23.836916 1173 ts_tablet_manager.cc:1434] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.033s user 0.002s sys 0.000s
I20260202 02:52:23.836716 1882 pending_rounds.cc:77] T c12ff20ca6b44ddba0c936a28cfb2eb9 P b1f05820b6034228bdad884cc348716d: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7249923038367436800 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "c12ff20ca6b44ddba0c936a28cfb2eb9" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 2 new_table_name: "default.Table_959d3e3da2814b3a986d29864e47246c" dest_uuid: "1f442ef85029461091de847118c34c0d" new_extra_config { } }
I20260202 02:52:23.837127 1860 raft_consensus.cc:2749] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260202 02:52:23.837271 1173 tablet_bootstrap.cc:492] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:23.837369 1882 raft_consensus.cc:2272] T c12ff20ca6b44ddba0c936a28cfb2eb9 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:23.837411 525 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_5fa327d665484811934aa3b11188a76d" } new_table_name: "default.Table_718e4de3a182443fb16b265396c79401" modify_external_catalogs: true
W20260202 02:52:23.837638 998 leader_election.cc:343] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:23.837940 1059 raft_consensus.cc:2468] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
I20260202 02:52:23.837982 998 leader_election.cc:304] T eb0ac55fbb534b29a86d6b3e8aa72ebc P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:23.838250 1860 raft_consensus.cc:2804] T eb0ac55fbb534b29a86d6b3e8aa72ebc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:23.838361 1860 raft_consensus.cc:493] T eb0ac55fbb534b29a86d6b3e8aa72ebc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:23.838449 1860 raft_consensus.cc:3060] T eb0ac55fbb534b29a86d6b3e8aa72ebc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:23.838671 859 leader_election.cc:343] T 59990791123c4d4eb620090a1e8e31e1 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:23.838876 859 leader_election.cc:304] T 59990791123c4d4eb620090a1e8e31e1 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
W20260202 02:52:23.839092 859 raft_consensus.cc:2686] T 59990791123c4d4eb620090a1e8e31e1 P 1521a6d262b7451bbef8debe970dd6b5: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260202 02:52:23.839344 1882 ts_tablet_manager.cc:1916] T c12ff20ca6b44ddba0c936a28cfb2eb9 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:23.839454 859 leader_election.cc:343] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:23.839565 859 leader_election.cc:304] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:23.840258 1865 raft_consensus.cc:2749] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:23.841081 1173 tablet_bootstrap.cc:654] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:23.841392 1860 raft_consensus.cc:515] T eb0ac55fbb534b29a86d6b3e8aa72ebc P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.842319 1860 leader_election.cc:290] T eb0ac55fbb534b29a86d6b3e8aa72ebc P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:23.842752 998 leader_election.cc:343] T 75f7824681344d8d88df9402694fff1b P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Not found: Tablet not found: 75f7824681344d8d88df9402694fff1b
I20260202 02:52:23.842898 998 leader_election.cc:304] T 75f7824681344d8d88df9402694fff1b P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
W20260202 02:52:23.843354 998 leader_election.cc:343] T 929eb48881b141718873d0e359e768a8 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:23.843498 998 leader_election.cc:304] T 929eb48881b141718873d0e359e768a8 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:23.843775 1764 raft_consensus.cc:2749] T 75f7824681344d8d88df9402694fff1b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:23.843959 1764 raft_consensus.cc:2749] T 929eb48881b141718873d0e359e768a8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:23.844558 514 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 550895b4010f4088b2cc34acff50e03f (table default.Table_6a546caebf9e4075816236ca1bd8d73f [id=427f58cc70ef47a6afac8de0353975f5], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 550895b4010f4088b2cc34acff50e03f consensus state has no leader
W20260202 02:52:23.844936 998 consensus_peers.cc:597] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c184d44d7f434b39a997a7387e5d3de0. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:23.845405 998 consensus_peers.cc:597] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:23.845489 1182 ts_tablet_manager.cc:1929] T 59990791123c4d4eb620090a1e8e31e1 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:23.845635 1182 log.cc:1199] T 59990791123c4d4eb620090a1e8e31e1 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/59990791123c4d4eb620090a1e8e31e1
I20260202 02:52:23.845610 998 leader_election.cc:304] T 757492288d7c44fb91b25b7e6e21e01b P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:23.845885 1860 raft_consensus.cc:2749] T 757492288d7c44fb91b25b7e6e21e01b P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260202 02:52:23.846019 1182 ts_tablet_manager.cc:1950] T 59990791123c4d4eb620090a1e8e31e1 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
W20260202 02:52:23.846334 998 leader_election.cc:343] T d67e91a022f1401ea724dff898c3b81b P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:23.846501 1182 tablet_replica.cc:333] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:23.846650 998 leader_election.cc:304] T d67e91a022f1401ea724dff898c3b81b P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:23.846712 1182 raft_consensus.cc:2243] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:23.846903 1182 raft_consensus.cc:2272] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:23.847306 1882 ts_tablet_manager.cc:1929] T c12ff20ca6b44ddba0c936a28cfb2eb9 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260202 02:52:23.847661 1850 raft_consensus.cc:3060] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:23.847968 1175 tablet_bootstrap.cc:492] T 79f694b740954d429145fba9157338ca P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:23.848129 1175 ts_tablet_manager.cc:1403] T 79f694b740954d429145fba9157338ca P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.075s user 0.000s sys 0.005s
I20260202 02:52:23.848275 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 59990791123c4d4eb620090a1e8e31e1 (table default.Table_26642341d3d54c0fa9af19fef3a5b06d [id=9ff2eb6ee89943918019f5e899ac362b]) successfully deleted
I20260202 02:52:23.848580 1882 log.cc:1199] T c12ff20ca6b44ddba0c936a28cfb2eb9 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/c12ff20ca6b44ddba0c936a28cfb2eb9
I20260202 02:52:23.848984 1882 ts_tablet_manager.cc:1950] T c12ff20ca6b44ddba0c936a28cfb2eb9 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:23.849014 1175 raft_consensus.cc:359] T 79f694b740954d429145fba9157338ca P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.849200 1175 raft_consensus.cc:385] T 79f694b740954d429145fba9157338ca P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:23.849292 1175 raft_consensus.cc:740] T 79f694b740954d429145fba9157338ca P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:23.849409 1860 raft_consensus.cc:2749] T d67e91a022f1401ea724dff898c3b81b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:23.849496 1175 consensus_queue.cc:260] T 79f694b740954d429145fba9157338ca P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.849591 1860 consensus_queue.cc:1048] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:23.850059 1175 ts_tablet_manager.cc:1434] T 79f694b740954d429145fba9157338ca P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260202 02:52:23.850332 1175 tablet_bootstrap.cc:492] T 042d1ebf357b4abe8814a407891ce265 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:23.850566 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "eb0ac55fbb534b29a86d6b3e8aa72ebc" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:23.852236 1175 tablet_bootstrap.cc:654] T 042d1ebf357b4abe8814a407891ce265 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:23.852416 1850 raft_consensus.cc:515] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.852830 774 raft_consensus.cc:2393] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b1f05820b6034228bdad884cc348716d in current term 1: Already voted for candidate 1f442ef85029461091de847118c34c0d in this term.
I20260202 02:52:23.853484 1882 tablet_replica.cc:333] T f58a7b25815c4b5bac568aa11c19610c P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:23.853704 1882 raft_consensus.cc:2243] T f58a7b25815c4b5bac568aa11c19610c P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:23.853927 1882 raft_consensus.cc:2272] T f58a7b25815c4b5bac568aa11c19610c P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:23.854512 1882 ts_tablet_manager.cc:1916] T f58a7b25815c4b5bac568aa11c19610c P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:23.854528 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet c12ff20ca6b44ddba0c936a28cfb2eb9 (table default.Table_959d3e3da2814b3a986d29864e47246c [id=bdf2f19a439d4be3aa2362cfde38ab7f]) successfully deleted
W20260202 02:52:23.854940 999 consensus_peers.cc:597] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c184d44d7f434b39a997a7387e5d3de0. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:23.855170 999 leader_election.cc:343] T eb0ac55fbb534b29a86d6b3e8aa72ebc P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: eb0ac55fbb534b29a86d6b3e8aa72ebc
I20260202 02:52:23.855808 526 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 7ff6d2fa085343a6b11be0898da6233d
I20260202 02:52:23.857234 742 tablet_service.cc:1552] Processing DeleteTablet for tablet 7ff6d2fa085343a6b11be0898da6233d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:23.857543 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet 7ff6d2fa085343a6b11be0898da6233d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.858104 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 7ff6d2fa085343a6b11be0898da6233d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:23.858402 1182 ts_tablet_manager.cc:1916] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:23.858637 1860 consensus_queue.cc:1048] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260202 02:52:23.861106 714 leader_election.cc:343] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: eb0ac55fbb534b29a86d6b3e8aa72ebc
I20260202 02:52:23.861351 1850 leader_election.cc:290] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:23.861811 520 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet b7880cb051c24bc998003f0effed3cf8
I20260202 02:52:23.862799 742 tablet_service.cc:1552] Processing DeleteTablet for tablet b7880cb051c24bc998003f0effed3cf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:42950
W20260202 02:52:23.862862 524 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7f323175ee254df789bcb04e65047dff consensus state has no leader
I20260202 02:52:23.863878 889 tablet_service.cc:1552] Processing DeleteTablet for tablet b7880cb051c24bc998003f0effed3cf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:23.864271 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet b7880cb051c24bc998003f0effed3cf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.865029 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_a5d6ab55014a4c068dceeb6a781bf3b1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:23.865664 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a5d6ab55014a4c068dceeb6a781bf3b1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:23.866153 520 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_52fba283eb604f81acbbc5ea3762d1f9" } new_table_name: "default.Table_f034976ca8c24ddd84eba68e745b6821" modify_external_catalogs: true
I20260202 02:52:23.867274 524 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_89983dadd3fc4966bf26418544c1729a" } new_table_name: "default.Table_1fd1e6708a1a4421bd5d679a3b6539e9" modify_external_catalogs: true
I20260202 02:52:23.870442 523 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 5986676f5484498da927b4c6da3f47ff
I20260202 02:52:23.870586 1174 tablet_bootstrap.cc:492] T d1e0e74f32a24729b250c5751677cfb0 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:23.870755 1174 ts_tablet_manager.cc:1403] T d1e0e74f32a24729b250c5751677cfb0 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.148s user 0.001s sys 0.004s
I20260202 02:52:23.871371 742 tablet_service.cc:1552] Processing DeleteTablet for tablet 5986676f5484498da927b4c6da3f47ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:23.871634 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 5986676f5484498da927b4c6da3f47ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:23.872582 1174 raft_consensus.cc:359] T d1e0e74f32a24729b250c5751677cfb0 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.872816 1174 raft_consensus.cc:385] T d1e0e74f32a24729b250c5751677cfb0 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:23.872906 1174 raft_consensus.cc:740] T d1e0e74f32a24729b250c5751677cfb0 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:23.873162 1174 consensus_queue.cc:260] T d1e0e74f32a24729b250c5751677cfb0 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.873656 1174 ts_tablet_manager.cc:1434] T d1e0e74f32a24729b250c5751677cfb0 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260202 02:52:23.879417 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet 5986676f5484498da927b4c6da3f47ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.879603 519 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet d0ce2121d65b4a228596e0ca9018883d
I20260202 02:52:23.879666 1060 raft_consensus.cc:1275] T 70306ed9dec54286885d965f2c012da4 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Refusing update from remote peer 1f442ef85029461091de847118c34c0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260202 02:52:23.881582 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet d0ce2121d65b4a228596e0ca9018883d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:23.882100 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_caf008ae2a414bdbbfdec276ff3725ea"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:23.882752 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_caf008ae2a414bdbbfdec276ff3725ea in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:23.882781 889 tablet_service.cc:1552] Processing DeleteTablet for tablet d0ce2121d65b4a228596e0ca9018883d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:23.883323 742 tablet_service.cc:1552] Processing DeleteTablet for tablet d0ce2121d65b4a228596e0ca9018883d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:23.884739 859 leader_election.cc:304] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:23.885222 519 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_6b9de049c34449509645f14959ae1200" } new_table_name: "default.Table_14f3398803354519b92b5291be072517" modify_external_catalogs: true
I20260202 02:52:23.885334 1865 raft_consensus.cc:2804] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:23.885461 1865 raft_consensus.cc:493] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:23.885533 1865 raft_consensus.cc:3060] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:23.885895 1723 consensus_queue.cc:1048] T 70306ed9dec54286885d965f2c012da4 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:23.886574 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_f7a491432d7b42cf82732e39dcb574ce"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:23.887190 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f7a491432d7b42cf82732e39dcb574ce in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:23.888463 528 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_b88bd2e65dd140d5a1c208c9d4aa9430" } modify_external_catalogs: true
I20260202 02:52:23.888640 528 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_b88bd2e65dd140d5a1c208c9d4aa9430" } modify_external_catalogs: true
I20260202 02:52:23.888684 1865 raft_consensus.cc:515] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.889168 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_efacf68164014a2d8873859a3650a913"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:23.889643 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_efacf68164014a2d8873859a3650a913 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:23.889906 1865 leader_election.cc:290] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:23.890120 1882 ts_tablet_manager.cc:1929] T f58a7b25815c4b5bac568aa11c19610c P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:23.890254 1882 log.cc:1199] T f58a7b25815c4b5bac568aa11c19610c P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/f58a7b25815c4b5bac568aa11c19610c
W20260202 02:52:23.892952 520 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c25e329023c04f609d8a743b4f7a95a4 consensus state has no leader
I20260202 02:52:23.893888 1882 ts_tablet_manager.cc:1950] T f58a7b25815c4b5bac568aa11c19610c P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
W20260202 02:52:23.894081 524 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 604b3b863c9540d3b55381dbfe1afdc3 consensus state has no leader
I20260202 02:52:23.896723 1865 raft_consensus.cc:493] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:23.896888 1865 raft_consensus.cc:515] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:23.897548 1865 leader_election.cc:290] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:23.898869 1180 ts_tablet_manager.cc:1929] T d2fa4c3d6ec44047abf5986e502c1590 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:23.899051 1180 log.cc:1199] T d2fa4c3d6ec44047abf5986e502c1590 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/d2fa4c3d6ec44047abf5986e502c1590
I20260202 02:52:23.899011 998 leader_election.cc:304] T eb0ac55fbb534b29a86d6b3e8aa72ebc P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:23.899452 1180 ts_tablet_manager.cc:1950] T d2fa4c3d6ec44047abf5986e502c1590 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:23.900012 1174 tablet_bootstrap.cc:492] T d21fca9a5b234c26879717f51c8ea0f0 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:23.900893 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "20ea39bc5d3d46f5a4ba427277d6c6a3" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:23.901551 857 leader_election.cc:343] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:23.901851 1174 tablet_bootstrap.cc:654] T d21fca9a5b234c26879717f51c8ea0f0 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:23.903200 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "25765fb687294e9db1289f76530dd877" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
W20260202 02:52:23.903891 857 leader_election.cc:343] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:23.905104 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet f58a7b25815c4b5bac568aa11c19610c (table default.Table_e5f4ac82d1ef4b85999695997eb9df2a [id=c97e407e519d4a538860f66931f6aefd]) successfully deleted
I20260202 02:52:23.917027 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 280) took 4167 ms. Trace:
I20260202 02:52:23.917199 1180 rpcz_store.cc:276] 0202 02:52:19.749984 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.750338 (+ 354us) service_pool.cc:225] Handling call
0202 02:52:23.742518 (+3992180us) ts_tablet_manager.cc:1151] Deleting tablet d2fa4c3d6ec44047abf5986e502c1590
0202 02:52:23.742532 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:23.755067 (+ 12535us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:23.898736 (+143669us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:23.916948 (+ 18212us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3944849}
I20260202 02:52:23.917716 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet d2fa4c3d6ec44047abf5986e502c1590 (table default.Table_cf37f87385ee44cea1ff80cc14050ff2 [id=697b6fe62ee845d895af350881a03556]) successfully deleted
I20260202 02:52:23.918262 1180 tablet_replica.cc:333] T d1e0e74f32a24729b250c5751677cfb0 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:23.923924 1860 raft_consensus.cc:2749] T eb0ac55fbb534b29a86d6b3e8aa72ebc P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260202 02:52:23.924501 1180 raft_consensus.cc:2243] T d1e0e74f32a24729b250c5751677cfb0 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:23.924913 1882 tablet_replica.cc:333] T bbf31e5619ed4f60b4d7c8b19ec8a549 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:23.924914 1180 raft_consensus.cc:2272] T d1e0e74f32a24729b250c5751677cfb0 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:23.925319 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "eb0ac55fbb534b29a86d6b3e8aa72ebc" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:23.925566 1060 raft_consensus.cc:2393] T eb0ac55fbb534b29a86d6b3e8aa72ebc P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f442ef85029461091de847118c34c0d in current term 1: Already voted for candidate b1f05820b6034228bdad884cc348716d in this term.
I20260202 02:52:23.925732 1180 ts_tablet_manager.cc:1916] T d1e0e74f32a24729b250c5751677cfb0 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:23.925806 1764 raft_consensus.cc:493] T d21fca9a5b234c26879717f51c8ea0f0 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:23.925935 1764 raft_consensus.cc:515] T d21fca9a5b234c26879717f51c8ea0f0 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.926512 1764 leader_election.cc:290] T d21fca9a5b234c26879717f51c8ea0f0 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:23.930078 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_87ac39d6a53346c5bdee4e2e5ef6e867"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:23.930511 921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d21fca9a5b234c26879717f51c8ea0f0" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:23.930711 921 raft_consensus.cc:2468] T d21fca9a5b234c26879717f51c8ea0f0 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
W20260202 02:52:23.930778 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_87ac39d6a53346c5bdee4e2e5ef6e867 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:23.931370 999 leader_election.cc:304] T d21fca9a5b234c26879717f51c8ea0f0 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:23.931674 1764 raft_consensus.cc:2804] T d21fca9a5b234c26879717f51c8ea0f0 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:23.931787 1764 raft_consensus.cc:493] T d21fca9a5b234c26879717f51c8ea0f0 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:23.931869 1764 raft_consensus.cc:3060] T d21fca9a5b234c26879717f51c8ea0f0 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:23.932221 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_bdbef281942540c7bf6d06c6c66b2acb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:23.932788 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bdbef281942540c7bf6d06c6c66b2acb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:23.934175 1173 tablet_bootstrap.cc:492] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:23.934355 1173 ts_tablet_manager.cc:1403] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.097s user 0.006s sys 0.001s
I20260202 02:52:23.934774 1882 raft_consensus.cc:2243] T bbf31e5619ed4f60b4d7c8b19ec8a549 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:23.934960 1882 raft_consensus.cc:2272] T bbf31e5619ed4f60b4d7c8b19ec8a549 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:23.935189 1173 raft_consensus.cc:359] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.935355 1173 raft_consensus.cc:385] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:23.935441 1173 raft_consensus.cc:740] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:23.935531 1882 ts_tablet_manager.cc:1916] T bbf31e5619ed4f60b4d7c8b19ec8a549 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:23.935745 1173 consensus_queue.cc:260] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.936414 1173 ts_tablet_manager.cc:1434] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:23.943034 526 catalog_manager.cc:5654] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d reported cstate change: term changed from 0 to 1, leader changed from <none> to b1f05820b6034228bdad884cc348716d (127.0.104.131). New cstate: current_term: 1 leader_uuid: "b1f05820b6034228bdad884cc348716d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } }
I20260202 02:52:23.943502 526 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 1f54c1dfe0a942b3bafd85522953a032 (table default.Table_a22f252ee8d44be5a9d7202afb726cfc [id=891d9355855640a28939b84b2dd30537]). Expected version 1 got 0
I20260202 02:52:23.943825 526 catalog_manager.cc:5654] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d reported cstate change: term changed from 0 to 1, leader changed from <none> to b1f05820b6034228bdad884cc348716d (127.0.104.131). New cstate: current_term: 1 leader_uuid: "b1f05820b6034228bdad884cc348716d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } }
I20260202 02:52:23.944255 526 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet ef073880898049e2be93d2cb6dc0c0bb (table default.Table_e8e9b4cdaf434b6cbbfed1a65bff1fc0 [id=78a7fb3a2eed4565b55d5dacb8a3d7df]). Expected version 1 got 0
I20260202 02:52:23.944543 1932 tablet.cc:1722] T 70306ed9dec54286885d965f2c012da4 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:23.946825 889 tablet_service.cc:1505] Processing CreateTablet for tablet 9f11a68d10374d06bc4a756a1d15bf09 (DEFAULT_TABLE table=default.Table_17dd200ab5824a4a90f05d78d31c6887 [id=d3627805d70e4e269f073250e883f3e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.947480 902 tablet_service.cc:1505] Processing CreateTablet for tablet cef87d5f74f14a6d86150958de7b1ace (DEFAULT_TABLE table=default.Table_aa12e8c2bff14a968a174b8879bba516 [id=50e58db611f247ee9dcd5b31776ace05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.949000 742 tablet_service.cc:1505] Processing CreateTablet for tablet 9f11a68d10374d06bc4a756a1d15bf09 (DEFAULT_TABLE table=default.Table_17dd200ab5824a4a90f05d78d31c6887 [id=d3627805d70e4e269f073250e883f3e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.949373 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f11a68d10374d06bc4a756a1d15bf09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.952700 758 tablet_service.cc:1505] Processing CreateTablet for tablet cef87d5f74f14a6d86150958de7b1ace (DEFAULT_TABLE table=default.Table_aa12e8c2bff14a968a174b8879bba516 [id=50e58db611f247ee9dcd5b31776ace05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.953167 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cef87d5f74f14a6d86150958de7b1ace. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.953442 1174 tablet_bootstrap.cc:492] T d21fca9a5b234c26879717f51c8ea0f0 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:23.953600 1174 ts_tablet_manager.cc:1403] T d21fca9a5b234c26879717f51c8ea0f0 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.054s user 0.003s sys 0.000s
I20260202 02:52:23.954384 527 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 70306ed9dec54286885d965f2c012da4 (table default.Table_d0e51ff8e83f4da19ce3c119dbec1ea7 [id=31abb0facd164caaa15591259eb951c2]). Expected version 1 got 0
I20260202 02:52:23.954623 527 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 20ea39bc5d3d46f5a4ba427277d6c6a3 (table default.Table_3b335a0806154f98a907f551596ba353 [id=0da4db61b3fe4cf4846edd5530b9327f]). Expected version 1 got 0
I20260202 02:52:23.954564 1174 raft_consensus.cc:359] T d21fca9a5b234c26879717f51c8ea0f0 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.954764 1174 raft_consensus.cc:385] T d21fca9a5b234c26879717f51c8ea0f0 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:23.954856 1174 raft_consensus.cc:740] T d21fca9a5b234c26879717f51c8ea0f0 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:23.955103 1174 consensus_queue.cc:260] T d21fca9a5b234c26879717f51c8ea0f0 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:23.956024 527 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 20ea39bc5d3d46f5a4ba427277d6c6a3 (table default.Table_3b335a0806154f98a907f551596ba353 [id=0da4db61b3fe4cf4846edd5530b9327f], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 20ea39bc5d3d46f5a4ba427277d6c6a3 consensus state has no leader
I20260202 02:52:23.956477 715 leader_election.cc:304] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:23.956624 527 master_service.cc:954] ReplaceTablet: received request to replace tablet 9c1271ce0f194021b588b82ac9d3db79 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:23.957252 893 tablet_service.cc:1552] Processing DeleteTablet for tablet 6154460838eb4e508f3646d3e3e4cf21 with delete_type TABLET_DATA_DELETED (replaced by tablet 5e20842de8cb4355b732685605c88834) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:23.958088 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_e4bcea76324043f09b2a8d4cdec7f371"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:23.958633 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e4bcea76324043f09b2a8d4cdec7f371 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:23.965078 1933 tablet.cc:1722] T 70306ed9dec54286885d965f2c012da4 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:23.972720 1277 raft_consensus.cc:2749] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260202 02:52:23.973001 519 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b427c7e897c64977b924e7d586420e69 consensus state has no leader
I20260202 02:52:23.973295 519 master_service.cc:954] ReplaceTablet: received request to replace tablet d21fca9a5b234c26879717f51c8ea0f0 from {username='slave'} at 127.0.0.1:47430
W20260202 02:52:23.973328 714 consensus_peers.cc:597] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b428356fba0c4fb5a6033c5d39740cc5. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:23.978374 857 consensus_peers.cc:597] T 622a113af04a41c6852c29fc8a8acd78 P 1521a6d262b7451bbef8debe970dd6b5 -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:23.936702 1173 tablet_bootstrap.cc:492] T 3711910e050c4515bed4294d1271d5a9 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:23.981333 1764 raft_consensus.cc:515] T d21fca9a5b234c26879717f51c8ea0f0 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.981344 1038 tablet_service.cc:1505] Processing CreateTablet for tablet 9f11a68d10374d06bc4a756a1d15bf09 (DEFAULT_TABLE table=default.Table_17dd200ab5824a4a90f05d78d31c6887 [id=d3627805d70e4e269f073250e883f3e3]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:23.987041 714 consensus_peers.cc:597] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 526f2c7b95184a0987e7aa94efc8475a. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:23.987139 523 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 70306ed9dec54286885d965f2c012da4 (table default.Table_d0e51ff8e83f4da19ce3c119dbec1ea7 [id=31abb0facd164caaa15591259eb951c2]). Expected version 1 got 0
I20260202 02:52:23.987951 1175 tablet_bootstrap.cc:492] T 042d1ebf357b4abe8814a407891ce265 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:23.988510 1175 ts_tablet_manager.cc:1403] T 042d1ebf357b4abe8814a407891ce265 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.138s user 0.003s sys 0.006s
I20260202 02:52:23.988623 893 tablet_service.cc:1552] Processing DeleteTablet for tablet 79f694b740954d429145fba9157338ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:21 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:23.990070 1175 raft_consensus.cc:359] T 042d1ebf357b4abe8814a407891ce265 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.990419 1175 raft_consensus.cc:385] T 042d1ebf357b4abe8814a407891ce265 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:23.990540 1175 raft_consensus.cc:740] T 042d1ebf357b4abe8814a407891ce265 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:23.990065 1174 ts_tablet_manager.cc:1434] T d21fca9a5b234c26879717f51c8ea0f0 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260202 02:52:23.990888 1175 consensus_queue.cc:260] T 042d1ebf357b4abe8814a407891ce265 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:23.992573 1882 ts_tablet_manager.cc:1929] T bbf31e5619ed4f60b4d7c8b19ec8a549 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:23.992749 1882 log.cc:1199] T bbf31e5619ed4f60b4d7c8b19ec8a549 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/bbf31e5619ed4f60b4d7c8b19ec8a549
I20260202 02:52:23.993116 1882 ts_tablet_manager.cc:1950] T bbf31e5619ed4f60b4d7c8b19ec8a549 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:23.993707 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f11a68d10374d06bc4a756a1d15bf09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.981357 1037 tablet_service.cc:1505] Processing CreateTablet for tablet cef87d5f74f14a6d86150958de7b1ace (DEFAULT_TABLE table=default.Table_aa12e8c2bff14a968a174b8879bba516 [id=50e58db611f247ee9dcd5b31776ace05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:23.995955 527 catalog_manager.cc:5654] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f442ef85029461091de847118c34c0d (127.0.104.129). New cstate: current_term: 1 leader_uuid: "1f442ef85029461091de847118c34c0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } }
I20260202 02:52:23.996982 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cef87d5f74f14a6d86150958de7b1ace. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.997792 1058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "20ea39bc5d3d46f5a4ba427277d6c6a3" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:23.998765 917 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d21fca9a5b234c26879717f51c8ea0f0" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:23.998975 917 raft_consensus.cc:3060] T d21fca9a5b234c26879717f51c8ea0f0 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:23.999341 1182 ts_tablet_manager.cc:1929] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:23.999491 1182 log.cc:1199] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/d0671586e45e45ffb7fd9e1be6e2bbf8
I20260202 02:52:23.999888 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f11a68d10374d06bc4a756a1d15bf09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:23.999917 1182 ts_tablet_manager.cc:1950] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:24.000485 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet d0671586e45e45ffb7fd9e1be6e2bbf8 (table default.Table_53cc5731c3e24222a97702bdde84a954 [id=b2fe4f8ae50642e9a6ea05759313b31b]) successfully deleted
I20260202 02:52:24.000569 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet c12ff20ca6b44ddba0c936a28cfb2eb9 with delete_type TABLET_DATA_DELETED (replaced by tablet 072bb7f0e4ab4935be49461bc8a3c163) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:24.000813 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "25765fb687294e9db1289f76530dd877" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:24.001236 1059 raft_consensus.cc:3060] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:24.001817 917 raft_consensus.cc:2468] T d21fca9a5b234c26879717f51c8ea0f0 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
W20260202 02:52:24.002743 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 7ff6d2fa085343a6b11be0898da6233d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7ff6d2fa085343a6b11be0898da6233d already in progress: creating tablet
I20260202 02:52:24.004505 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cef87d5f74f14a6d86150958de7b1ace. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.004628 1059 raft_consensus.cc:2468] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
W20260202 02:52:24.005002 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet b7880cb051c24bc998003f0effed3cf8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b7880cb051c24bc998003f0effed3cf8 already in progress: creating tablet
I20260202 02:52:24.005721 1182 tablet_replica.cc:333] T 5986676f5484498da927b4c6da3f47ff P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
W20260202 02:52:24.005687 859 leader_election.cc:343] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:24.005862 859 leader_election.cc:304] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:24.005970 1182 raft_consensus.cc:2243] T 5986676f5484498da927b4c6da3f47ff P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:24.006152 1865 raft_consensus.cc:2749] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:24.006565 859 leader_election.cc:304] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:24.006783 1182 raft_consensus.cc:2272] T 5986676f5484498da927b4c6da3f47ff P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:24.006884 1865 raft_consensus.cc:2804] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:24.007016 1865 raft_consensus.cc:697] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:24.007340 1865 consensus_queue.cc:237] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.007558 1723 consensus_queue.cc:1048] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:24.007895 1175 ts_tablet_manager.cc:1434] T 042d1ebf357b4abe8814a407891ce265 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.019s user 0.002s sys 0.002s
I20260202 02:52:24.008198 1175 tablet_bootstrap.cc:492] T af5aa68e47bb420eac300754d274fa73 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:24.008971 1764 leader_election.cc:290] T d21fca9a5b234c26879717f51c8ea0f0 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:24.011685 999 leader_election.cc:304] T d21fca9a5b234c26879717f51c8ea0f0 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:24.011765 1175 tablet_bootstrap.cc:654] T af5aa68e47bb420eac300754d274fa73 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:24.012040 423 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 51916.7us
I20260202 02:52:24.016068 1182 ts_tablet_manager.cc:1916] T 5986676f5484498da927b4c6da3f47ff P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:24.016479 999 consensus_peers.cc:597] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f93e8d3c4faf48eaa98c3cbf88d585e2. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:24.020615 1764 raft_consensus.cc:2804] T d21fca9a5b234c26879717f51c8ea0f0 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:24.020821 1764 raft_consensus.cc:697] T d21fca9a5b234c26879717f51c8ea0f0 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:24.021104 1764 consensus_queue.cc:237] T d21fca9a5b234c26879717f51c8ea0f0 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.022079 1173 tablet_bootstrap.cc:654] T 3711910e050c4515bed4294d1271d5a9 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:24.024087 524 master_service.cc:954] ReplaceTablet: received request to replace tablet dc4c3f52f4f44a84b0d0e3c9a4a49f09 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:24.024487 1174 tablet_bootstrap.cc:492] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:24.026257 1174 tablet_bootstrap.cc:654] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:24.028777 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet bbf31e5619ed4f60b4d7c8b19ec8a549 (table default.Table_ef11a1ba065d483c9f7862345ab1aedb [id=0e54d0abe11c46fa8dbf0befb46219cb]) successfully deleted
I20260202 02:52:24.028928 1059 raft_consensus.cc:1275] T 622a113af04a41c6852c29fc8a8acd78 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Refusing update from remote peer 1521a6d262b7451bbef8debe970dd6b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260202 02:52:24.032032 1175 tablet_bootstrap.cc:492] T af5aa68e47bb420eac300754d274fa73 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:24.032298 1175 ts_tablet_manager.cc:1403] T af5aa68e47bb420eac300754d274fa73 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.001s
I20260202 02:52:24.033314 1175 raft_consensus.cc:359] T af5aa68e47bb420eac300754d274fa73 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.033593 1175 raft_consensus.cc:385] T af5aa68e47bb420eac300754d274fa73 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:24.033758 1175 raft_consensus.cc:740] T af5aa68e47bb420eac300754d274fa73 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:24.034078 1175 consensus_queue.cc:260] T af5aa68e47bb420eac300754d274fa73 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.034633 1180 ts_tablet_manager.cc:1929] T d1e0e74f32a24729b250c5751677cfb0 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:24.034763 1180 log.cc:1199] T d1e0e74f32a24729b250c5751677cfb0 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/d1e0e74f32a24729b250c5751677cfb0
I20260202 02:52:24.035099 1180 ts_tablet_manager.cc:1950] T d1e0e74f32a24729b250c5751677cfb0 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:24.035868 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 277) took 4288 ms. Trace:
I20260202 02:52:24.035920 1175 ts_tablet_manager.cc:1434] T af5aa68e47bb420eac300754d274fa73 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260202 02:52:24.035995 1180 rpcz_store.cc:276] 0202 02:52:19.747866 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.748211 (+ 345us) service_pool.cc:225] Handling call
0202 02:52:23.918153 (+4169942us) ts_tablet_manager.cc:1151] Deleting tablet d1e0e74f32a24729b250c5751677cfb0
0202 02:52:23.918168 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:23.967045 (+ 48877us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:24.034481 (+ 67436us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:24.035830 (+ 1349us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4097864}
I20260202 02:52:24.036558 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet d1e0e74f32a24729b250c5751677cfb0 (table default.Table_5db99844c1bb484a863a9e289a19da3e [id=f1f6c97fc88340da8bab0347ad6430ab]) successfully deleted
I20260202 02:52:24.037323 1175 tablet_bootstrap.cc:492] T 2b8ba9dd450948ada395207ae1c5b325 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:24.039196 1175 tablet_bootstrap.cc:654] T 2b8ba9dd450948ada395207ae1c5b325 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:24.042194 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d21fca9a5b234c26879717f51c8ea0f0" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:24.042459 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d21fca9a5b234c26879717f51c8ea0f0" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:24.042557 779 raft_consensus.cc:2468] T d21fca9a5b234c26879717f51c8ea0f0 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:24.043285 1182 ts_tablet_manager.cc:1929] T 5986676f5484498da927b4c6da3f47ff P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260202 02:52:24.044306 1182 log.cc:1199] T 5986676f5484498da927b4c6da3f47ff P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/5986676f5484498da927b4c6da3f47ff
I20260202 02:52:24.046113 1182 ts_tablet_manager.cc:1950] T 5986676f5484498da927b4c6da3f47ff P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:24.046967 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 5986676f5484498da927b4c6da3f47ff (table default.Table_755b6ce0796b45b8be9df04e59dfe565 [id=d8485f23f63b493781bea557dbe8a0c1]) successfully deleted
W20260202 02:52:24.047811 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet d0ce2121d65b4a228596e0ca9018883d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d0ce2121d65b4a228596e0ca9018883d already in progress: creating tablet
I20260202 02:52:24.048290 1182 tablet_replica.cc:333] T 6154460838eb4e508f3646d3e3e4cf21 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:24.048895 1182 raft_consensus.cc:2243] T 6154460838eb4e508f3646d3e3e4cf21 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:24.051024 1182 raft_consensus.cc:2272] T 6154460838eb4e508f3646d3e3e4cf21 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:24.055303 1182 ts_tablet_manager.cc:1916] T 6154460838eb4e508f3646d3e3e4cf21 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:24.058694 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 285) took 4231 ms. Trace:
I20260202 02:52:24.058851 1180 rpcz_store.cc:276] 0202 02:52:19.827300 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.827632 (+ 332us) service_pool.cc:225] Handling call
0202 02:52:24.058510 (+4230878us) ts_tablet_manager.cc:1151] Deleting tablet 02b434fa058b45529ff3938b053263c6
0202 02:52:24.058520 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:24.058643 (+ 123us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4224753}
I20260202 02:52:24.058979 528 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 2b8ba9dd450948ada395207ae1c5b325
I20260202 02:52:24.060560 1180 tablet_replica.cc:333] T bc1539e5c8aa4d6e98f687728b891530 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:24.060848 1180 raft_consensus.cc:2243] T bc1539e5c8aa4d6e98f687728b891530 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:24.061065 1180 raft_consensus.cc:2272] T bc1539e5c8aa4d6e98f687728b891530 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:24.065282 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_1e4db591b5664638a2cd494429582cdf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:24.065963 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1e4db591b5664638a2cd494429582cdf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:24.066454 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 2b8ba9dd450948ada395207ae1c5b325 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:24.066532 528 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_87ac39d6a53346c5bdee4e2e5ef6e867" } modify_external_catalogs: true
I20260202 02:52:24.066741 528 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_87ac39d6a53346c5bdee4e2e5ef6e867" } modify_external_catalogs: true
I20260202 02:52:24.068837 758 tablet_service.cc:1552] Processing DeleteTablet for tablet 2b8ba9dd450948ada395207ae1c5b325 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:42950
W20260202 02:52:24.069460 423 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 50209.5us
I20260202 02:52:24.069808 520 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_dc83c58402214d3eb5cd548d2bc9a1a6" } modify_external_catalogs: true
I20260202 02:52:24.070015 520 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_dc83c58402214d3eb5cd548d2bc9a1a6" } modify_external_catalogs: true
W20260202 02:52:24.071053 714 consensus_peers.cc:597] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5986676f5484498da927b4c6da3f47ff. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:24.071203 521 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_61741238f8ad47cbb8fe712419268ff0" } new_table_name: "default.Table_7d121235cada49b2ba3f2f966bc02fc8" modify_external_catalogs: true
I20260202 02:52:24.071798 1180 ts_tablet_manager.cc:1916] T bc1539e5c8aa4d6e98f687728b891530 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:24.072753 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 02b434fa058b45529ff3938b053263c6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 02b434fa058b45529ff3938b053263c6 already in progress: creating tablet
I20260202 02:52:24.073398 522 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_51648e2a42614c3d9c5820901d153932" } modify_external_catalogs: true
I20260202 02:52:24.073709 522 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_51648e2a42614c3d9c5820901d153932" } modify_external_catalogs: true
I20260202 02:52:24.078681 519 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet d21fca9a5b234c26879717f51c8ea0f0
I20260202 02:52:24.079298 519 catalog_manager.cc:6724] ReplaceTablet: tablet d21fca9a5b234c26879717f51c8ea0f0 deleted and replaced by tablet 28f6161d0d0a4b41af4560f0f3bee0f7
I20260202 02:52:24.079690 758 tablet_service.cc:1552] Processing DeleteTablet for tablet d21fca9a5b234c26879717f51c8ea0f0 with delete_type TABLET_DATA_DELETED (replaced by tablet 28f6161d0d0a4b41af4560f0f3bee0f7) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:24.081257 1037 tablet_service.cc:1552] Processing DeleteTablet for tablet 2b8ba9dd450948ada395207ae1c5b325 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:24.081696 889 tablet_service.cc:1552] Processing DeleteTablet for tablet d21fca9a5b234c26879717f51c8ea0f0 with delete_type TABLET_DATA_DELETED (replaced by tablet 28f6161d0d0a4b41af4560f0f3bee0f7) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:24.082124 1865 consensus_queue.cc:1048] T 622a113af04a41c6852c29fc8a8acd78 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:24.082832 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_db26223b0d604aeabd9431ef5f27da88"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:24.083137 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet d21fca9a5b234c26879717f51c8ea0f0 with delete_type TABLET_DATA_DELETED (replaced by tablet 28f6161d0d0a4b41af4560f0f3bee0f7) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:24.083457 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_db26223b0d604aeabd9431ef5f27da88 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:24.084965 1174 tablet_bootstrap.cc:492] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:24.084970 1882 tablet_replica.cc:333] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:24.085150 1174 ts_tablet_manager.cc:1403] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.061s user 0.003s sys 0.001s
I20260202 02:52:24.086486 1174 raft_consensus.cc:359] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.086694 1174 raft_consensus.cc:385] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:24.086774 1174 raft_consensus.cc:740] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:24.087018 1174 consensus_queue.cc:260] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.087656 1174 ts_tablet_manager.cc:1434] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:24.088001 1174 tablet_bootstrap.cc:492] T ffd80badce1048f3ac95366cad05ae53 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:24.089284 1860 consensus_queue.cc:1048] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260202 02:52:24.089773 1174 tablet_bootstrap.cc:654] T ffd80badce1048f3ac95366cad05ae53 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:24.096756 1882 raft_consensus.cc:2243] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:24.097484 1882 raft_consensus.cc:2272] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:24.098156 524 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet dc4c3f52f4f44a84b0d0e3c9a4a49f09
I20260202 02:52:24.098133 1889 tablet.cc:1722] T 70306ed9dec54286885d965f2c012da4 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:24.098459 1882 ts_tablet_manager.cc:1916] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:24.098692 524 catalog_manager.cc:6724] ReplaceTablet: tablet dc4c3f52f4f44a84b0d0e3c9a4a49f09 deleted and replaced by tablet 3191a80374724effa8389adc842e3448
I20260202 02:52:24.098898 758 tablet_service.cc:1552] Processing DeleteTablet for tablet dc4c3f52f4f44a84b0d0e3c9a4a49f09 with delete_type TABLET_DATA_DELETED (replaced by tablet 3191a80374724effa8389adc842e3448) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:24.099741 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_346e800aad794a65acc77ee0bfb0b82c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:24.100373 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_346e800aad794a65acc77ee0bfb0b82c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:24.101709 889 tablet_service.cc:1552] Processing DeleteTablet for tablet dc4c3f52f4f44a84b0d0e3c9a4a49f09 with delete_type TABLET_DATA_DELETED (replaced by tablet 3191a80374724effa8389adc842e3448) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:24.102115 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet dc4c3f52f4f44a84b0d0e3c9a4a49f09 with delete_type TABLET_DATA_DELETED (replaced by tablet 3191a80374724effa8389adc842e3448) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:24.107992 525 master_service.cc:954] ReplaceTablet: received request to replace tablet c9ef08b5830147a3a55049285a40535b from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:24.108543 528 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet ec7665db4c3344fa8f96b8d3b0539c10
I20260202 02:52:24.109735 1723 consensus_queue.cc:1048] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:24.113298 528 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_698a93b0be2e4203836e0522ac44acc3" } modify_external_catalogs: true
I20260202 02:52:24.113526 528 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_698a93b0be2e4203836e0522ac44acc3" } modify_external_catalogs: true
W20260202 02:52:24.116117 521 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c977c146bac34638874b7a990604a0d5 consensus state has no leader
I20260202 02:52:24.116492 520 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet 8c027ca37e3f4158ab6c4cd8d5854612
I20260202 02:52:24.116990 522 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet da5435f969074364bc8f8a6ddc5e4f8a
I20260202 02:52:24.118316 758 tablet_service.cc:1552] Processing DeleteTablet for tablet da5435f969074364bc8f8a6ddc5e4f8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:24.118534 889 tablet_service.cc:1552] Processing DeleteTablet for tablet da5435f969074364bc8f8a6ddc5e4f8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:24.119803 1173 tablet_bootstrap.cc:492] T 3711910e050c4515bed4294d1271d5a9 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:24.119897 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet da5435f969074364bc8f8a6ddc5e4f8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:24.120005 1173 ts_tablet_manager.cc:1403] T 3711910e050c4515bed4294d1271d5a9 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.183s user 0.005s sys 0.012s
I20260202 02:52:24.120628 520 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_520d85c11ce24a88a0a5f162d14e8a61" } new_table_name: "default.Table_ed8a8cefa5de4d90a2d57043595329ae" modify_external_catalogs: true
I20260202 02:52:24.120589 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_62e7c5405f2b46248650ac463717478b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:24.120875 1173 raft_consensus.cc:359] T 3711910e050c4515bed4294d1271d5a9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:24.121093 1173 raft_consensus.cc:385] T 3711910e050c4515bed4294d1271d5a9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:24.121162 1173 raft_consensus.cc:740] T 3711910e050c4515bed4294d1271d5a9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
W20260202 02:52:24.121227 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_62e7c5405f2b46248650ac463717478b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:24.121443 1173 consensus_queue.cc:260] T 3711910e050c4515bed4294d1271d5a9 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:24.122303 1173 ts_tablet_manager.cc:1434] T 3711910e050c4515bed4294d1271d5a9 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:24.122380 522 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_5eefa56618e84f8d92f8d87132ccd7d4" } new_table_name: "default.Table_1b9d1f82c9004ea5b02312053140e514" modify_external_catalogs: true
I20260202 02:52:24.122622 1173 tablet_bootstrap.cc:492] T 6154460838eb4e508f3646d3e3e4cf21 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:24.124392 1173 tablet_bootstrap.cc:654] T 6154460838eb4e508f3646d3e3e4cf21 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:24.127471 1175 tablet_bootstrap.cc:492] T 2b8ba9dd450948ada395207ae1c5b325 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:24.127672 1175 ts_tablet_manager.cc:1403] T 2b8ba9dd450948ada395207ae1c5b325 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.090s user 0.004s sys 0.000s
I20260202 02:52:24.128507 1175 raft_consensus.cc:359] T 2b8ba9dd450948ada395207ae1c5b325 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.128811 1175 raft_consensus.cc:385] T 2b8ba9dd450948ada395207ae1c5b325 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:24.129117 1175 raft_consensus.cc:740] T 2b8ba9dd450948ada395207ae1c5b325 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:24.129472 1175 consensus_queue.cc:260] T 2b8ba9dd450948ada395207ae1c5b325 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.130112 1175 ts_tablet_manager.cc:1434] T 2b8ba9dd450948ada395207ae1c5b325 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:24.130523 1175 tablet_bootstrap.cc:492] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:24.136353 1882 ts_tablet_manager.cc:1929] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:24.137842 1882 log.cc:1199] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/bc1539e5c8aa4d6e98f687728b891530
I20260202 02:52:24.138549 1882 ts_tablet_manager.cc:1950] T bc1539e5c8aa4d6e98f687728b891530 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:24.138576 1180 ts_tablet_manager.cc:1929] T bc1539e5c8aa4d6e98f687728b891530 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:24.138743 1180 log.cc:1199] T bc1539e5c8aa4d6e98f687728b891530 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/bc1539e5c8aa4d6e98f687728b891530
I20260202 02:52:24.142061 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet bc1539e5c8aa4d6e98f687728b891530 (table default.Table_f1e417bf19b94bc491c3937276c669fd [id=9a8a252de32848288c9b3752c7d3feb7]) successfully deleted
I20260202 02:52:24.145329 1175 tablet_bootstrap.cc:654] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:24.146332 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet a5622ef925bc46cca76f481ddfd12afd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a5622ef925bc46cca76f481ddfd12afd already in progress: creating tablet
W20260202 02:52:24.147105 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 68823eff1bc04108ba83487605aec0ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 68823eff1bc04108ba83487605aec0ea already in progress: creating tablet
I20260202 02:52:24.147526 1882 tablet_replica.cc:333] T 583e547d0a1943f09f4efe82070fc777 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:24.149933 1882 raft_consensus.cc:2243] T 583e547d0a1943f09f4efe82070fc777 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:24.150250 1882 raft_consensus.cc:2272] T 583e547d0a1943f09f4efe82070fc777 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:24.150315 1174 tablet_bootstrap.cc:492] T ffd80badce1048f3ac95366cad05ae53 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:24.150485 1174 ts_tablet_manager.cc:1403] T ffd80badce1048f3ac95366cad05ae53 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.063s user 0.003s sys 0.000s
I20260202 02:52:24.151302 1174 raft_consensus.cc:359] T ffd80badce1048f3ac95366cad05ae53 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.151494 1174 raft_consensus.cc:385] T ffd80badce1048f3ac95366cad05ae53 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:24.151587 1174 raft_consensus.cc:740] T ffd80badce1048f3ac95366cad05ae53 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:24.151801 1174 consensus_queue.cc:260] T ffd80badce1048f3ac95366cad05ae53 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.152494 1174 ts_tablet_manager.cc:1434] T ffd80badce1048f3ac95366cad05ae53 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260202 02:52:24.152690 1173 tablet_bootstrap.cc:492] T 6154460838eb4e508f3646d3e3e4cf21 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:24.152807 1174 tablet_bootstrap.cc:492] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:24.152918 1173 ts_tablet_manager.cc:1403] T 6154460838eb4e508f3646d3e3e4cf21 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.030s user 0.004s sys 0.000s
I20260202 02:52:24.153867 1173 raft_consensus.cc:359] T 6154460838eb4e508f3646d3e3e4cf21 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.154145 1173 raft_consensus.cc:385] T 6154460838eb4e508f3646d3e3e4cf21 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:24.154271 1173 raft_consensus.cc:740] T 6154460838eb4e508f3646d3e3e4cf21 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:24.154534 1173 consensus_queue.cc:260] T 6154460838eb4e508f3646d3e3e4cf21 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.155172 1173 ts_tablet_manager.cc:1434] T 6154460838eb4e508f3646d3e3e4cf21 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:24.155611 1173 tablet_bootstrap.cc:492] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:24.153918 1182 ts_tablet_manager.cc:1929] T 6154460838eb4e508f3646d3e3e4cf21 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:24.156499 1182 log.cc:1199] T 6154460838eb4e508f3646d3e3e4cf21 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/6154460838eb4e508f3646d3e3e4cf21
I20260202 02:52:24.156883 1182 ts_tablet_manager.cc:1950] T 6154460838eb4e508f3646d3e3e4cf21 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:24.157490 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 6154460838eb4e508f3646d3e3e4cf21 (table default.Table_6e6ab095d9a6497d8a89bcc865b6f42f [id=7d8f7ac4c2ee4b72b2ff25aff060010f]) successfully deleted
I20260202 02:52:24.156178 1882 ts_tablet_manager.cc:1916] T 583e547d0a1943f09f4efe82070fc777 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:24.157773 1180 ts_tablet_manager.cc:1950] T bc1539e5c8aa4d6e98f687728b891530 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:24.158563 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 283) took 4333 ms. Trace:
I20260202 02:52:24.158671 1180 rpcz_store.cc:276] 0202 02:52:19.825552 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.825715 (+ 163us) service_pool.cc:225] Handling call
0202 02:52:24.060472 (+4234757us) ts_tablet_manager.cc:1151] Deleting tablet bc1539e5c8aa4d6e98f687728b891530
0202 02:52:24.060485 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:24.074843 (+ 14358us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:24.138427 (+ 63584us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:24.158509 (+ 20082us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4226117}
I20260202 02:52:24.159053 1180 tablet_replica.cc:333] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:24.159202 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet bc1539e5c8aa4d6e98f687728b891530 (table default.Table_f1e417bf19b94bc491c3937276c669fd [id=9a8a252de32848288c9b3752c7d3feb7]) successfully deleted
I20260202 02:52:24.159245 1180 raft_consensus.cc:2243] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:24.159436 1180 raft_consensus.cc:2272] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:24.159394 524 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_6d32a9b50b2644b1a5b287ce8882ebab" } new_table_name: "default.Table_e4018d1c57714a24be5d3a20c1ccaef4" modify_external_catalogs: true
I20260202 02:52:24.163163 525 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet c9ef08b5830147a3a55049285a40535b
I20260202 02:52:24.164180 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet c9ef08b5830147a3a55049285a40535b with delete_type TABLET_DATA_DELETED (replaced by tablet 5d2fc0a74684434d85b9922cb4e32fdc) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:24.164196 758 tablet_service.cc:1552] Processing DeleteTablet for tablet c9ef08b5830147a3a55049285a40535b with delete_type TABLET_DATA_DELETED (replaced by tablet 5d2fc0a74684434d85b9922cb4e32fdc) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:24.165421 1173 tablet_bootstrap.cc:654] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:24.167052 889 tablet_service.cc:1552] Processing DeleteTablet for tablet c9ef08b5830147a3a55049285a40535b with delete_type TABLET_DATA_DELETED (replaced by tablet 5d2fc0a74684434d85b9922cb4e32fdc) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:24.167824 1933 tablet.cc:1722] T 622a113af04a41c6852c29fc8a8acd78 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:24.173400 525 catalog_manager.cc:6724] ReplaceTablet: tablet c9ef08b5830147a3a55049285a40535b deleted and replaced by tablet 5d2fc0a74684434d85b9922cb4e32fdc
I20260202 02:52:24.173826 1182 tablet_replica.cc:333] T 79f694b740954d429145fba9157338ca P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:24.176599 1182 raft_consensus.cc:2243] T 79f694b740954d429145fba9157338ca P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:24.179395 525 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_6cd4b2d180cf4fdebad15ee5ea5ea7de" } new_table_name: "default.Table_48faa250b04a4b61a37f532584081968" modify_external_catalogs: true
W20260202 02:52:24.180034 520 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 745fc215251e4ccb9a2abed02573dd84 has no consensus state
I20260202 02:52:24.181406 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_3d5171d76fb147efb648b0e99b6c3d9d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:24.181694 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_0c6fcdb4acc84119977343919bed1258"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:24.182062 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3d5171d76fb147efb648b0e99b6c3d9d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:24.182209 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0c6fcdb4acc84119977343919bed1258 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:24.183871 1182 raft_consensus.cc:2272] T 79f694b740954d429145fba9157338ca P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:24.184310 999 consensus_peers.cc:597] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9884c90cf1d245b2a7eaaf00d71b1c3e. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:24.185150 1182 ts_tablet_manager.cc:1916] T 79f694b740954d429145fba9157338ca P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:24.185293 1174 tablet_bootstrap.cc:654] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:24.188664 1889 tablet.cc:1722] T 622a113af04a41c6852c29fc8a8acd78 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:24.189229 1882 ts_tablet_manager.cc:1929] T 583e547d0a1943f09f4efe82070fc777 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:24.190194 1173 tablet_bootstrap.cc:492] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:24.190469 1173 ts_tablet_manager.cc:1403] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.035s user 0.004s sys 0.000s
I20260202 02:52:24.191339 1173 raft_consensus.cc:359] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.191633 1173 raft_consensus.cc:385] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:24.191757 1173 raft_consensus.cc:740] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:24.192092 1173 consensus_queue.cc:260] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.193102 1173 ts_tablet_manager.cc:1434] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:24.193177 1175 tablet_bootstrap.cc:492] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:24.193399 1175 ts_tablet_manager.cc:1403] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.063s user 0.000s sys 0.006s
I20260202 02:52:24.193465 1173 tablet_bootstrap.cc:492] T d0671586e45e45ffb7fd9e1be6e2bbf8 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:24.194684 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_5538c7d2137648efb81b8057751d9972"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:24.195253 1173 tablet_bootstrap.cc:654] T d0671586e45e45ffb7fd9e1be6e2bbf8 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:24.195307 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5538c7d2137648efb81b8057751d9972 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:24.196180 1882 log.cc:1199] T 583e547d0a1943f09f4efe82070fc777 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/583e547d0a1943f09f4efe82070fc777
I20260202 02:52:24.196630 1882 ts_tablet_manager.cc:1950] T 583e547d0a1943f09f4efe82070fc777 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:24.197314 1175 raft_consensus.cc:359] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:24.197605 1175 raft_consensus.cc:385] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:24.197739 1175 raft_consensus.cc:740] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:24.197718 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 583e547d0a1943f09f4efe82070fc777 (table default.Table_2669ca9662b942c08bcbfd7641cfe2e3 [id=fb813e4c5b0849f8a488c7a22d1defae]) successfully deleted
I20260202 02:52:24.198107 1180 ts_tablet_manager.cc:1916] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:24.198092 1175 consensus_queue.cc:260] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:24.200318 1175 ts_tablet_manager.cc:1434] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.007s user 0.000s sys 0.001s
I20260202 02:52:24.201656 1175 tablet_bootstrap.cc:492] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:24.201794 1894 raft_consensus.cc:493] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:24.202095 1894 raft_consensus.cc:515] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.203065 1894 leader_election.cc:290] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:24.203459 917 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e1284fde67bc4d65b067a54c416ccc33" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:24.203683 917 raft_consensus.cc:2468] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:24.201944 1882 tablet_replica.cc:333] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:24.203825 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e1284fde67bc4d65b067a54c416ccc33" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:24.204092 1882 raft_consensus.cc:2243] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:24.204201 999 leader_election.cc:304] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:24.204470 1882 pending_rounds.cc:70] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d: Trying to abort 1 pending ops.
I20260202 02:52:24.204577 1882 pending_rounds.cc:77] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923040944046080 op_type: NO_OP noop_request { }
I20260202 02:52:24.204730 1882 raft_consensus.cc:2889] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:24.204874 1894 raft_consensus.cc:2804] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:24.204962 1894 raft_consensus.cc:493] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:24.205057 1894 raft_consensus.cc:3060] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:24.207985 1894 raft_consensus.cc:515] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:24.204741 998 leader_election.cc:343] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:24.211947 1764 raft_consensus.cc:493] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:24.212138 1894 leader_election.cc:290] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:24.212388 1764 raft_consensus.cc:515] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.212610 1175 tablet_bootstrap.cc:654] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:24.212755 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e1284fde67bc4d65b067a54c416ccc33" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:24.212774 917 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e1284fde67bc4d65b067a54c416ccc33" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:24.212976 917 raft_consensus.cc:3060] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:24.215849 917 raft_consensus.cc:2468] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:24.216533 1277 consensus_queue.cc:1048] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:24.218636 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_d6bec77a74b1431fb1c008f55f482b3a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:24.219238 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d6bec77a74b1431fb1c008f55f482b3a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:24.218782 423 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 143132us
I20260202 02:52:24.221796 1174 tablet_bootstrap.cc:492] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:24.221977 1174 ts_tablet_manager.cc:1403] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.069s user 0.003s sys 0.000s
I20260202 02:52:24.222797 1174 raft_consensus.cc:359] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.223017 1174 raft_consensus.cc:385] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:24.223105 1174 raft_consensus.cc:740] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:24.223328 1174 consensus_queue.cc:260] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.223793 1174 ts_tablet_manager.cc:1434] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260202 02:52:24.223924 1764 leader_election.cc:290] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:24.213351 998 leader_election.cc:343] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:24.224686 999 leader_election.cc:304] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:24.225721 1764 raft_consensus.cc:2804] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:24.225745 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f6746ccbf37e477196a9706396d1e3fd" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:24.225737 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f6746ccbf37e477196a9706396d1e3fd" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:24.225869 921 raft_consensus.cc:1275] T ef073880898049e2be93d2cb6dc0c0bb P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Refusing update from remote peer b1f05820b6034228bdad884cc348716d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260202 02:52:24.225970 923 raft_consensus.cc:2468] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:24.225991 1764 raft_consensus.cc:697] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:24.226290 772 raft_consensus.cc:1275] T ef073880898049e2be93d2cb6dc0c0bb P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Refusing update from remote peer b1f05820b6034228bdad884cc348716d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260202 02:52:24.226507 999 leader_election.cc:304] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:24.204879 1882 raft_consensus.cc:2272] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:24.226804 1860 consensus_queue.cc:1048] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:24.227293 1860 raft_consensus.cc:2804] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:24.227401 1860 raft_consensus.cc:493] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:24.227478 1860 raft_consensus.cc:3060] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:24.218393 1059 raft_consensus.cc:1275] T 78f122342dd144b48de89902faa0f94f P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Refusing update from remote peer 1f442ef85029461091de847118c34c0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260202 02:52:24.229090 998 leader_election.cc:343] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:24.229462 1882 ts_tablet_manager.cc:1916] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:24.229589 1721 consensus_queue.cc:1048] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:24.230194 1860 raft_consensus.cc:515] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.230736 1860 leader_election.cc:290] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:24.231479 1764 consensus_queue.cc:237] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.232188 921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f6746ccbf37e477196a9706396d1e3fd" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:24.232432 921 raft_consensus.cc:3060] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:24.235271 921 raft_consensus.cc:2468] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:24.235826 999 leader_election.cc:304] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:24.236238 1860 raft_consensus.cc:2804] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:24.236423 1860 raft_consensus.cc:697] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:24.236750 1860 consensus_queue.cc:237] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:24.238322 524 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 16b47a2831f54d809ed0b16deaec7e3d consensus state has no leader
I20260202 02:52:24.238819 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_85ea5ebb89ae4a8fb42f2f5c96ab588c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:24.239497 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_85ea5ebb89ae4a8fb42f2f5c96ab588c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:24.241084 526 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 70306ed9dec54286885d965f2c012da4 (table default.Table_d0e51ff8e83f4da19ce3c119dbec1ea7 [id=31abb0facd164caaa15591259eb951c2]). Expected version 1 got 0
I20260202 02:52:24.241458 526 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 25765fb687294e9db1289f76530dd877 (table default.Table_8c47663f416b467cad37110bd09d85ad [id=ea0c52a72da8422ba14345fbfc242d27]). Expected version 1 got 0
I20260202 02:52:24.241818 527 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_d0e51ff8e83f4da19ce3c119dbec1ea7 [id=31abb0facd164caaa15591259eb951c2] alter complete (version 1)
I20260202 02:52:24.242573 1721 raft_consensus.cc:493] T 3711910e050c4515bed4294d1271d5a9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:24.242748 1721 raft_consensus.cc:515] T 3711910e050c4515bed4294d1271d5a9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:24.242882 1182 ts_tablet_manager.cc:1929] T 79f694b740954d429145fba9157338ca P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:24.243019 1182 log.cc:1199] T 79f694b740954d429145fba9157338ca P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/79f694b740954d429145fba9157338ca
I20260202 02:52:24.243784 1721 leader_election.cc:290] T 3711910e050c4515bed4294d1271d5a9 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:24.245018 758 tablet_service.cc:1552] Processing DeleteTablet for tablet eb0ac55fbb534b29a86d6b3e8aa72ebc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:24.245476 742 tablet_service.cc:1552] Processing DeleteTablet for tablet d1e0e74f32a24729b250c5751677cfb0 with delete_type TABLET_DATA_DELETED (replaced by tablet 2c31b45dc8004eae993e972e9faa3ff8) from {username='slave'} at 127.0.0.1:42950
W20260202 02:52:24.247848 999 leader_election.cc:343] T 3711910e050c4515bed4294d1271d5a9 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 3711910e050c4515bed4294d1271d5a9
I20260202 02:52:24.248297 742 tablet_service.cc:1505] Processing CreateTablet for tablet 4e90f273439d414caf16d63e1c605d3d (DEFAULT_TABLE table=default.Table_e11021f60bfe480ab07b2242bb36e94f [id=ae5d5815dcce405a897ffbce9d745591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.248756 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e90f273439d414caf16d63e1c605d3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.248745 902 tablet_service.cc:1505] Processing CreateTablet for tablet 3338964dfed548f980f58147f0283691 (DEFAULT_TABLE table=default.Table_077acfd01c1648ef96128a866323ec3e [id=12587d4b156547d185b0e2e07100aa62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.251006 754 tablet_service.cc:1505] Processing CreateTablet for tablet a5f864fd8a3d4c4a83e7f68abb8a0a83 (DEFAULT_TABLE table=default.Table_b9aa847125ed4116b6fde1cfc1cafa27 [id=47df1b09bcd24f29b1296f9cf52ac008]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.251397 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5f864fd8a3d4c4a83e7f68abb8a0a83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.254364 899 tablet_service.cc:1505] Processing CreateTablet for tablet 23671408fcbc454cbd918328015cf6e6 (DEFAULT_TABLE table=default.Table_2c56436bf39647f2b5af982d04f990dc [id=e9024f48361941d2bf97d0fb9ea93cb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.254828 1932 tablet.cc:1722] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:24.256657 751 tablet_service.cc:1505] Processing CreateTablet for tablet 42321f4e77484027a485fdd584de3ae3 (DEFAULT_TABLE table=default.Table_aee0b3d530954ca3ba6492f8263dd7ff [id=384621c2a9d94b0db7a7928a3bd8fffc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.256770 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f6746ccbf37e477196a9706396d1e3fd" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:24.257112 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42321f4e77484027a485fdd584de3ae3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.257668 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3711910e050c4515bed4294d1271d5a9" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:24.258841 998 leader_election.cc:343] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:24.259217 998 leader_election.cc:343] T 3711910e050c4515bed4294d1271d5a9 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:24.259330 998 leader_election.cc:304] T 3711910e050c4515bed4294d1271d5a9 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:24.260479 1894 raft_consensus.cc:2749] T 3711910e050c4515bed4294d1271d5a9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:24.261309 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 3338964dfed548f980f58147f0283691 (DEFAULT_TABLE table=default.Table_077acfd01c1648ef96128a866323ec3e [id=12587d4b156547d185b0e2e07100aa62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.261999 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3338964dfed548f980f58147f0283691. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.264741 1044 tablet_service.cc:1505] Processing CreateTablet for tablet a5f864fd8a3d4c4a83e7f68abb8a0a83 (DEFAULT_TABLE table=default.Table_b9aa847125ed4116b6fde1cfc1cafa27 [id=47df1b09bcd24f29b1296f9cf52ac008]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.265234 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5f864fd8a3d4c4a83e7f68abb8a0a83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.267248 886 tablet_service.cc:1505] Processing CreateTablet for tablet 916f52df8e0a4838b46772ecab4f852c (DEFAULT_TABLE table=default.Table_e7dff4eb18124743848a9f61ba603f6f [id=976c3ac7dde84abab2a5666eb46576e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.268708 1042 tablet_service.cc:1505] Processing CreateTablet for tablet 42321f4e77484027a485fdd584de3ae3 (DEFAULT_TABLE table=default.Table_aee0b3d530954ca3ba6492f8263dd7ff [id=384621c2a9d94b0db7a7928a3bd8fffc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.269119 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42321f4e77484027a485fdd584de3ae3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.270234 747 tablet_service.cc:1505] Processing CreateTablet for tablet 916f52df8e0a4838b46772ecab4f852c (DEFAULT_TABLE table=default.Table_e7dff4eb18124743848a9f61ba603f6f [id=976c3ac7dde84abab2a5666eb46576e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.270639 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 916f52df8e0a4838b46772ecab4f852c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.272724 759 tablet_service.cc:1505] Processing CreateTablet for tablet 23671408fcbc454cbd918328015cf6e6 (DEFAULT_TABLE table=default.Table_2c56436bf39647f2b5af982d04f990dc [id=e9024f48361941d2bf97d0fb9ea93cb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.273119 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23671408fcbc454cbd918328015cf6e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.277779 758 tablet_service.cc:1505] Processing CreateTablet for tablet 3338964dfed548f980f58147f0283691 (DEFAULT_TABLE table=default.Table_077acfd01c1648ef96128a866323ec3e [id=12587d4b156547d185b0e2e07100aa62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.278219 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3338964dfed548f980f58147f0283691. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:24.280175 525 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 28f6161d0d0a4b41af4560f0f3bee0f7 has no consensus state
I20260202 02:52:24.283905 1882 ts_tablet_manager.cc:1929] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:24.284067 1882 log.cc:1199] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/c184d44d7f434b39a997a7387e5d3de0
I20260202 02:52:24.284529 1882 ts_tablet_manager.cc:1950] T c184d44d7f434b39a997a7387e5d3de0 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:24.285514 1038 tablet_service.cc:1505] Processing CreateTablet for tablet 4e90f273439d414caf16d63e1c605d3d (DEFAULT_TABLE table=default.Table_e11021f60bfe480ab07b2242bb36e94f [id=ae5d5815dcce405a897ffbce9d745591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.285916 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e90f273439d414caf16d63e1c605d3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.288118 1963 consensus_queue.cc:1048] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260202 02:52:24.290153 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 23671408fcbc454cbd918328015cf6e6 (DEFAULT_TABLE table=default.Table_2c56436bf39647f2b5af982d04f990dc [id=e9024f48361941d2bf97d0fb9ea93cb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.290592 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23671408fcbc454cbd918328015cf6e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.290470 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_79cf9f546b684c598c9447efe6997ca7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:24.291066 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_79cf9f546b684c598c9447efe6997ca7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:24.292426 525 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_6da292362a6c43738dbeb895750b918a" } modify_external_catalogs: true
I20260202 02:52:24.292618 525 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_6da292362a6c43738dbeb895750b918a" } modify_external_catalogs: true
I20260202 02:52:24.301553 1889 tablet.cc:1722] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:24.307135 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 916f52df8e0a4838b46772ecab4f852c (DEFAULT_TABLE table=default.Table_e7dff4eb18124743848a9f61ba603f6f [id=976c3ac7dde84abab2a5666eb46576e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.307631 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 916f52df8e0a4838b46772ecab4f852c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.308583 520 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet ef073880898049e2be93d2cb6dc0c0bb (table default.Table_e8e9b4cdaf434b6cbbfed1a65bff1fc0 [id=78a7fb3a2eed4565b55d5dacb8a3d7df]). Expected version 1 got 0
I20260202 02:52:24.308792 520 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 042d1ebf357b4abe8814a407891ce265 (table default.Table_950f6f5ae1ef43399de785df25c896e9 [id=cab83305ba6648e5b307441e71671a82]). Expected version 1 got 0
I20260202 02:52:24.309152 520 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 78f122342dd144b48de89902faa0f94f (table default.Table_1b9d1f82c9004ea5b02312053140e514 [id=b4ab2ba86e1a49338a98d855ded99f5d]). Expected version 2 got 0
I20260202 02:52:24.309556 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 349) took 1106 ms. Trace:
I20260202 02:52:24.309702 1882 rpcz_store.cc:276] 0202 02:52:23.203427 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.203840 (+ 413us) service_pool.cc:225] Handling call
0202 02:52:24.201851 (+998011us) ts_tablet_manager.cc:1151] Deleting tablet c184d44d7f434b39a997a7387e5d3de0
0202 02:52:24.201863 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:24.265282 (+ 63419us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:24.283762 (+ 18480us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:24.309489 (+ 25727us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":225,"tablet-delete.queue_time_us":985347}
I20260202 02:52:24.310241 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 365) took 1092 ms. Trace:
I20260202 02:52:24.310329 1882 rpcz_store.cc:276] 0202 02:52:23.217535 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.218593 (+ 1058us) service_pool.cc:225] Handling call
0202 02:52:24.310069 (+1091476us) ts_tablet_manager.cc:1151] Deleting tablet 2b352d9a815e4bccafb7e2251500c5bc
0202 02:52:24.310079 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:24.310182 (+ 103us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1091239}
I20260202 02:52:24.310702 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 356) took 1097 ms. Trace:
I20260202 02:52:24.310806 1882 rpcz_store.cc:276] 0202 02:52:23.213062 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.213397 (+ 335us) service_pool.cc:225] Handling call
0202 02:52:24.310546 (+1097149us) ts_tablet_manager.cc:1151] Deleting tablet 79f694b740954d429145fba9157338ca
0202 02:52:24.310554 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:24.310649 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1091196}
I20260202 02:52:24.309330 520 catalog_manager.cc:5654] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130). New cstate: current_term: 1 leader_uuid: "1521a6d262b7451bbef8debe970dd6b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: HEALTHY } } }
I20260202 02:52:24.311136 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 357) took 1097 ms. Trace:
I20260202 02:52:24.311225 1882 rpcz_store.cc:276] 0202 02:52:23.213175 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.213463 (+ 288us) service_pool.cc:225] Handling call
0202 02:52:24.310994 (+1097531us) ts_tablet_manager.cc:1151] Deleting tablet 59ad016e15ec4a488921eb564029b689
0202 02:52:24.311002 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:24.311083 (+ 81us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1091489}
I20260202 02:52:24.311507 1882 tablet_replica.cc:333] T 723921dfe54c4e66af5cab76a25fee0a P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:24.311618 520 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 25765fb687294e9db1289f76530dd877 (table default.Table_8c47663f416b467cad37110bd09d85ad [id=ea0c52a72da8422ba14345fbfc242d27]). Expected version 1 got 0
I20260202 02:52:24.320317 747 tablet_service.cc:1505] Processing CreateTablet for tablet e33f95e4f17a479c9acef5764c6c4c56 (DEFAULT_TABLE table=default.Table_ae54fc1a1cfc4475823c8c3b5cf86e5f [id=5c2778a5ed864321bb59e445b4db671a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.321269 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e33f95e4f17a479c9acef5764c6c4c56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.322034 1882 raft_consensus.cc:2243] T 723921dfe54c4e66af5cab76a25fee0a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:24.322331 1882 raft_consensus.cc:2272] T 723921dfe54c4e66af5cab76a25fee0a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:24.322964 1882 ts_tablet_manager.cc:1916] T 723921dfe54c4e66af5cab76a25fee0a P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:24.328912 1041 tablet_service.cc:1505] Processing CreateTablet for tablet e33f95e4f17a479c9acef5764c6c4c56 (DEFAULT_TABLE table=default.Table_ae54fc1a1cfc4475823c8c3b5cf86e5f [id=5c2778a5ed864321bb59e445b4db671a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.329362 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e33f95e4f17a479c9acef5764c6c4c56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.330511 1173 tablet_bootstrap.cc:492] T d0671586e45e45ffb7fd9e1be6e2bbf8 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:24.330651 1173 ts_tablet_manager.cc:1403] T d0671586e45e45ffb7fd9e1be6e2bbf8 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.137s user 0.003s sys 0.010s
I20260202 02:52:24.331449 1173 raft_consensus.cc:359] T d0671586e45e45ffb7fd9e1be6e2bbf8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.331622 1173 raft_consensus.cc:385] T d0671586e45e45ffb7fd9e1be6e2bbf8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:24.331697 1173 raft_consensus.cc:740] T d0671586e45e45ffb7fd9e1be6e2bbf8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:24.331939 1173 consensus_queue.cc:260] T d0671586e45e45ffb7fd9e1be6e2bbf8 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.332521 1173 ts_tablet_manager.cc:1434] T d0671586e45e45ffb7fd9e1be6e2bbf8 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:24.328265 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet c184d44d7f434b39a997a7387e5d3de0 (table default.Table_ab29f752f1e140da9038d9e11ee61f39 [id=9ef504467a544f299b48499e49adf27f]) successfully deleted
W20260202 02:52:24.336292 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 2b352d9a815e4bccafb7e2251500c5bc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2b352d9a815e4bccafb7e2251500c5bc already in progress: creating tablet
W20260202 02:52:24.336760 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 79f694b740954d429145fba9157338ca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 79f694b740954d429145fba9157338ca already in progress: creating tablet
I20260202 02:52:24.337003 1889 tablet.cc:1722] T 78f122342dd144b48de89902faa0f94f P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260202 02:52:24.337147 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 59ad016e15ec4a488921eb564029b689 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 59ad016e15ec4a488921eb564029b689 already in progress: creating tablet
I20260202 02:52:24.338863 521 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_61c2ba441aa746edbb73f8b5de4204ba" } modify_external_catalogs: true
I20260202 02:52:24.338913 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_5ce52f099d05416f87636eaf90c9f958"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:24.339087 521 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_61c2ba441aa746edbb73f8b5de4204ba" } modify_external_catalogs: true
W20260202 02:52:24.339471 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5ce52f099d05416f87636eaf90c9f958 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:24.340570 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_0c6fcdb4acc84119977343919bed1258" } new_table_name: "default.Table_e72399a9ba4b4a7a9259eec83a2f1f71" modify_external_catalogs: true
I20260202 02:52:24.342738 1173 tablet_bootstrap.cc:492] T 59990791123c4d4eb620090a1e8e31e1 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:24.344734 1173 tablet_bootstrap.cc:654] T 59990791123c4d4eb620090a1e8e31e1 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:24.348656 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 5ffdf97dff3e4dca89507f488c5b0f8e (DEFAULT_TABLE table=default.Table_18fec323d6b04e10a687de6361b643b2 [id=1ffcc11561524fe8a7b09a7ddc92d5b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.349254 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ffdf97dff3e4dca89507f488c5b0f8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.355168 755 tablet_service.cc:1505] Processing CreateTablet for tablet 5ffdf97dff3e4dca89507f488c5b0f8e (DEFAULT_TABLE table=default.Table_18fec323d6b04e10a687de6361b643b2 [id=1ffcc11561524fe8a7b09a7ddc92d5b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.355667 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ffdf97dff3e4dca89507f488c5b0f8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.356089 1175 tablet_bootstrap.cc:492] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:24.358202 522 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_9181c8b857b540ff846c397f9f1a5484" } modify_external_catalogs: true
I20260202 02:52:24.358433 522 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_9181c8b857b540ff846c397f9f1a5484" } modify_external_catalogs: true
I20260202 02:52:24.361644 884 tablet_service.cc:1505] Processing CreateTablet for tablet 1c8317adc47a4c399f2db650ed3d2e53 (DEFAULT_TABLE table=default.Table_e45bc97d737440b5b6c3ad7bb03d2583 [id=9a80d6a66bae4977b3e3dc572d890865]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.363441 1182 ts_tablet_manager.cc:1950] T 79f694b740954d429145fba9157338ca P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:24.363803 889 tablet_service.cc:1505] Processing CreateTablet for tablet 4e90f273439d414caf16d63e1c605d3d (DEFAULT_TABLE table=default.Table_e11021f60bfe480ab07b2242bb36e94f [id=ae5d5815dcce405a897ffbce9d745591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.364017 1182 tablet_replica.cc:333] T 2b8ba9dd450948ada395207ae1c5b325 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:24.364199 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3338964dfed548f980f58147f0283691. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.367918 893 tablet_service.cc:1505] Processing CreateTablet for tablet a5f864fd8a3d4c4a83e7f68abb8a0a83 (DEFAULT_TABLE table=default.Table_b9aa847125ed4116b6fde1cfc1cafa27 [id=47df1b09bcd24f29b1296f9cf52ac008]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.368376 1933 tablet.cc:1722] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:24.368319 898 tablet_service.cc:1505] Processing CreateTablet for tablet 42321f4e77484027a485fdd584de3ae3 (DEFAULT_TABLE table=default.Table_aee0b3d530954ca3ba6492f8263dd7ff [id=384621c2a9d94b0db7a7928a3bd8fffc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.368865 1865 raft_consensus.cc:493] T 2b8ba9dd450948ada395207ae1c5b325 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:24.369041 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 916f52df8e0a4838b46772ecab4f852c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.369179 894 tablet_service.cc:1505] Processing CreateTablet for tablet e33f95e4f17a479c9acef5764c6c4c56 (DEFAULT_TABLE table=default.Table_ae54fc1a1cfc4475823c8c3b5cf86e5f [id=5c2778a5ed864321bb59e445b4db671a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.372507 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23671408fcbc454cbd918328015cf6e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.372694 1966 tablet.cc:1722] T ef073880898049e2be93d2cb6dc0c0bb P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:24.373098 888 tablet_service.cc:1505] Processing CreateTablet for tablet 5ffdf97dff3e4dca89507f488c5b0f8e (DEFAULT_TABLE table=default.Table_18fec323d6b04e10a687de6361b643b2 [id=1ffcc11561524fe8a7b09a7ddc92d5b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.369052 1865 raft_consensus.cc:515] T 2b8ba9dd450948ada395207ae1c5b325 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.373970 1182 raft_consensus.cc:2243] T 2b8ba9dd450948ada395207ae1c5b325 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:24.374320 1182 raft_consensus.cc:2272] T 2b8ba9dd450948ada395207ae1c5b325 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:24.374408 1865 leader_election.cc:290] T 2b8ba9dd450948ada395207ae1c5b325 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:24.375281 1182 ts_tablet_manager.cc:1916] T 2b8ba9dd450948ada395207ae1c5b325 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:24.376562 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c8317adc47a4c399f2db650ed3d2e53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.376834 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 79f694b740954d429145fba9157338ca (table default.Table_662111850b514df5ad7c69aa21783397 [id=d595db36d702401ab81d5401ad3a76e2]) successfully deleted
I20260202 02:52:24.380043 1175 ts_tablet_manager.cc:1403] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.155s user 0.005s sys 0.000s
I20260202 02:52:24.380752 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42321f4e77484027a485fdd584de3ae3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.381381 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e90f273439d414caf16d63e1c605d3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.381932 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e33f95e4f17a479c9acef5764c6c4c56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.382349 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ffdf97dff3e4dca89507f488c5b0f8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.383435 1175 raft_consensus.cc:359] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.383759 1175 raft_consensus.cc:385] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:24.383877 1175 raft_consensus.cc:740] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:24.384133 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5f864fd8a3d4c4a83e7f68abb8a0a83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.384248 1175 consensus_queue.cc:260] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.385015 1175 ts_tablet_manager.cc:1434] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260202 02:52:24.385280 1174 tablet_bootstrap.cc:492] T 1f54c1dfe0a942b3bafd85522953a032 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:24.392418 1882 ts_tablet_manager.cc:1929] T 723921dfe54c4e66af5cab76a25fee0a P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:24.398314 528 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 587cfd0ae62f4cfd937faf6890e78f4a
I20260202 02:52:24.399835 528 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_a22f252ee8d44be5a9d7202afb726cfc" } modify_external_catalogs: true
I20260202 02:52:24.401856 902 tablet_service.cc:1552] Processing DeleteTablet for tablet 587cfd0ae62f4cfd937faf6890e78f4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:24.402232 528 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_a22f252ee8d44be5a9d7202afb726cfc" } modify_external_catalogs: true
I20260202 02:52:24.404026 1175 tablet_bootstrap.cc:492] T a5622ef925bc46cca76f481ddfd12afd P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:24.409540 742 tablet_service.cc:1552] Processing DeleteTablet for tablet 587cfd0ae62f4cfd937faf6890e78f4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:24.410040 759 tablet_service.cc:1505] Processing CreateTablet for tablet af29c7a2ad9d4797b08052606151a7d5 (DEFAULT_TABLE table=default.Table_6806575803c4468fa7b7ef990004b446 [id=944fbe1acd894017bae3c54f135ee9b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.411379 754 tablet_service.cc:1505] Processing CreateTablet for tablet 1c8317adc47a4c399f2db650ed3d2e53 (DEFAULT_TABLE table=default.Table_e45bc97d737440b5b6c3ad7bb03d2583 [id=9a80d6a66bae4977b3e3dc572d890865]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.419150 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af29c7a2ad9d4797b08052606151a7d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.421291 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_d1dc608853d84069a5d8fdfcc68cc10a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:24.424263 714 consensus_peers.cc:597] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bd7b0a0fcb7645fdb9edfeccb3d18268. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:24.425848 1882 log.cc:1199] T 723921dfe54c4e66af5cab76a25fee0a P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/723921dfe54c4e66af5cab76a25fee0a
I20260202 02:52:24.426672 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c8317adc47a4c399f2db650ed3d2e53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.430631 902 tablet_service.cc:1505] Processing CreateTablet for tablet af29c7a2ad9d4797b08052606151a7d5 (DEFAULT_TABLE table=default.Table_6806575803c4468fa7b7ef990004b446 [id=944fbe1acd894017bae3c54f135ee9b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.431084 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af29c7a2ad9d4797b08052606151a7d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.431936 1932 tablet.cc:1722] T ef073880898049e2be93d2cb6dc0c0bb P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:24.434975 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet 587cfd0ae62f4cfd937faf6890e78f4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:24.437165 1174 tablet_bootstrap.cc:654] T 1f54c1dfe0a942b3bafd85522953a032 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:24.446893 1180 ts_tablet_manager.cc:1929] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:24.447072 1180 log.cc:1199] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/267bd4752c5f4b99a3fe778ba8f38a2f
I20260202 02:52:24.448318 1180 ts_tablet_manager.cc:1950] T 267bd4752c5f4b99a3fe778ba8f38a2f P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:24.449262 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 284) took 4622 ms. Trace:
I20260202 02:52:24.449712 1180 rpcz_store.cc:276] 0202 02:52:19.827158 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.830563 (+ 3405us) service_pool.cc:225] Handling call
0202 02:52:24.158960 (+4328397us) ts_tablet_manager.cc:1151] Deleting tablet 267bd4752c5f4b99a3fe778ba8f38a2f
0202 02:52:24.158978 (+ 18us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:24.347291 (+188313us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:24.446568 (+ 99277us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:24.449224 (+ 2656us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4288908}
I20260202 02:52:24.450649 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 290) took 4539 ms. Trace:
I20260202 02:52:24.450846 1180 rpcz_store.cc:276] 0202 02:52:19.911664 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:19.920225 (+ 8561us) service_pool.cc:225] Handling call
0202 02:52:24.450446 (+4530221us) ts_tablet_manager.cc:1151] Deleting tablet 9c1271ce0f194021b588b82ac9d3db79
0202 02:52:24.450458 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:24.450614 (+ 156us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4529885}
I20260202 02:52:24.451370 1180 tablet_replica.cc:333] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:24.451388 1175 tablet_bootstrap.cc:654] T a5622ef925bc46cca76f481ddfd12afd P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:24.451646 1180 raft_consensus.cc:2243] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:24.452138 1180 raft_consensus.cc:2272] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:24.460554 1182 ts_tablet_manager.cc:1929] T 2b8ba9dd450948ada395207ae1c5b325 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:24.467996 1182 log.cc:1199] T 2b8ba9dd450948ada395207ae1c5b325 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/2b8ba9dd450948ada395207ae1c5b325
I20260202 02:52:24.468782 1182 ts_tablet_manager.cc:1950] T 2b8ba9dd450948ada395207ae1c5b325 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:24.469197 1180 ts_tablet_manager.cc:1916] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:24.474820 1182 tablet_replica.cc:333] T d21fca9a5b234c26879717f51c8ea0f0 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:24.475131 1182 raft_consensus.cc:2243] T d21fca9a5b234c26879717f51c8ea0f0 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:24.475518 1182 raft_consensus.cc:2272] T d21fca9a5b234c26879717f51c8ea0f0 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:24.477255 1175 tablet_bootstrap.cc:492] T a5622ef925bc46cca76f481ddfd12afd P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:24.478174 1175 ts_tablet_manager.cc:1403] T a5622ef925bc46cca76f481ddfd12afd P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.077s user 0.005s sys 0.000s
I20260202 02:52:24.479022 1175 raft_consensus.cc:359] T a5622ef925bc46cca76f481ddfd12afd P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:24.479219 1175 raft_consensus.cc:385] T a5622ef925bc46cca76f481ddfd12afd P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:24.479315 1175 raft_consensus.cc:740] T a5622ef925bc46cca76f481ddfd12afd P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:24.479549 1175 consensus_queue.cc:260] T a5622ef925bc46cca76f481ddfd12afd P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:24.479696 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 1c8317adc47a4c399f2db650ed3d2e53 (DEFAULT_TABLE table=default.Table_e45bc97d737440b5b6c3ad7bb03d2583 [id=9a80d6a66bae4977b3e3dc572d890865]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.480175 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c8317adc47a4c399f2db650ed3d2e53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.479665 1182 ts_tablet_manager.cc:1916] T d21fca9a5b234c26879717f51c8ea0f0 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:24.484513 1175 ts_tablet_manager.cc:1434] T a5622ef925bc46cca76f481ddfd12afd P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260202 02:52:24.485194 1175 tablet_bootstrap.cc:492] T 68823eff1bc04108ba83487605aec0ea P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:24.485947 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2b8ba9dd450948ada395207ae1c5b325" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:24.486845 859 leader_election.cc:343] T 2b8ba9dd450948ada395207ae1c5b325 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:24.492007 1175 tablet_bootstrap.cc:654] T 68823eff1bc04108ba83487605aec0ea P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:24.495110 1182 ts_tablet_manager.cc:1929] T d21fca9a5b234c26879717f51c8ea0f0 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:24.498025 1173 tablet_bootstrap.cc:492] T 59990791123c4d4eb620090a1e8e31e1 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:24.498255 1182 log.cc:1199] T d21fca9a5b234c26879717f51c8ea0f0 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/d21fca9a5b234c26879717f51c8ea0f0
I20260202 02:52:24.498759 1182 ts_tablet_manager.cc:1950] T d21fca9a5b234c26879717f51c8ea0f0 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:24.498229 1173 ts_tablet_manager.cc:1403] T 59990791123c4d4eb620090a1e8e31e1 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.156s user 0.009s sys 0.007s
I20260202 02:52:24.500448 1764 consensus_queue.cc:1048] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:24.501751 1882 ts_tablet_manager.cc:1950] T 723921dfe54c4e66af5cab76a25fee0a P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:24.513965 1175 tablet_bootstrap.cc:492] T 68823eff1bc04108ba83487605aec0ea P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:24.514158 1175 ts_tablet_manager.cc:1403] T 68823eff1bc04108ba83487605aec0ea P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.029s user 0.006s sys 0.000s
I20260202 02:52:24.514951 1175 raft_consensus.cc:359] T 68823eff1bc04108ba83487605aec0ea P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.515362 1175 raft_consensus.cc:385] T 68823eff1bc04108ba83487605aec0ea P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:24.515779 1175 raft_consensus.cc:740] T 68823eff1bc04108ba83487605aec0ea P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:24.516116 1175 consensus_queue.cc:260] T 68823eff1bc04108ba83487605aec0ea P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.517016 1175 ts_tablet_manager.cc:1434] T 68823eff1bc04108ba83487605aec0ea P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260202 02:52:24.517377 1175 tablet_bootstrap.cc:492] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:24.519155 1175 tablet_bootstrap.cc:654] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:24.519982 1865 raft_consensus.cc:493] T 68823eff1bc04108ba83487605aec0ea P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:24.520175 1865 raft_consensus.cc:515] T 68823eff1bc04108ba83487605aec0ea P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.520220 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2b8ba9dd450948ada395207ae1c5b325" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:24.520741 857 leader_election.cc:343] T 2b8ba9dd450948ada395207ae1c5b325 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:24.520895 857 leader_election.cc:304] T 2b8ba9dd450948ada395207ae1c5b325 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:24.520902 1865 leader_election.cc:290] T 68823eff1bc04108ba83487605aec0ea P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:24.521114 857 raft_consensus.cc:2686] T 2b8ba9dd450948ada395207ae1c5b325 P 1521a6d262b7451bbef8debe970dd6b5: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260202 02:52:24.522739 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "68823eff1bc04108ba83487605aec0ea" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:24.523350 857 leader_election.cc:343] T 68823eff1bc04108ba83487605aec0ea P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:24.523778 1174 tablet_bootstrap.cc:492] T 1f54c1dfe0a942b3bafd85522953a032 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:24.523959 1174 ts_tablet_manager.cc:1403] T 1f54c1dfe0a942b3bafd85522953a032 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.161s user 0.006s sys 0.000s
I20260202 02:52:24.525058 1174 raft_consensus.cc:359] T 1f54c1dfe0a942b3bafd85522953a032 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.525264 1174 raft_consensus.cc:385] T 1f54c1dfe0a942b3bafd85522953a032 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:24.525362 1174 raft_consensus.cc:740] T 1f54c1dfe0a942b3bafd85522953a032 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:24.525583 1174 consensus_queue.cc:260] T 1f54c1dfe0a942b3bafd85522953a032 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.526103 1174 ts_tablet_manager.cc:1434] T 1f54c1dfe0a942b3bafd85522953a032 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260202 02:52:24.535218 772 raft_consensus.cc:3060] T 1f54c1dfe0a942b3bafd85522953a032 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:24.535434 1174 tablet_bootstrap.cc:492] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:24.537526 1174 tablet_bootstrap.cc:654] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:24.545223 1175 tablet_bootstrap.cc:492] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
W20260202 02:52:24.544610 715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 182509us
I20260202 02:52:24.545697 1175 ts_tablet_manager.cc:1403] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.028s user 0.004s sys 0.000s
I20260202 02:52:24.546613 1175 raft_consensus.cc:359] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.546944 1175 raft_consensus.cc:385] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:24.547084 1175 raft_consensus.cc:740] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:24.547390 1175 consensus_queue.cc:260] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.548039 1175 ts_tablet_manager.cc:1434] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:24.549580 1175 tablet_bootstrap.cc:492] T 9c1271ce0f194021b588b82ac9d3db79 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:24.512912 1173 raft_consensus.cc:359] T 59990791123c4d4eb620090a1e8e31e1 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.552829 1173 raft_consensus.cc:385] T 59990791123c4d4eb620090a1e8e31e1 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:24.552942 1173 raft_consensus.cc:740] T 59990791123c4d4eb620090a1e8e31e1 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:24.553211 1173 consensus_queue.cc:260] T 59990791123c4d4eb620090a1e8e31e1 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.553421 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 354) took 1342 ms. Trace:
I20260202 02:52:24.553561 1882 rpcz_store.cc:276] 0202 02:52:23.211147 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.211411 (+ 264us) service_pool.cc:225] Handling call
0202 02:52:24.311415 (+1100004us) ts_tablet_manager.cc:1151] Deleting tablet 723921dfe54c4e66af5cab76a25fee0a
0202 02:52:24.311423 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:24.349234 (+ 37811us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:24.368265 (+ 19031us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:24.553365 (+185100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1091411}
I20260202 02:52:24.553752 1173 ts_tablet_manager.cc:1434] T 59990791123c4d4eb620090a1e8e31e1 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.054s user 0.001s sys 0.001s
I20260202 02:52:24.522001 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "68823eff1bc04108ba83487605aec0ea" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:24.538924 1721 consensus_queue.cc:1048] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:24.539453 1860 consensus_queue.cc:1048] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:24.555879 1882 tablet_replica.cc:333] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:24.556221 1882 raft_consensus.cc:2243] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:24.556470 1173 tablet_bootstrap.cc:492] T 2f0e8c5e2fd44370ae150a206f4e6cbc P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:24.556842 1882 raft_consensus.cc:2272] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:24.557010 859 leader_election.cc:343] T 68823eff1bc04108ba83487605aec0ea P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:24.558039 1882 ts_tablet_manager.cc:1916] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:24.560525 1173 tablet_bootstrap.cc:654] T 2f0e8c5e2fd44370ae150a206f4e6cbc P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:24.560451 859 leader_election.cc:304] T 68823eff1bc04108ba83487605aec0ea P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
W20260202 02:52:24.560824 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d1dc608853d84069a5d8fdfcc68cc10a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:24.561034 1865 raft_consensus.cc:2749] T 68823eff1bc04108ba83487605aec0ea P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:24.562072 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 267bd4752c5f4b99a3fe778ba8f38a2f (table default.Table_dda472e63fac4873ab3ccefcac3635cd [id=40105de1d8074d3e983ff62557d14f48]) successfully deleted
I20260202 02:52:24.539907 1764 consensus_queue.cc:1048] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:24.567184 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 723921dfe54c4e66af5cab76a25fee0a (table default.Table_bdfbf5f796b949e196b1c1609fbb3655 [id=c30593053ed7490588ff567e30984807]) successfully deleted
I20260202 02:52:24.571272 1175 tablet_bootstrap.cc:654] T 9c1271ce0f194021b588b82ac9d3db79 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:24.572635 514 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 99036.5us
W20260202 02:52:24.572811 999 consensus_peers.cc:597] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: eaa47f3388144e8b852804c167f232ce. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:24.574277 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 2b8ba9dd450948ada395207ae1c5b325 (table default.Table_b88bd2e65dd140d5a1c208c9d4aa9430 [id=29f81d15b06d4f8abe2aba83493cfa6a]) successfully deleted
I20260202 02:52:24.574613 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet d21fca9a5b234c26879717f51c8ea0f0 (table default.Table_48faa250b04a4b61a37f532584081968 [id=a77fa0c30d27405294adaef676bc76f7]) successfully deleted
W20260202 02:52:24.574797 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet dc4c3f52f4f44a84b0d0e3c9a4a49f09 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dc4c3f52f4f44a84b0d0e3c9a4a49f09 already in progress: creating tablet
W20260202 02:52:24.574954 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet da5435f969074364bc8f8a6ddc5e4f8a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet da5435f969074364bc8f8a6ddc5e4f8a already in progress: creating tablet
W20260202 02:52:24.575106 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet c9ef08b5830147a3a55049285a40535b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c9ef08b5830147a3a55049285a40535b already in progress: creating tablet
W20260202 02:52:24.575254 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 587cfd0ae62f4cfd937faf6890e78f4a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 587cfd0ae62f4cfd937faf6890e78f4a already in progress: creating tablet
W20260202 02:52:24.575706 513 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 111871us
I20260202 02:52:24.581601 1173 tablet_bootstrap.cc:492] T 2f0e8c5e2fd44370ae150a206f4e6cbc P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:24.581782 1173 ts_tablet_manager.cc:1403] T 2f0e8c5e2fd44370ae150a206f4e6cbc P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.025s user 0.003s sys 0.004s
I20260202 02:52:24.582543 1764 raft_consensus.cc:493] T 6154460838eb4e508f3646d3e3e4cf21 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:24.582731 1764 raft_consensus.cc:515] T 6154460838eb4e508f3646d3e3e4cf21 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.583530 1173 raft_consensus.cc:359] T 2f0e8c5e2fd44370ae150a206f4e6cbc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.583752 1764 leader_election.cc:290] T 6154460838eb4e508f3646d3e3e4cf21 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:24.583972 1173 raft_consensus.cc:385] T 2f0e8c5e2fd44370ae150a206f4e6cbc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:24.584087 1173 raft_consensus.cc:740] T 2f0e8c5e2fd44370ae150a206f4e6cbc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
W20260202 02:52:24.584393 999 leader_election.cc:343] T 6154460838eb4e508f3646d3e3e4cf21 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 6154460838eb4e508f3646d3e3e4cf21
I20260202 02:52:24.584491 1173 consensus_queue.cc:260] T 2f0e8c5e2fd44370ae150a206f4e6cbc P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:24.585224 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 9c1271ce0f194021b588b82ac9d3db79 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9c1271ce0f194021b588b82ac9d3db79 already in progress: creating tablet
I20260202 02:52:24.586704 1025 tablet_service.cc:1505] Processing CreateTablet for tablet af29c7a2ad9d4797b08052606151a7d5 (DEFAULT_TABLE table=default.Table_6806575803c4468fa7b7ef990004b446 [id=944fbe1acd894017bae3c54f135ee9b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.587180 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af29c7a2ad9d4797b08052606151a7d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.591459 1175 tablet_bootstrap.cc:492] T 9c1271ce0f194021b588b82ac9d3db79 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:24.591645 1175 ts_tablet_manager.cc:1403] T 9c1271ce0f194021b588b82ac9d3db79 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.042s user 0.006s sys 0.004s
I20260202 02:52:24.592574 525 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 75569511179b41038e521fa3346a92ae
I20260202 02:52:24.592502 1175 raft_consensus.cc:359] T 9c1271ce0f194021b588b82ac9d3db79 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.592701 1175 raft_consensus.cc:385] T 9c1271ce0f194021b588b82ac9d3db79 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:24.592777 1175 raft_consensus.cc:740] T 9c1271ce0f194021b588b82ac9d3db79 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:24.593036 1175 consensus_queue.cc:260] T 9c1271ce0f194021b588b82ac9d3db79 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.593364 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_6596d734c9de42ccbf1af1b077742369"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:24.593542 1175 ts_tablet_manager.cc:1434] T 9c1271ce0f194021b588b82ac9d3db79 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:24.593963 1175 tablet_bootstrap.cc:492] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
W20260202 02:52:24.593962 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6596d734c9de42ccbf1af1b077742369 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:24.595882 519 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet ef073880898049e2be93d2cb6dc0c0bb (table default.Table_e8e9b4cdaf434b6cbbfed1a65bff1fc0 [id=78a7fb3a2eed4565b55d5dacb8a3d7df]). Expected version 1 got 0
I20260202 02:52:24.597170 1175 tablet_bootstrap.cc:654] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:24.597937 519 master_service.cc:954] ReplaceTablet: received request to replace tablet a1f39e5b02f747819e636cddb4292eb6 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:24.604601 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 75569511179b41038e521fa3346a92ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:24.604429 526 catalog_manager.cc:5654] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d reported cstate change: term changed from 0 to 1, leader changed from <none> to b1f05820b6034228bdad884cc348716d (127.0.104.131). New cstate: current_term: 1 leader_uuid: "b1f05820b6034228bdad884cc348716d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } }
I20260202 02:52:24.604694 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet 75569511179b41038e521fa3346a92ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:24.605024 526 catalog_manager.cc:5654] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d reported cstate change: term changed from 0 to 1, leader changed from <none> to b1f05820b6034228bdad884cc348716d (127.0.104.131). New cstate: current_term: 1 leader_uuid: "b1f05820b6034228bdad884cc348716d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } }
W20260202 02:52:24.606233 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 75569511179b41038e521fa3346a92ae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 75569511179b41038e521fa3346a92ae already in progress: creating tablet
I20260202 02:52:24.610173 1173 ts_tablet_manager.cc:1434] T 2f0e8c5e2fd44370ae150a206f4e6cbc P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.028s user 0.002s sys 0.000s
I20260202 02:52:24.610553 1173 tablet_bootstrap.cc:492] T 91ec01d25f8f441cbe3783cacbee51db P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:24.612064 522 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet fe9ea4c0502f4976b440465992f2ea2c
W20260202 02:52:24.612310 527 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b34dc1cab5874fa6aee55d79c6e09ade has no consensus state
I20260202 02:52:24.612084 521 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet a1f39e5b02f747819e636cddb4292eb6
I20260202 02:52:24.613736 521 master_service.cc:954] ReplaceTablet: received request to replace tablet 042d1ebf357b4abe8814a407891ce265 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:24.614387 888 tablet_service.cc:1552] Processing DeleteTablet for tablet fe9ea4c0502f4976b440465992f2ea2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:24.614518 1037 tablet_service.cc:1552] Processing DeleteTablet for tablet a1f39e5b02f747819e636cddb4292eb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:24.616032 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet fe9ea4c0502f4976b440465992f2ea2c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fe9ea4c0502f4976b440465992f2ea2c already in progress: creating tablet
I20260202 02:52:24.617077 1175 tablet_bootstrap.cc:492] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:24.617285 1175 ts_tablet_manager.cc:1403] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.002s
I20260202 02:52:24.617453 888 tablet_service.cc:1552] Processing DeleteTablet for tablet a1f39e5b02f747819e636cddb4292eb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:24.617412 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 3706562a22634eb6b72e36bb6a2d1ae5 (DEFAULT_TABLE table=default.Table_1cc0209a40ad471fb1c41012a476df79 [id=b941bc2edc03444394d1ba8a596a0d27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.617867 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3706562a22634eb6b72e36bb6a2d1ae5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.618193 1175 raft_consensus.cc:359] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.618368 1175 raft_consensus.cc:385] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:24.618446 1175 raft_consensus.cc:740] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
W20260202 02:52:24.620927 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet a1f39e5b02f747819e636cddb4292eb6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a1f39e5b02f747819e636cddb4292eb6 already in progress: creating tablet
I20260202 02:52:24.622069 888 tablet_service.cc:1505] Processing CreateTablet for tablet 3706562a22634eb6b72e36bb6a2d1ae5 (DEFAULT_TABLE table=default.Table_1cc0209a40ad471fb1c41012a476df79 [id=b941bc2edc03444394d1ba8a596a0d27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.622519 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3706562a22634eb6b72e36bb6a2d1ae5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.623958 1038 tablet_service.cc:1505] Processing CreateTablet for tablet 3857680ccbdc4dd4ad8247da96254434 (DEFAULT_TABLE table=default.Table_a5d6ab55014a4c068dceeb6a781bf3b1 [id=d20907a677b742ff8a793c13a1faf28d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.624387 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3857680ccbdc4dd4ad8247da96254434. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.625399 1764 raft_consensus.cc:493] T 2f0e8c5e2fd44370ae150a206f4e6cbc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:24.625572 1764 raft_consensus.cc:515] T 2f0e8c5e2fd44370ae150a206f4e6cbc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.626369 1764 leader_election.cc:290] T 2f0e8c5e2fd44370ae150a206f4e6cbc P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:24.626900 902 tablet_service.cc:1505] Processing CreateTablet for tablet 3857680ccbdc4dd4ad8247da96254434 (DEFAULT_TABLE table=default.Table_a5d6ab55014a4c068dceeb6a781bf3b1 [id=d20907a677b742ff8a793c13a1faf28d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.627333 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3857680ccbdc4dd4ad8247da96254434. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.627332 1882 ts_tablet_manager.cc:1929] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:24.627472 1882 log.cc:1199] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/267bd4752c5f4b99a3fe778ba8f38a2f
I20260202 02:52:24.627861 1882 ts_tablet_manager.cc:1950] T 267bd4752c5f4b99a3fe778ba8f38a2f P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:24.623958 1173 tablet_bootstrap.cc:654] T 91ec01d25f8f441cbe3783cacbee51db P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:24.630416 889 tablet_service.cc:1505] Processing CreateTablet for tablet 361c0c3832614ec78695c28720b841e0 (DEFAULT_TABLE table=default.Table_b9ea6590fd0b48608fc1ebe68e7eae9b [id=1bdd648e20a04bb688d744b934563ff7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.630882 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 361c0c3832614ec78695c28720b841e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.632414 1865 raft_consensus.cc:493] T af5aa68e47bb420eac300754d274fa73 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:24.632609 1865 raft_consensus.cc:515] T af5aa68e47bb420eac300754d274fa73 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.633306 1865 leader_election.cc:290] T af5aa68e47bb420eac300754d274fa73 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:24.634176 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "af5aa68e47bb420eac300754d274fa73" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:24.635272 859 leader_election.cc:343] T af5aa68e47bb420eac300754d274fa73 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:24.635284 1041 tablet_service.cc:1505] Processing CreateTablet for tablet 361c0c3832614ec78695c28720b841e0 (DEFAULT_TABLE table=default.Table_b9ea6590fd0b48608fc1ebe68e7eae9b [id=1bdd648e20a04bb688d744b934563ff7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.635748 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 361c0c3832614ec78695c28720b841e0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:24.637032 999 leader_election.cc:343] T 2f0e8c5e2fd44370ae150a206f4e6cbc P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 2f0e8c5e2fd44370ae150a206f4e6cbc
I20260202 02:52:24.640911 1042 tablet_service.cc:1505] Processing CreateTablet for tablet 745fc215251e4ccb9a2abed02573dd84 (DEFAULT_TABLE table=default.Table_ed8a8cefa5de4d90a2d57043595329ae [id=49c21c897521434c922d18a65b4e2c45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.641324 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 745fc215251e4ccb9a2abed02573dd84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.644685 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 1d98c7a46139413b8f0d3697b74b1e52 (DEFAULT_TABLE table=default.Table_38e8a46dd2dd498baa45935fa4670b09 [id=97791994f92a47519b28e44ec02df5ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.645130 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d98c7a46139413b8f0d3697b74b1e52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.646018 1175 consensus_queue.cc:260] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.648684 1175 ts_tablet_manager.cc:1434] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.031s user 0.001s sys 0.001s
I20260202 02:52:24.649044 1175 tablet_bootstrap.cc:492] T 8774c6bc0fe9414ea2f17ed859d9392a P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:24.654189 1175 tablet_bootstrap.cc:654] T 8774c6bc0fe9414ea2f17ed859d9392a P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:24.656966 889 tablet_service.cc:1505] Processing CreateTablet for tablet 745fc215251e4ccb9a2abed02573dd84 (DEFAULT_TABLE table=default.Table_ed8a8cefa5de4d90a2d57043595329ae [id=49c21c897521434c922d18a65b4e2c45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.658058 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 745fc215251e4ccb9a2abed02573dd84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.657593 902 tablet_service.cc:1505] Processing CreateTablet for tablet 1d98c7a46139413b8f0d3697b74b1e52 (DEFAULT_TABLE table=default.Table_38e8a46dd2dd498baa45935fa4670b09 [id=97791994f92a47519b28e44ec02df5ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.661021 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d98c7a46139413b8f0d3697b74b1e52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.679297 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 75569511179b41038e521fa3346a92ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:24.681370 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f0e8c5e2fd44370ae150a206f4e6cbc" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:24.681486 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet fe9ea4c0502f4976b440465992f2ea2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:24.682348 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "af5aa68e47bb420eac300754d274fa73" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:24.683070 857 leader_election.cc:343] T af5aa68e47bb420eac300754d274fa73 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:24.683252 857 leader_election.cc:304] T af5aa68e47bb420eac300754d274fa73 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:24.683612 1865 raft_consensus.cc:2749] T af5aa68e47bb420eac300754d274fa73 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:24.679611 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6154460838eb4e508f3646d3e3e4cf21" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:24.685878 1277 raft_consensus.cc:493] T ffd80badce1048f3ac95366cad05ae53 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:24.686079 1277 raft_consensus.cc:515] T ffd80badce1048f3ac95366cad05ae53 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.687345 528 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 1f54c1dfe0a942b3bafd85522953a032
I20260202 02:52:24.687773 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ffd80badce1048f3ac95366cad05ae53" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:24.688035 1060 raft_consensus.cc:2393] T ffd80badce1048f3ac95366cad05ae53 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f442ef85029461091de847118c34c0d in current term 1: Already voted for candidate b1f05820b6034228bdad884cc348716d in this term.
I20260202 02:52:24.688082 528 master_service.cc:954] ReplaceTablet: received request to replace tablet c25e329023c04f609d8a743b4f7a95a4 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:24.688971 1277 leader_election.cc:290] T ffd80badce1048f3ac95366cad05ae53 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:24.689139 759 tablet_service.cc:1552] Processing DeleteTablet for tablet fe9ea4c0502f4976b440465992f2ea2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:24.689309 754 tablet_service.cc:1552] Processing DeleteTablet for tablet ffd80badce1048f3ac95366cad05ae53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:24.690393 519 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_17dd200ab5824a4a90f05d78d31c6887" } new_table_name: "default.Table_509184aa45184b7c8228aba2de1c528a" modify_external_catalogs: true
I20260202 02:52:24.691144 747 tablet_service.cc:1505] Processing CreateTablet for tablet 361c0c3832614ec78695c28720b841e0 (DEFAULT_TABLE table=default.Table_b9ea6590fd0b48608fc1ebe68e7eae9b [id=1bdd648e20a04bb688d744b934563ff7]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:24.691577 998 leader_election.cc:343] T 2f0e8c5e2fd44370ae150a206f4e6cbc P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:24.691709 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 361c0c3832614ec78695c28720b841e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.691713 998 leader_election.cc:304] T 2f0e8c5e2fd44370ae150a206f4e6cbc P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
W20260202 02:52:24.692191 998 leader_election.cc:343] T 6154460838eb4e508f3646d3e3e4cf21 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:24.692293 998 leader_election.cc:304] T 6154460838eb4e508f3646d3e3e4cf21 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:24.692303 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_6a546caebf9e4075816236ca1bd8d73f" } new_table_name: "default.Table_c9a1147d25bd49e48b70b58b373acfc7" modify_external_catalogs: true
I20260202 02:52:24.692720 1860 raft_consensus.cc:2749] T 2f0e8c5e2fd44370ae150a206f4e6cbc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:24.692936 1860 raft_consensus.cc:2749] T 6154460838eb4e508f3646d3e3e4cf21 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:24.692947 902 tablet_service.cc:1552] Processing DeleteTablet for tablet 1f54c1dfe0a942b3bafd85522953a032 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:24.693259 1182 tablet_replica.cc:333] T 1f54c1dfe0a942b3bafd85522953a032 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:24.693491 1182 raft_consensus.cc:2243] T 1f54c1dfe0a942b3bafd85522953a032 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:24.693662 1182 raft_consensus.cc:2272] T 1f54c1dfe0a942b3bafd85522953a032 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:24.693697 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_72656c0c3e7f4157b3c57bcd71d097b7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:24.694278 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_72656c0c3e7f4157b3c57bcd71d097b7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:24.695421 1182 ts_tablet_manager.cc:1916] T 1f54c1dfe0a942b3bafd85522953a032 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:24.695483 523 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_bb9eeb45b01e4ad3b5a6bc6ef8ab60e9" } new_table_name: "default.Table_7913f8fb6eac487a971545a11cf257a9" modify_external_catalogs: true
I20260202 02:52:24.698377 524 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_6132c5ab92374585bde17a139c7cc390" } new_table_name: "default.Table_d6ba351d68ec4a10939a09519b7d4b67" modify_external_catalogs: true
I20260202 02:52:24.706023 751 tablet_service.cc:1505] Processing CreateTablet for tablet 3857680ccbdc4dd4ad8247da96254434 (DEFAULT_TABLE table=default.Table_a5d6ab55014a4c068dceeb6a781bf3b1 [id=d20907a677b742ff8a793c13a1faf28d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.708202 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet 1f54c1dfe0a942b3bafd85522953a032 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:24.708285 754 tablet_service.cc:1552] Processing DeleteTablet for tablet a1f39e5b02f747819e636cddb4292eb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:24.708616 755 tablet_service.cc:1505] Processing CreateTablet for tablet 745fc215251e4ccb9a2abed02573dd84 (DEFAULT_TABLE table=default.Table_ed8a8cefa5de4d90a2d57043595329ae [id=49c21c897521434c922d18a65b4e2c45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.709563 525 master_service.cc:954] ReplaceTablet: received request to replace tablet cd1b6662da254889ae4bbe8d0b1ef1bb from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:24.710757 520 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_718e4de3a182443fb16b265396c79401 [id=f7c5b24c065443d89a2b1f9c34409c9d] alter complete (version 1)
I20260202 02:52:24.710867 1175 tablet_bootstrap.cc:492] T 8774c6bc0fe9414ea2f17ed859d9392a P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:24.711066 1175 ts_tablet_manager.cc:1403] T 8774c6bc0fe9414ea2f17ed859d9392a P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.062s user 0.007s sys 0.003s
I20260202 02:52:24.712074 1175 raft_consensus.cc:359] T 8774c6bc0fe9414ea2f17ed859d9392a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.712311 1175 raft_consensus.cc:385] T 8774c6bc0fe9414ea2f17ed859d9392a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:24.712646 1175 raft_consensus.cc:740] T 8774c6bc0fe9414ea2f17ed859d9392a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:24.712915 1175 consensus_queue.cc:260] T 8774c6bc0fe9414ea2f17ed859d9392a P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.713821 1175 ts_tablet_manager.cc:1434] T 8774c6bc0fe9414ea2f17ed859d9392a P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260202 02:52:24.714387 1175 tablet_bootstrap.cc:492] T a1f39e5b02f747819e636cddb4292eb6 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:24.714687 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 2b8ba9dd450948ada395207ae1c5b325 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:24.715394 902 tablet_service.cc:1552] Processing DeleteTablet for tablet 38fdbc6df7ec40b9afe4e0dfe0119e3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:24.716454 742 tablet_service.cc:1505] Processing CreateTablet for tablet 1d98c7a46139413b8f0d3697b74b1e52 (DEFAULT_TABLE table=default.Table_38e8a46dd2dd498baa45935fa4670b09 [id=97791994f92a47519b28e44ec02df5ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.717810 1180 ts_tablet_manager.cc:1929] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:24.719580 1175 tablet_bootstrap.cc:654] T a1f39e5b02f747819e636cddb4292eb6 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:24.720511 759 tablet_service.cc:1505] Processing CreateTablet for tablet 3706562a22634eb6b72e36bb6a2d1ae5 (DEFAULT_TABLE table=default.Table_1cc0209a40ad471fb1c41012a476df79 [id=b941bc2edc03444394d1ba8a596a0d27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:24.721047 1180 log.cc:1199] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/b428356fba0c4fb5a6033c5d39740cc5
I20260202 02:52:24.721166 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3857680ccbdc4dd4ad8247da96254434. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.721529 1180 ts_tablet_manager.cc:1950] T b428356fba0c4fb5a6033c5d39740cc5 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:24.721899 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d98c7a46139413b8f0d3697b74b1e52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.722404 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 297) took 4535 ms. Trace:
I20260202 02:52:24.722518 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 3711910e050c4515bed4294d1271d5a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:24.722518 1180 rpcz_store.cc:276] 0202 02:52:20.187003 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:20.187063 (+ 60us) service_pool.cc:225] Handling call
0202 02:52:24.451302 (+4264239us) ts_tablet_manager.cc:1151] Deleting tablet b428356fba0c4fb5a6033c5d39740cc5
0202 02:52:24.451312 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:24.475489 (+ 24177us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:24.717676 (+242187us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:24.722345 (+ 4669us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4254883}
I20260202 02:52:24.722883 1180 tablet_replica.cc:333] T 583e547d0a1943f09f4efe82070fc777 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:24.723003 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 745fc215251e4ccb9a2abed02573dd84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.723006 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet 6154460838eb4e508f3646d3e3e4cf21 with delete_type TABLET_DATA_DELETED (replaced by tablet 5e20842de8cb4355b732685605c88834) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:24.723110 1180 raft_consensus.cc:2243] T 583e547d0a1943f09f4efe82070fc777 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:24.723313 1180 raft_consensus.cc:2272] T 583e547d0a1943f09f4efe82070fc777 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:24.723853 1180 ts_tablet_manager.cc:1916] T 583e547d0a1943f09f4efe82070fc777 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:24.724220 1972 raft_consensus.cc:493] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:24.724431 1972 raft_consensus.cc:515] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:24.724619 519 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9f11a68d10374d06bc4a756a1d15bf09 consensus state has no leader
I20260202 02:52:24.726909 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet b428356fba0c4fb5a6033c5d39740cc5 (table default.Table_c37ce6d5cf70417c83a68c31d02881ef [id=e87761e6b64a4476a05fcbcf73f0a2f1]) successfully deleted
W20260202 02:52:24.727382 527 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 550895b4010f4088b2cc34acff50e03f consensus state has no leader
I20260202 02:52:24.727705 527 master_service.cc:954] ReplaceTablet: received request to replace tablet 1f379a29e2454f689d68f5951837512b from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:24.727892 1182 ts_tablet_manager.cc:1929] T 1f54c1dfe0a942b3bafd85522953a032 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:24.728041 1182 log.cc:1199] T 1f54c1dfe0a942b3bafd85522953a032 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/1f54c1dfe0a942b3bafd85522953a032
I20260202 02:52:24.728446 1182 ts_tablet_manager.cc:1950] T 1f54c1dfe0a942b3bafd85522953a032 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:24.730273 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 1f54c1dfe0a942b3bafd85522953a032 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:24.730448 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3706562a22634eb6b72e36bb6a2d1ae5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:24.730818 526 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 91ec01d25f8f441cbe3783cacbee51db (table default.Table_882b22c2c2b0446c83239be15da80b95 [id=255f4d94812f45d0b4423150098460ff]). Expected version 1 got 0
I20260202 02:52:24.731037 526 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 28a900bcaa8b41718a285e595886d805 (table default.Table_911bb209b4554ee8b9bc09edaf488d63 [id=cd07f9746ec94d9ebe956a7ca0d19b1c]). Expected version 1 got 0
W20260202 02:52:24.731343 714 leader_election.cc:343] T ffd80badce1048f3ac95366cad05ae53 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: ffd80badce1048f3ac95366cad05ae53
I20260202 02:52:24.731470 714 leader_election.cc:304] T ffd80badce1048f3ac95366cad05ae53 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:24.731894 1277 raft_consensus.cc:3060] T ffd80badce1048f3ac95366cad05ae53 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:24.733803 1969 consensus_queue.cc:1048] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260202 02:52:24.735858 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_82c4120b31ef4f7fbb290c51dc0f7866"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:24.734649 1182 tablet_replica.cc:333] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
W20260202 02:52:24.736579 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_82c4120b31ef4f7fbb290c51dc0f7866 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:24.736718 1182 raft_consensus.cc:2243] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:24.736925 1182 raft_consensus.cc:2272] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:24.737514 1182 ts_tablet_manager.cc:1916] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:24.738678 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_4ba0da0467a14feb8978f827568eb41f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:24.739349 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4ba0da0467a14feb8978f827568eb41f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:24.741020 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 1f54c1dfe0a942b3bafd85522953a032 (table default.Table_a22f252ee8d44be5a9d7202afb726cfc [id=891d9355855640a28939b84b2dd30537]) successfully deleted
W20260202 02:52:24.741546 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 2b8ba9dd450948ada395207ae1c5b325 because the tablet was not found. No further retry: Not found: Tablet not found: 2b8ba9dd450948ada395207ae1c5b325
I20260202 02:52:24.744252 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "929eb48881b141718873d0e359e768a8" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:24.744547 1060 raft_consensus.cc:2468] T 929eb48881b141718873d0e359e768a8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:24.756410 1175 tablet_bootstrap.cc:492] T a1f39e5b02f747819e636cddb4292eb6 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
W20260202 02:52:24.756563 714 leader_election.cc:343] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 929eb48881b141718873d0e359e768a8
I20260202 02:52:24.758163 1175 ts_tablet_manager.cc:1403] T a1f39e5b02f747819e636cddb4292eb6 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.044s user 0.002s sys 0.005s
I20260202 02:52:24.759253 715 leader_election.cc:304] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:24.756599 1972 leader_election.cc:290] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:24.760987 1865 raft_consensus.cc:493] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:24.761176 1865 raft_consensus.cc:515] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.761461 1175 raft_consensus.cc:359] T a1f39e5b02f747819e636cddb4292eb6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.761686 1175 raft_consensus.cc:385] T a1f39e5b02f747819e636cddb4292eb6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:24.761869 1865 leader_election.cc:290] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:24.761790 1175 raft_consensus.cc:740] T a1f39e5b02f747819e636cddb4292eb6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:24.762449 1175 consensus_queue.cc:260] T a1f39e5b02f747819e636cddb4292eb6 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.762866 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28a900bcaa8b41718a285e595886d805" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:24.766693 1850 raft_consensus.cc:2804] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:24.766865 1850 raft_consensus.cc:493] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:24.766968 1850 raft_consensus.cc:3060] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:24.768043 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 347) took 1568 ms. Trace:
I20260202 02:52:24.769333 1882 rpcz_store.cc:276] 0202 02:52:23.199760 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.199814 (+ 54us) service_pool.cc:225] Handling call
0202 02:52:24.555806 (+1355992us) ts_tablet_manager.cc:1151] Deleting tablet 267bd4752c5f4b99a3fe778ba8f38a2f
0202 02:52:24.555818 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:24.621867 (+ 66049us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:24.627206 (+ 5339us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:24.767977 (+140771us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1331111}
I20260202 02:52:24.771944 1972 consensus_queue.cc:1048] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260202 02:52:24.775031 521 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 042d1ebf357b4abe8814a407891ce265
I20260202 02:52:24.775619 521 catalog_manager.cc:6724] ReplaceTablet: tablet 042d1ebf357b4abe8814a407891ce265 deleted and replaced by tablet 2c2a535de2294cc697d75561d0463642
I20260202 02:52:24.775648 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28a900bcaa8b41718a285e595886d805" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:24.776705 777 raft_consensus.cc:3060] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:24.777946 524 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1116872cc25a4ab5b1a3a6bbba1f0da7 consensus state has no leader
I20260202 02:52:24.779201 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_39f7c709fcc14c2d9c0449cb64b980d0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:24.779847 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_39f7c709fcc14c2d9c0449cb64b980d0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:24.781533 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_df0389a8470d4dcfa6d9b90366867bae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:24.782069 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_df0389a8470d4dcfa6d9b90366867bae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:24.783349 521 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_7d121235cada49b2ba3f2f966bc02fc8" } modify_external_catalogs: true
I20260202 02:52:24.783541 521 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_7d121235cada49b2ba3f2f966bc02fc8" } modify_external_catalogs: true
I20260202 02:52:24.783844 1850 raft_consensus.cc:515] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.784651 1850 leader_election.cc:290] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:24.785081 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_378dba7a0b4f47c586bcdd3f53712ceb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:24.785738 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_378dba7a0b4f47c586bcdd3f53712ceb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:24.785773 1174 tablet_bootstrap.cc:492] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:24.786756 1174 ts_tablet_manager.cc:1403] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.251s user 0.005s sys 0.009s
I20260202 02:52:24.787861 528 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet c25e329023c04f609d8a743b4f7a95a4
W20260202 02:52:24.764130 859 leader_election.cc:343] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:24.788393 528 catalog_manager.cc:6724] ReplaceTablet: tablet c25e329023c04f609d8a743b4f7a95a4 deleted and replaced by tablet 0a5bbb994cfa40f0add3163816b65269
W20260202 02:52:24.789098 857 leader_election.cc:343] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:24.789448 857 leader_election.cc:304] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:24.785570 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "929eb48881b141718873d0e359e768a8" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:24.790383 1059 raft_consensus.cc:3060] T 929eb48881b141718873d0e359e768a8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:24.790369 902 tablet_service.cc:1552] Processing DeleteTablet for tablet 042d1ebf357b4abe8814a407891ce265 with delete_type TABLET_DATA_DELETED (replaced by tablet 2c2a535de2294cc697d75561d0463642) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:24.790551 1936 raft_consensus.cc:2749] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:24.790616 889 tablet_service.cc:1552] Processing DeleteTablet for tablet c25e329023c04f609d8a743b4f7a95a4 with delete_type TABLET_DATA_DELETED (replaced by tablet 0a5bbb994cfa40f0add3163816b65269) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:24.767355 1182 ts_tablet_manager.cc:1929] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:24.777971 1865 raft_consensus.cc:493] T a1f39e5b02f747819e636cddb4292eb6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:24.791231 1182 log.cc:1199] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/38fdbc6df7ec40b9afe4e0dfe0119e3e
I20260202 02:52:24.791289 1865 raft_consensus.cc:515] T a1f39e5b02f747819e636cddb4292eb6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.791638 1182 ts_tablet_manager.cc:1950] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:24.792022 1865 leader_election.cc:290] T a1f39e5b02f747819e636cddb4292eb6 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:24.792596 1058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a1f39e5b02f747819e636cddb4292eb6" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:24.793079 1059 raft_consensus.cc:2468] T 929eb48881b141718873d0e359e768a8 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
W20260202 02:52:24.793431 857 consensus_peers.cc:597] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5 -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:24.793629 715 leader_election.cc:304] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:24.793694 1182 tablet_replica.cc:333] T 042d1ebf357b4abe8814a407891ce265 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:24.793890 1182 raft_consensus.cc:2243] T 042d1ebf357b4abe8814a407891ce265 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260202 02:52:24.794011 714 leader_election.cc:343] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 929eb48881b141718873d0e359e768a8
I20260202 02:52:24.794073 1182 raft_consensus.cc:2272] T 042d1ebf357b4abe8814a407891ce265 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:24.794188 859 leader_election.cc:343] T a1f39e5b02f747819e636cddb4292eb6 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:24.794540 1850 raft_consensus.cc:2804] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:24.794643 1182 ts_tablet_manager.cc:1916] T 042d1ebf357b4abe8814a407891ce265 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:24.794622 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 38fdbc6df7ec40b9afe4e0dfe0119e3e (table default.Table_87277c5d80184738b96b773f026b32a5 [id=52687dbd0e554cfb81d88e8fcdd4dd5d]) successfully deleted
I20260202 02:52:24.794991 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet c25e329023c04f609d8a743b4f7a95a4 with delete_type TABLET_DATA_DELETED (replaced by tablet 0a5bbb994cfa40f0add3163816b65269) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:24.789822 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet 042d1ebf357b4abe8814a407891ce265 with delete_type TABLET_DATA_DELETED (replaced by tablet 2c2a535de2294cc697d75561d0463642) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:24.795426 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 267bd4752c5f4b99a3fe778ba8f38a2f (table default.Table_dda472e63fac4873ab3ccefcac3635cd [id=40105de1d8074d3e983ff62557d14f48]) successfully deleted
I20260202 02:52:24.763447 1175 ts_tablet_manager.cc:1434] T a1f39e5b02f747819e636cddb4292eb6 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.005s user 0.000s sys 0.001s
I20260202 02:52:24.796012 1175 tablet_bootstrap.cc:492] T 13cd1383348b458894fdecc22bf24b74 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:24.796916 1882 tablet_replica.cc:333] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:24.794543 749 tablet_service.cc:1552] Processing DeleteTablet for tablet c25e329023c04f609d8a743b4f7a95a4 with delete_type TABLET_DATA_DELETED (replaced by tablet 0a5bbb994cfa40f0add3163816b65269) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:24.795433 1850 raft_consensus.cc:697] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:24.801720 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a1f39e5b02f747819e636cddb4292eb6" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:24.802932 857 leader_election.cc:343] T a1f39e5b02f747819e636cddb4292eb6 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:24.803083 857 leader_election.cc:304] T a1f39e5b02f747819e636cddb4292eb6 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:24.803699 1175 tablet_bootstrap.cc:654] T 13cd1383348b458894fdecc22bf24b74 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:24.803706 1865 raft_consensus.cc:2749] T a1f39e5b02f747819e636cddb4292eb6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:24.805574 758 tablet_service.cc:1552] Processing DeleteTablet for tablet 042d1ebf357b4abe8814a407891ce265 with delete_type TABLET_DATA_DELETED (replaced by tablet 2c2a535de2294cc697d75561d0463642) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:24.808429 1850 consensus_queue.cc:237] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.807423 1173 tablet_bootstrap.cc:492] T 91ec01d25f8f441cbe3783cacbee51db P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:24.810869 1173 ts_tablet_manager.cc:1403] T 91ec01d25f8f441cbe3783cacbee51db P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.200s user 0.017s sys 0.011s
I20260202 02:52:24.811383 1182 ts_tablet_manager.cc:1929] T 042d1ebf357b4abe8814a407891ce265 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:24.811554 1182 log.cc:1199] T 042d1ebf357b4abe8814a407891ce265 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/042d1ebf357b4abe8814a407891ce265
I20260202 02:52:24.811692 1173 raft_consensus.cc:359] T 91ec01d25f8f441cbe3783cacbee51db P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.811908 1173 raft_consensus.cc:385] T 91ec01d25f8f441cbe3783cacbee51db P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:24.811918 1182 ts_tablet_manager.cc:1950] T 042d1ebf357b4abe8814a407891ce265 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:24.811985 1173 raft_consensus.cc:740] T 91ec01d25f8f441cbe3783cacbee51db P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:24.812225 1173 consensus_queue.cc:260] T 91ec01d25f8f441cbe3783cacbee51db P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.813020 1173 ts_tablet_manager.cc:1434] T 91ec01d25f8f441cbe3783cacbee51db P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:24.813853 1882 raft_consensus.cc:2243] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:24.814230 1882 pending_rounds.cc:70] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d: Trying to abort 1 pending ops.
I20260202 02:52:24.814298 1882 pending_rounds.cc:77] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923041009020928 op_type: NO_OP noop_request { }
I20260202 02:52:24.814865 1173 tablet_bootstrap.cc:492] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:24.817131 1173 tablet_bootstrap.cc:654] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:24.821275 528 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_b9aa847125ed4116b6fde1cfc1cafa27" } modify_external_catalogs: true
I20260202 02:52:24.821533 528 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_b9aa847125ed4116b6fde1cfc1cafa27" } modify_external_catalogs: true
I20260202 02:52:24.823182 525 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet cd1b6662da254889ae4bbe8d0b1ef1bb
I20260202 02:52:24.823717 525 catalog_manager.cc:6724] ReplaceTablet: tablet cd1b6662da254889ae4bbe8d0b1ef1bb deleted and replaced by tablet 83f30c7f03504cdbb379fc0469324306
I20260202 02:52:24.824141 1174 raft_consensus.cc:359] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.824395 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_ca7d5182d8e340adbb15c7c1457862d7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:24.825074 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ca7d5182d8e340adbb15c7c1457862d7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:24.827232 526 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_e8e9b4cdaf434b6cbbfed1a65bff1fc0 [id=78a7fb3a2eed4565b55d5dacb8a3d7df] alter complete (version 1)
I20260202 02:52:24.827503 889 tablet_service.cc:1552] Processing DeleteTablet for tablet cd1b6662da254889ae4bbe8d0b1ef1bb with delete_type TABLET_DATA_DELETED (replaced by tablet 83f30c7f03504cdbb379fc0469324306) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:24.827947 1174 raft_consensus.cc:385] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:24.828088 1174 raft_consensus.cc:740] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:24.828979 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet cd1b6662da254889ae4bbe8d0b1ef1bb with delete_type TABLET_DATA_DELETED (replaced by tablet 83f30c7f03504cdbb379fc0469324306) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:24.830152 1174 consensus_queue.cc:260] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.833837 1277 raft_consensus.cc:2749] T ffd80badce1048f3ac95366cad05ae53 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:24.834292 1175 tablet_bootstrap.cc:492] T 13cd1383348b458894fdecc22bf24b74 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:24.834476 1175 ts_tablet_manager.cc:1403] T 13cd1383348b458894fdecc22bf24b74 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.039s user 0.001s sys 0.002s
I20260202 02:52:24.835347 1175 raft_consensus.cc:359] T 13cd1383348b458894fdecc22bf24b74 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.835572 1175 raft_consensus.cc:385] T 13cd1383348b458894fdecc22bf24b74 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:24.835654 1175 raft_consensus.cc:740] T 13cd1383348b458894fdecc22bf24b74 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:24.835953 1175 consensus_queue.cc:260] T 13cd1383348b458894fdecc22bf24b74 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.836570 1175 ts_tablet_manager.cc:1434] T 13cd1383348b458894fdecc22bf24b74 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260202 02:52:24.814412 1882 raft_consensus.cc:2889] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:24.836977 1175 tablet_bootstrap.cc:492] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:24.837152 1882 raft_consensus.cc:2272] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:24.841269 1174 ts_tablet_manager.cc:1434] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.044s user 0.003s sys 0.000s
I20260202 02:52:24.841892 1180 ts_tablet_manager.cc:1929] T 583e547d0a1943f09f4efe82070fc777 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:24.842078 1180 log.cc:1199] T 583e547d0a1943f09f4efe82070fc777 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/583e547d0a1943f09f4efe82070fc777
I20260202 02:52:24.842895 520 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_3a2daddab50e4578898828f3a305b8e7" } new_table_name: "default.Table_abd89cee70a04e2aa3a0d59b7eb943c0" modify_external_catalogs: true
I20260202 02:52:24.844892 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 042d1ebf357b4abe8814a407891ce265 (table default.Table_950f6f5ae1ef43399de785df25c896e9 [id=cab83305ba6648e5b307441e71671a82]) successfully deleted
I20260202 02:52:24.845031 1865 raft_consensus.cc:493] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:24.845219 1865 raft_consensus.cc:515] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.845966 1882 ts_tablet_manager.cc:1916] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:24.846092 1865 leader_election.cc:290] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:24.848114 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet c25e329023c04f609d8a743b4f7a95a4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c25e329023c04f609d8a743b4f7a95a4 already in progress: creating tablet
W20260202 02:52:24.848269 515 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 1116872cc25a4ab5b1a3a6bbba1f0da7 (table default.Table_d6ba351d68ec4a10939a09519b7d4b67 [id=52a6d31f993d4c74a520d504390cf12b], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 1116872cc25a4ab5b1a3a6bbba1f0da7 consensus state has no leader
I20260202 02:52:24.846844 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "08d8d781bb4e4acf915f11afb4fb2e22" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:24.848999 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet cd1b6662da254889ae4bbe8d0b1ef1bb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cd1b6662da254889ae4bbe8d0b1ef1bb already in progress: creating tablet
I20260202 02:52:24.850333 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "08d8d781bb4e4acf915f11afb4fb2e22" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:24.850772 754 tablet_service.cc:1552] Processing DeleteTablet for tablet cd1b6662da254889ae4bbe8d0b1ef1bb with delete_type TABLET_DATA_DELETED (replaced by tablet 83f30c7f03504cdbb379fc0469324306) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:24.852950 1175 tablet_bootstrap.cc:654] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:24.857883 857 leader_election.cc:343] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:24.858263 527 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_c2f3ac0894bd4f4f9702b6429da397bc" } modify_external_catalogs: true
I20260202 02:52:24.858492 527 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_c2f3ac0894bd4f4f9702b6429da397bc" } modify_external_catalogs: true
W20260202 02:52:24.864814 859 leader_election.cc:343] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:24.865211 859 leader_election.cc:304] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:24.865474 1175 tablet_bootstrap.cc:492] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:24.865691 1865 raft_consensus.cc:2749] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:24.865959 1175 ts_tablet_manager.cc:1403] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.029s user 0.001s sys 0.004s
I20260202 02:52:24.866892 1175 raft_consensus.cc:359] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.867311 1175 raft_consensus.cc:385] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:24.867472 1175 raft_consensus.cc:740] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:24.867890 1175 consensus_queue.cc:260] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.868785 1175 ts_tablet_manager.cc:1434] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260202 02:52:24.870309 1180 ts_tablet_manager.cc:1950] T 583e547d0a1943f09f4efe82070fc777 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:24.870519 1174 tablet_bootstrap.cc:492] T 622a113af04a41c6852c29fc8a8acd78 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:24.872730 1175 tablet_bootstrap.cc:492] T c25e329023c04f609d8a743b4f7a95a4 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:24.875685 1721 raft_consensus.cc:493] T d0671586e45e45ffb7fd9e1be6e2bbf8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:24.875880 1721 raft_consensus.cc:515] T d0671586e45e45ffb7fd9e1be6e2bbf8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.876652 1721 leader_election.cc:290] T d0671586e45e45ffb7fd9e1be6e2bbf8 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:24.877240 1175 tablet_bootstrap.cc:654] T c25e329023c04f609d8a743b4f7a95a4 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:24.878229 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_6f5a23937e4f4572b861c8889b453872"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:24.878914 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6f5a23937e4f4572b861c8889b453872 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:24.879935 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d0671586e45e45ffb7fd9e1be6e2bbf8" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:24.880610 998 leader_election.cc:343] T d0671586e45e45ffb7fd9e1be6e2bbf8 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:24.882179 1173 tablet_bootstrap.cc:492] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:24.882411 1173 ts_tablet_manager.cc:1403] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.068s user 0.004s sys 0.000s
I20260202 02:52:24.883327 1173 raft_consensus.cc:359] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:24.883599 1173 raft_consensus.cc:385] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:24.883733 1173 raft_consensus.cc:740] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:24.884204 1173 consensus_queue.cc:260] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:24.884909 1173 ts_tablet_manager.cc:1434] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260202 02:52:24.888450 999 leader_election.cc:343] T d0671586e45e45ffb7fd9e1be6e2bbf8 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: d0671586e45e45ffb7fd9e1be6e2bbf8
I20260202 02:52:24.888651 999 leader_election.cc:304] T d0671586e45e45ffb7fd9e1be6e2bbf8 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:24.889092 1173 tablet_bootstrap.cc:492] T 79f694b740954d429145fba9157338ca P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:24.889122 1721 raft_consensus.cc:2749] T d0671586e45e45ffb7fd9e1be6e2bbf8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:24.894039 1173 tablet_bootstrap.cc:654] T 79f694b740954d429145fba9157338ca P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:24.898242 1173 tablet_bootstrap.cc:492] T 79f694b740954d429145fba9157338ca P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:24.898361 775 raft_consensus.cc:3060] T d21fca9a5b234c26879717f51c8ea0f0 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:24.898429 1173 ts_tablet_manager.cc:1403] T 79f694b740954d429145fba9157338ca P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260202 02:52:24.898838 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 298) took 4695 ms. Trace:
I20260202 02:52:24.902041 1721 consensus_queue.cc:1048] T d21fca9a5b234c26879717f51c8ea0f0 P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:24.912073 1175 tablet_bootstrap.cc:492] T c25e329023c04f609d8a743b4f7a95a4 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:24.912256 1175 ts_tablet_manager.cc:1403] T c25e329023c04f609d8a743b4f7a95a4 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.040s user 0.003s sys 0.001s
I20260202 02:52:24.913035 521 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet c977c146bac34638874b7a990604a0d5
I20260202 02:52:24.898957 1180 rpcz_store.cc:276] 0202 02:52:20.203533 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:20.203603 (+ 70us) service_pool.cc:225] Handling call
0202 02:52:24.722792 (+4519189us) ts_tablet_manager.cc:1151] Deleting tablet 583e547d0a1943f09f4efe82070fc777
0202 02:52:24.722802 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:24.782804 (+ 60002us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:24.839566 (+ 56762us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:24.898785 (+ 59219us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4518938}
I20260202 02:52:24.914356 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 583e547d0a1943f09f4efe82070fc777 (table default.Table_2669ca9662b942c08bcbfd7641cfe2e3 [id=fb813e4c5b0849f8a488c7a22d1defae]) successfully deleted
I20260202 02:52:24.914577 751 tablet_service.cc:1552] Processing DeleteTablet for tablet c977c146bac34638874b7a990604a0d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:24.915007 889 tablet_service.cc:1552] Processing DeleteTablet for tablet c977c146bac34638874b7a990604a0d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:24.915798 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 299) took 4661 ms. Trace:
I20260202 02:52:24.916860 521 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_cf37f87385ee44cea1ff80cc14050ff2" } modify_external_catalogs: true
I20260202 02:52:24.916932 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet c977c146bac34638874b7a990604a0d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:24.917625 521 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_cf37f87385ee44cea1ff80cc14050ff2" } modify_external_catalogs: true
I20260202 02:52:24.917959 1180 rpcz_store.cc:276] 0202 02:52:20.254055 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:20.275010 (+ 20955us) service_pool.cc:225] Handling call
0202 02:52:24.915579 (+4640569us) ts_tablet_manager.cc:1151] Deleting tablet 38301e13e17c41fcae912c4b38c26a45
0202 02:52:24.915594 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:24.915756 (+ 162us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4640194}
W20260202 02:52:24.918220 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet c977c146bac34638874b7a990604a0d5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c977c146bac34638874b7a990604a0d5 already in progress: creating tablet
I20260202 02:52:24.922663 1180 tablet_replica.cc:333] T 80d443bdb66d4cc68ae283e08c410fe6 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:24.922952 1180 raft_consensus.cc:2243] T 80d443bdb66d4cc68ae283e08c410fe6 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:24.923173 1180 raft_consensus.cc:2272] T 80d443bdb66d4cc68ae283e08c410fe6 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:24.923821 1180 ts_tablet_manager.cc:1916] T 80d443bdb66d4cc68ae283e08c410fe6 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:24.924114 1175 raft_consensus.cc:359] T c25e329023c04f609d8a743b4f7a95a4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.924371 1175 raft_consensus.cc:385] T c25e329023c04f609d8a743b4f7a95a4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:24.924445 1175 raft_consensus.cc:740] T c25e329023c04f609d8a743b4f7a95a4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:24.924678 1175 consensus_queue.cc:260] T c25e329023c04f609d8a743b4f7a95a4 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.925288 1175 ts_tablet_manager.cc:1434] T c25e329023c04f609d8a743b4f7a95a4 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.013s user 0.002s sys 0.000s
I20260202 02:52:24.925673 1173 raft_consensus.cc:359] T 79f694b740954d429145fba9157338ca P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.925897 1173 raft_consensus.cc:385] T 79f694b740954d429145fba9157338ca P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:24.925977 1173 raft_consensus.cc:740] T 79f694b740954d429145fba9157338ca P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:24.926383 1175 tablet_bootstrap.cc:492] T 38301e13e17c41fcae912c4b38c26a45 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:24.927803 1173 consensus_queue.cc:260] T 79f694b740954d429145fba9157338ca P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.917830 1174 tablet_bootstrap.cc:654] T 622a113af04a41c6852c29fc8a8acd78 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:24.929253 1173 ts_tablet_manager.cc:1434] T 79f694b740954d429145fba9157338ca P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.031s user 0.002s sys 0.001s
I20260202 02:52:24.931149 1173 tablet_bootstrap.cc:492] T 2b8ba9dd450948ada395207ae1c5b325 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:24.933573 1173 tablet_bootstrap.cc:654] T 2b8ba9dd450948ada395207ae1c5b325 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:24.936566 524 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_b9ea6590fd0b48608fc1ebe68e7eae9b" } modify_external_catalogs: true
I20260202 02:52:24.936818 524 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_b9ea6590fd0b48608fc1ebe68e7eae9b" } modify_external_catalogs: true
I20260202 02:52:24.948807 1173 tablet_bootstrap.cc:492] T 2b8ba9dd450948ada395207ae1c5b325 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:24.948989 1173 ts_tablet_manager.cc:1403] T 2b8ba9dd450948ada395207ae1c5b325 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.004s
I20260202 02:52:24.949680 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_4799ebe642db4543be7712203e1fe532"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:24.949785 1173 raft_consensus.cc:359] T 2b8ba9dd450948ada395207ae1c5b325 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.949946 1173 raft_consensus.cc:385] T 2b8ba9dd450948ada395207ae1c5b325 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:24.950026 1173 raft_consensus.cc:740] T 2b8ba9dd450948ada395207ae1c5b325 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:24.950274 1173 consensus_queue.cc:260] T 2b8ba9dd450948ada395207ae1c5b325 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:24.950369 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4799ebe642db4543be7712203e1fe532 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:24.950831 1173 ts_tablet_manager.cc:1434] T 2b8ba9dd450948ada395207ae1c5b325 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:24.951131 1173 tablet_bootstrap.cc:492] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:24.963514 526 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_1b9d1f82c9004ea5b02312053140e514 [id=b4ab2ba86e1a49338a98d855ded99f5d] alter complete (version 2)
I20260202 02:52:24.964362 1173 tablet_bootstrap.cc:654] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:24.966588 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet 2f0e8c5e2fd44370ae150a206f4e6cbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:24.966856 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 59990791123c4d4eb620090a1e8e31e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:24.967007 1037 tablet_service.cc:1552] Processing DeleteTablet for tablet d0671586e45e45ffb7fd9e1be6e2bbf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:24.972307 1175 tablet_bootstrap.cc:654] T 38301e13e17c41fcae912c4b38c26a45 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:24.972862 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 08d8d781bb4e4acf915f11afb4fb2e22 with delete_type TABLET_DATA_DELETED (replaced by tablet 7921daa000f54c8dbed04473e21cf0a5) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:24.973155 1182 tablet_replica.cc:333] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:24.973311 894 tablet_service.cc:1552] Processing DeleteTablet for tablet 68823eff1bc04108ba83487605aec0ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:21 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:24.973369 1182 raft_consensus.cc:2243] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:24.973162 902 tablet_service.cc:1552] Processing DeleteTablet for tablet 9c1271ce0f194021b588b82ac9d3db79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:24.973541 1182 raft_consensus.cc:2272] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:24.973567 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 1f54c1dfe0a942b3bafd85522953a032 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:24.973611 889 tablet_service.cc:1552] Processing DeleteTablet for tablet a5622ef925bc46cca76f481ddfd12afd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:21 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:24.974090 1182 ts_tablet_manager.cc:1916] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:24.977101 999 consensus_peers.cc:597] T d21fca9a5b234c26879717f51c8ea0f0 P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d21fca9a5b234c26879717f51c8ea0f0. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:24.981303 1182 ts_tablet_manager.cc:1929] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:24.981428 1182 log.cc:1199] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/08d8d781bb4e4acf915f11afb4fb2e22
I20260202 02:52:24.981549 1175 tablet_bootstrap.cc:492] T 38301e13e17c41fcae912c4b38c26a45 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:24.981724 1175 ts_tablet_manager.cc:1403] T 38301e13e17c41fcae912c4b38c26a45 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.055s user 0.006s sys 0.000s
I20260202 02:52:24.982493 1175 raft_consensus.cc:359] T 38301e13e17c41fcae912c4b38c26a45 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.982679 1175 raft_consensus.cc:385] T 38301e13e17c41fcae912c4b38c26a45 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:24.982735 1175 raft_consensus.cc:740] T 38301e13e17c41fcae912c4b38c26a45 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:24.982951 1175 consensus_queue.cc:260] T 38301e13e17c41fcae912c4b38c26a45 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:24.983541 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_bcaaa936a01e4aafbb3dc42f01abff56"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:24.984236 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bcaaa936a01e4aafbb3dc42f01abff56 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:24.986236 519 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_668e8ec9913f43fa8499eff32be8db0a" } new_table_name: "default.Table_f48ed1c7e02349adad7ce22c0e51ed81" modify_external_catalogs: true
I20260202 02:52:24.986397 1175 ts_tablet_manager.cc:1434] T 38301e13e17c41fcae912c4b38c26a45 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260202 02:52:24.986687 1175 tablet_bootstrap.cc:492] T 06b80a1c9251420cbc16fc95ae98aca9 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:24.981740 1182 ts_tablet_manager.cc:1950] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:24.987424 1182 tablet_replica.cc:333] T 68823eff1bc04108ba83487605aec0ea P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:24.987599 1182 raft_consensus.cc:2243] T 68823eff1bc04108ba83487605aec0ea P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:24.987677 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 08d8d781bb4e4acf915f11afb4fb2e22 (table default.Table_2acd399bc33146fe9f1b6e2a15e0b101 [id=487a204c7845401ab7391ae8464fdf1d]) successfully deleted
I20260202 02:52:24.987771 1182 raft_consensus.cc:2272] T 68823eff1bc04108ba83487605aec0ea P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:24.988602 1182 ts_tablet_manager.cc:1916] T 68823eff1bc04108ba83487605aec0ea P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:24.990888 1882 ts_tablet_manager.cc:1929] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:24.991178 1882 log.cc:1199] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/c064ea0dade840b4840a5715c2d83f69
I20260202 02:52:24.991703 1882 ts_tablet_manager.cc:1950] T c064ea0dade840b4840a5715c2d83f69 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:24.991824 1175 tablet_bootstrap.cc:654] T 06b80a1c9251420cbc16fc95ae98aca9 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:25.009723 1174 tablet_bootstrap.cc:492] T 622a113af04a41c6852c29fc8a8acd78 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:25.009917 1174 ts_tablet_manager.cc:1403] T 622a113af04a41c6852c29fc8a8acd78 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.168s user 0.007s sys 0.000s
I20260202 02:52:25.010872 1174 raft_consensus.cc:359] T 622a113af04a41c6852c29fc8a8acd78 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:25.011091 1174 raft_consensus.cc:385] T 622a113af04a41c6852c29fc8a8acd78 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:25.011194 1174 raft_consensus.cc:740] T 622a113af04a41c6852c29fc8a8acd78 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:25.011444 1174 consensus_queue.cc:260] T 622a113af04a41c6852c29fc8a8acd78 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:25.012754 1174 ts_tablet_manager.cc:1434] T 622a113af04a41c6852c29fc8a8acd78 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.003s user 0.002s sys 0.002s
I20260202 02:52:25.013376 774 raft_consensus.cc:3060] T 622a113af04a41c6852c29fc8a8acd78 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:25.015168 2004 log.cc:927] Time spent T d21fca9a5b234c26879717f51c8ea0f0 P 1f442ef85029461091de847118c34c0d: Append to log took a long time: real 0.056s user 0.000s sys 0.000s
I20260202 02:52:25.018659 1174 tablet_bootstrap.cc:492] T f6746ccbf37e477196a9706396d1e3fd P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:25.018378 1173 tablet_bootstrap.cc:492] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:25.020480 1173 ts_tablet_manager.cc:1403] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.069s user 0.005s sys 0.002s
I20260202 02:52:25.020629 1174 tablet_bootstrap.cc:654] T f6746ccbf37e477196a9706396d1e3fd P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:25.021831 1173 raft_consensus.cc:359] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:25.023674 1175 tablet_bootstrap.cc:492] T 06b80a1c9251420cbc16fc95ae98aca9 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:25.023844 1175 ts_tablet_manager.cc:1403] T 06b80a1c9251420cbc16fc95ae98aca9 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.037s user 0.002s sys 0.002s
I20260202 02:52:25.022061 1173 raft_consensus.cc:385] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:25.024731 1173 raft_consensus.cc:740] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:25.022709 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 352) took 1812 ms. Trace:
I20260202 02:52:25.025004 1882 rpcz_store.cc:276] 0202 02:52:23.210073 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.210174 (+ 101us) service_pool.cc:225] Handling call
0202 02:52:24.796819 (+1586645us) ts_tablet_manager.cc:1151] Deleting tablet c064ea0dade840b4840a5715c2d83f69
0202 02:52:24.796833 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:24.863252 (+ 66419us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:24.940412 (+ 77160us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:25.022646 (+ 82234us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1571511}
I20260202 02:52:25.025628 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet c064ea0dade840b4840a5715c2d83f69 (table default.Table_718e4de3a182443fb16b265396c79401 [id=f7c5b24c065443d89a2b1f9c34409c9d]) successfully deleted
I20260202 02:52:25.026331 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 366) took 1807 ms. Trace:
I20260202 02:52:25.026495 1882 rpcz_store.cc:276] 0202 02:52:23.219029 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.224683 (+ 5654us) service_pool.cc:225] Handling call
0202 02:52:25.025923 (+1801240us) ts_tablet_manager.cc:1151] Deleting tablet 08d8d781bb4e4acf915f11afb4fb2e22
0202 02:52:25.025939 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:25.026265 (+ 326us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1793171}
W20260202 02:52:25.031651 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 08d8d781bb4e4acf915f11afb4fb2e22 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 08d8d781bb4e4acf915f11afb4fb2e22 already in progress: creating tablet
I20260202 02:52:25.032419 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 371) took 1782 ms. Trace:
I20260202 02:52:25.035663 1882 rpcz_store.cc:276] 0202 02:52:23.249641 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.249718 (+ 77us) service_pool.cc:225] Handling call
0202 02:52:25.032155 (+1782437us) ts_tablet_manager.cc:1151] Deleting tablet 26eeef6eebbf4f2cb44f69255fb37594
0202 02:52:25.032170 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:25.032345 (+ 175us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1775069}
I20260202 02:52:25.036695 2014 tablet.cc:1722] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260202 02:52:25.041200 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 26eeef6eebbf4f2cb44f69255fb37594 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 26eeef6eebbf4f2cb44f69255fb37594 already in progress: creating tablet
I20260202 02:52:25.041647 1882 tablet_replica.cc:333] T 526f2c7b95184a0987e7aa94efc8475a P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:25.041863 1882 raft_consensus.cc:2243] T 526f2c7b95184a0987e7aa94efc8475a P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:25.042533 1180 ts_tablet_manager.cc:1929] T 80d443bdb66d4cc68ae283e08c410fe6 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:25.042696 1180 log.cc:1199] T 80d443bdb66d4cc68ae283e08c410fe6 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/80d443bdb66d4cc68ae283e08c410fe6
I20260202 02:52:25.043121 1180 ts_tablet_manager.cc:1950] T 80d443bdb66d4cc68ae283e08c410fe6 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:25.043467 1882 raft_consensus.cc:2272] T 526f2c7b95184a0987e7aa94efc8475a P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:25.045104 520 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 09504070c9ec4bcfbb59e3fb4c189775 consensus state has no leader
I20260202 02:52:25.049168 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 300) took 4763 ms. Trace:
I20260202 02:52:25.049312 1180 rpcz_store.cc:276] 0202 02:52:20.285573 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:20.285697 (+ 124us) service_pool.cc:225] Handling call
0202 02:52:24.922559 (+4636862us) ts_tablet_manager.cc:1151] Deleting tablet 80d443bdb66d4cc68ae283e08c410fe6
0202 02:52:24.922576 (+ 17us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:25.000457 (+ 77881us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:25.042370 (+ 41913us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:25.049101 (+ 6731us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4636447}
I20260202 02:52:25.049727 1180 tablet_replica.cc:333] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:25.049923 1180 raft_consensus.cc:2243] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:25.050122 1180 raft_consensus.cc:2272] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:25.050860 1180 ts_tablet_manager.cc:1916] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:25.051007 1882 ts_tablet_manager.cc:1916] T 526f2c7b95184a0987e7aa94efc8475a P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:25.051607 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 80d443bdb66d4cc68ae283e08c410fe6 (table default.Table_78e75899f83c46f09c66dc84192f5438 [id=0bd47d48c02944a19ce210fc83794879]) successfully deleted
I20260202 02:52:25.058704 1865 raft_consensus.cc:493] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:25.058921 1865 raft_consensus.cc:515] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.059581 1865 leader_election.cc:290] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:25.060789 525 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_7a967fd89a5243c4a02abdaf16f6d62e" } new_table_name: "default.Table_17a421804b4e4a088917d6fcca9a0d27" modify_external_catalogs: true
I20260202 02:52:25.061691 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "91ec01d25f8f441cbe3783cacbee51db" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:25.061792 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "91ec01d25f8f441cbe3783cacbee51db" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:25.061946 1060 raft_consensus.cc:2468] T 91ec01d25f8f441cbe3783cacbee51db P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
W20260202 02:52:25.062312 519 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 44c8d5b74c8e4c499f7f68e8185efc17 consensus state has no leader
W20260202 02:52:25.062386 857 leader_election.cc:343] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:25.062604 519 master_service.cc:954] ReplaceTablet: received request to replace tablet ccd6bc8bd0504bf18d108c73685ef9ff from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:25.064157 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_0ee7c010ec144c2e9d633447e4d84dcf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:25.064785 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0ee7c010ec144c2e9d633447e4d84dcf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:25.065646 751 tablet_service.cc:1505] Processing CreateTablet for tablet 09a8aa4dd70b4fe2806293d2620e521c (DEFAULT_TABLE table=default.Table_5ce52f099d05416f87636eaf90c9f958 [id=555fe298a1c744aca99bf6949e8d5ce4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.066241 742 tablet_service.cc:1505] Processing CreateTablet for tablet bb145df67e1a44569a3dbc91c024b584 (DEFAULT_TABLE table=default.Table_79cf9f546b684c598c9447efe6997ca7 [id=bf2ff444381f4d62a22258ffbfe1c040]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.066725 759 tablet_service.cc:1505] Processing CreateTablet for tablet 8ab5c6fbec394b9bab322a197181a618 (DEFAULT_TABLE table=default.Table_85ea5ebb89ae4a8fb42f2f5c96ab588c [id=a1681a194792400f8e5c295d166a0778]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.067315 859 leader_election.cc:304] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:25.067791 1936 raft_consensus.cc:2804] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:25.067870 1936 raft_consensus.cc:493] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:25.067934 1936 raft_consensus.cc:3060] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:25.068572 522 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet cb634d4bae554ef3b39eeaa9958adc6c (table default.Table_e5f4ac82d1ef4b85999695997eb9df2a [id=c97e407e519d4a538860f66931f6aefd]). Expected version 2 got 0
I20260202 02:52:25.069646 1175 raft_consensus.cc:359] T 06b80a1c9251420cbc16fc95ae98aca9 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:25.069871 1175 raft_consensus.cc:385] T 06b80a1c9251420cbc16fc95ae98aca9 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:25.069916 1175 raft_consensus.cc:740] T 06b80a1c9251420cbc16fc95ae98aca9 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:25.070143 1175 consensus_queue.cc:260] T 06b80a1c9251420cbc16fc95ae98aca9 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:25.070472 522 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_222f75e4260d41b2813393c25a2a6e14" } new_table_name: "default.Table_7c01ea0d668a490099dbd42db17162ec" modify_external_catalogs: true
I20260202 02:52:25.070605 1175 ts_tablet_manager.cc:1434] T 06b80a1c9251420cbc16fc95ae98aca9 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.047s user 0.000s sys 0.003s
I20260202 02:52:25.070922 1175 tablet_bootstrap.cc:492] T 3d1a7578e83447669e3c3972a426ad44 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:25.073611 889 tablet_service.cc:1505] Processing CreateTablet for tablet 09a8aa4dd70b4fe2806293d2620e521c (DEFAULT_TABLE table=default.Table_5ce52f099d05416f87636eaf90c9f958 [id=555fe298a1c744aca99bf6949e8d5ce4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.073818 884 tablet_service.cc:1552] Processing DeleteTablet for tablet a1f39e5b02f747819e636cddb4292eb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:25.074046 893 tablet_service.cc:1552] Processing DeleteTablet for tablet 13cd1383348b458894fdecc22bf24b74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:25.074128 888 tablet_service.cc:1505] Processing CreateTablet for tablet bb145df67e1a44569a3dbc91c024b584 (DEFAULT_TABLE table=default.Table_79cf9f546b684c598c9447efe6997ca7 [id=bf2ff444381f4d62a22258ffbfe1c040]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.074267 899 tablet_service.cc:1552] Processing DeleteTablet for tablet c25e329023c04f609d8a743b4f7a95a4 with delete_type TABLET_DATA_DELETED (replaced by tablet 0a5bbb994cfa40f0add3163816b65269) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:25.074682 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 09a8aa4dd70b4fe2806293d2620e521c (DEFAULT_TABLE table=default.Table_5ce52f099d05416f87636eaf90c9f958 [id=555fe298a1c744aca99bf6949e8d5ce4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.075115 755 tablet_service.cc:1505] Processing CreateTablet for tablet 7e94cacaa0714f92ad6a2b2ab6171f44 (DEFAULT_TABLE table=default.Table_d6bec77a74b1431fb1c008f55f482b3a [id=7abfd5e060064195adb951fe38b7f715]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.075232 1044 tablet_service.cc:1505] Processing CreateTablet for tablet bb145df67e1a44569a3dbc91c024b584 (DEFAULT_TABLE table=default.Table_79cf9f546b684c598c9447efe6997ca7 [id=bf2ff444381f4d62a22258ffbfe1c040]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.075593 1865 consensus_queue.cc:1048] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:25.075625 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 8ab5c6fbec394b9bab322a197181a618 (DEFAULT_TABLE table=default.Table_85ea5ebb89ae4a8fb42f2f5c96ab588c [id=a1681a194792400f8e5c295d166a0778]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.075737 902 tablet_service.cc:1505] Processing CreateTablet for tablet 8ab5c6fbec394b9bab322a197181a618 (DEFAULT_TABLE table=default.Table_85ea5ebb89ae4a8fb42f2f5c96ab588c [id=a1681a194792400f8e5c295d166a0778]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.076048 894 tablet_service.cc:1552] Processing DeleteTablet for tablet 8774c6bc0fe9414ea2f17ed859d9392a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:25.076929 747 tablet_service.cc:1505] Processing CreateTablet for tablet 940e656cd34e4e53896fda6d38132b6e (DEFAULT_TABLE table=default.Table_3d5171d76fb147efb648b0e99b6c3d9d [id=68d698b5760e455a8584cb0069f97f7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.077196 899 tablet_service.cc:1505] Processing CreateTablet for tablet 940e656cd34e4e53896fda6d38132b6e (DEFAULT_TABLE table=default.Table_3d5171d76fb147efb648b0e99b6c3d9d [id=68d698b5760e455a8584cb0069f97f7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.077632 893 tablet_service.cc:1505] Processing CreateTablet for tablet b34dc1cab5874fa6aee55d79c6e09ade (DEFAULT_TABLE table=default.Table_e72399a9ba4b4a7a9259eec83a2f1f71 [id=2a2750072b914dc6b6f3adadd645206a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.077745 894 tablet_service.cc:1505] Processing CreateTablet for tablet 7e94cacaa0714f92ad6a2b2ab6171f44 (DEFAULT_TABLE table=default.Table_d6bec77a74b1431fb1c008f55f482b3a [id=7abfd5e060064195adb951fe38b7f715]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.078820 1042 tablet_service.cc:1505] Processing CreateTablet for tablet 7e94cacaa0714f92ad6a2b2ab6171f44 (DEFAULT_TABLE table=default.Table_d6bec77a74b1431fb1c008f55f482b3a [id=7abfd5e060064195adb951fe38b7f715]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.078977 754 tablet_service.cc:1505] Processing CreateTablet for tablet b34dc1cab5874fa6aee55d79c6e09ade (DEFAULT_TABLE table=default.Table_e72399a9ba4b4a7a9259eec83a2f1f71 [id=2a2750072b914dc6b6f3adadd645206a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.079348 1041 tablet_service.cc:1505] Processing CreateTablet for tablet 940e656cd34e4e53896fda6d38132b6e (DEFAULT_TABLE table=default.Table_3d5171d76fb147efb648b0e99b6c3d9d [id=68d698b5760e455a8584cb0069f97f7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.081503 1038 tablet_service.cc:1505] Processing CreateTablet for tablet b34dc1cab5874fa6aee55d79c6e09ade (DEFAULT_TABLE table=default.Table_e72399a9ba4b4a7a9259eec83a2f1f71 [id=2a2750072b914dc6b6f3adadd645206a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.081933 758 tablet_service.cc:1505] Processing CreateTablet for tablet 47e50559a7c04ae080fcc67342c631c9 (DEFAULT_TABLE table=default.Table_db26223b0d604aeabd9431ef5f27da88 [id=b8037b42177d4cb89d45bfba424fe797]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.082122 884 tablet_service.cc:1505] Processing CreateTablet for tablet 47e50559a7c04ae080fcc67342c631c9 (DEFAULT_TABLE table=default.Table_db26223b0d604aeabd9431ef5f27da88 [id=b8037b42177d4cb89d45bfba424fe797]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.082249 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 47e50559a7c04ae080fcc67342c631c9 (DEFAULT_TABLE table=default.Table_db26223b0d604aeabd9431ef5f27da88 [id=b8037b42177d4cb89d45bfba424fe797]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.082930 898 tablet_service.cc:1505] Processing CreateTablet for tablet 470f9ce5e38f4d22b4850e986a4d5c52 (DEFAULT_TABLE table=default.Table_e4bcea76324043f09b2a8d4cdec7f371 [id=4cb26dd223bc4f418094e3ec7c207998]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.083184 520 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 91ec01d25f8f441cbe3783cacbee51db (table default.Table_882b22c2c2b0446c83239be15da80b95 [id=255f4d94812f45d0b4423150098460ff]). Expected version 1 got 0
I20260202 02:52:25.083407 520 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 20ea39bc5d3d46f5a4ba427277d6c6a3 (table default.Table_3b335a0806154f98a907f551596ba353 [id=0da4db61b3fe4cf4846edd5530b9327f]). Expected version 1 got 0
I20260202 02:52:25.083356 749 tablet_service.cc:1505] Processing CreateTablet for tablet 470f9ce5e38f4d22b4850e986a4d5c52 (DEFAULT_TABLE table=default.Table_e4bcea76324043f09b2a8d4cdec7f371 [id=4cb26dd223bc4f418094e3ec7c207998]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.083515 2015 tablet.cc:1722] T 761f5a51335a4de5977f5f7fef0d4260 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:25.084656 1028 tablet_service.cc:1505] Processing CreateTablet for tablet 470f9ce5e38f4d22b4850e986a4d5c52 (DEFAULT_TABLE table=default.Table_e4bcea76324043f09b2a8d4cdec7f371 [id=4cb26dd223bc4f418094e3ec7c207998]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.085999 886 tablet_service.cc:1505] Processing CreateTablet for tablet 353bbad7f24f40e3b6316f619c749017 (DEFAULT_TABLE table=default.Table_bdbef281942540c7bf6d06c6c66b2acb [id=bb2dfdb752ee45aea4f8658346231390]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.086529 746 tablet_service.cc:1505] Processing CreateTablet for tablet 353bbad7f24f40e3b6316f619c749017 (DEFAULT_TABLE table=default.Table_bdbef281942540c7bf6d06c6c66b2acb [id=bb2dfdb752ee45aea4f8658346231390]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.087749 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 353bbad7f24f40e3b6316f619c749017 (DEFAULT_TABLE table=default.Table_bdbef281942540c7bf6d06c6c66b2acb [id=bb2dfdb752ee45aea4f8658346231390]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.025058 1173 consensus_queue.cc:260] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:25.091094 1173 ts_tablet_manager.cc:1434] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.070s user 0.000s sys 0.002s
I20260202 02:52:25.093375 901 tablet_service.cc:1505] Processing CreateTablet for tablet 7f227a89344b4aa6a62f6b58f679fbef (DEFAULT_TABLE table=default.Table_efacf68164014a2d8873859a3650a913 [id=baa4d4e86e9b475899c69cc47683f853]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.094866 528 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet a5f864fd8a3d4c4a83e7f68abb8a0a83
I20260202 02:52:25.095155 1174 tablet_bootstrap.cc:492] T f6746ccbf37e477196a9706396d1e3fd P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:25.095350 1174 ts_tablet_manager.cc:1403] T f6746ccbf37e477196a9706396d1e3fd P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.082s user 0.000s sys 0.006s
I20260202 02:52:25.096180 1174 raft_consensus.cc:359] T f6746ccbf37e477196a9706396d1e3fd P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.096414 1174 raft_consensus.cc:385] T f6746ccbf37e477196a9706396d1e3fd P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:25.096493 1174 raft_consensus.cc:740] T f6746ccbf37e477196a9706396d1e3fd P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:25.097630 892 tablet_service.cc:1552] Processing DeleteTablet for tablet a5f864fd8a3d4c4a83e7f68abb8a0a83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:25.098845 1175 tablet_bootstrap.cc:654] T 3d1a7578e83447669e3c3972a426ad44 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:25.099691 528 master_service.cc:954] ReplaceTablet: received request to replace tablet ac2ab12f5040488aa306cb69b107b485 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:25.100558 744 tablet_service.cc:1552] Processing DeleteTablet for tablet a5f864fd8a3d4c4a83e7f68abb8a0a83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:25.101199 1034 tablet_service.cc:1552] Processing DeleteTablet for tablet a5f864fd8a3d4c4a83e7f68abb8a0a83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:25.103389 744 tablet_service.cc:1505] Processing CreateTablet for tablet 7f227a89344b4aa6a62f6b58f679fbef (DEFAULT_TABLE table=default.Table_efacf68164014a2d8873859a3650a913 [id=baa4d4e86e9b475899c69cc47683f853]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.100478 892 tablet_service.cc:1552] Processing DeleteTablet for tablet 38301e13e17c41fcae912c4b38c26a45 with delete_type TABLET_DATA_DELETED (replaced by tablet 79badde1001b432b95f58b5161ace15c) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:25.105065 1034 tablet_service.cc:1505] Processing CreateTablet for tablet 7f227a89344b4aa6a62f6b58f679fbef (DEFAULT_TABLE table=default.Table_efacf68164014a2d8873859a3650a913 [id=baa4d4e86e9b475899c69cc47683f853]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.105726 1043 tablet_service.cc:1505] Processing CreateTablet for tablet 25f1d698c4ee4be7a33380184b5e2b4d (DEFAULT_TABLE table=default.Table_f7a491432d7b42cf82732e39dcb574ce [id=2e83ee87482a4173b12fbfff30355138]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.106550 1173 tablet_bootstrap.cc:492] T 042d1ebf357b4abe8814a407891ce265 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:25.107501 750 tablet_service.cc:1505] Processing CreateTablet for tablet 25f1d698c4ee4be7a33380184b5e2b4d (DEFAULT_TABLE table=default.Table_f7a491432d7b42cf82732e39dcb574ce [id=2e83ee87482a4173b12fbfff30355138]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.107983 2018 tablet.cc:1722] T 761f5a51335a4de5977f5f7fef0d4260 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:25.108088 757 tablet_service.cc:1505] Processing CreateTablet for tablet 36db4ee7f5404cffaa3c14ad58e71701 (DEFAULT_TABLE table=default.Table_caf008ae2a414bdbbfdec276ff3725ea [id=45814f7e71644658870918b288682928]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.108394 1173 tablet_bootstrap.cc:654] T 042d1ebf357b4abe8814a407891ce265 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:25.108604 741 tablet_service.cc:1505] Processing CreateTablet for tablet 455e7d1197c144eda1cf1ef405aeb262 (DEFAULT_TABLE table=default.Table_5538c7d2137648efb81b8057751d9972 [id=d3fcc9cc54ba4938858eb27aa840f5d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.109122 745 tablet_service.cc:1505] Processing CreateTablet for tablet 5d2fc0a74684434d85b9922cb4e32fdc (DEFAULT_TABLE table=default.Table_ac6e92f3b1f540a7af72ae905e0481a7 [id=63269345f7e842dd9d4d250191e12c6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.110857 1035 tablet_service.cc:1505] Processing CreateTablet for tablet 36db4ee7f5404cffaa3c14ad58e71701 (DEFAULT_TABLE table=default.Table_caf008ae2a414bdbbfdec276ff3725ea [id=45814f7e71644658870918b288682928]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.110272 1174 consensus_queue.cc:260] T f6746ccbf37e477196a9706396d1e3fd P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.111385 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 455e7d1197c144eda1cf1ef405aeb262 (DEFAULT_TABLE table=default.Table_5538c7d2137648efb81b8057751d9972 [id=d3fcc9cc54ba4938858eb27aa840f5d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.112041 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 06b80a1c9251420cbc16fc95ae98aca9 with delete_type TABLET_DATA_DELETED (replaced by tablet 2f5aae089fcc492f9ce55f84674a452c) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:25.113013 885 tablet_service.cc:1505] Processing CreateTablet for tablet 25f1d698c4ee4be7a33380184b5e2b4d (DEFAULT_TABLE table=default.Table_f7a491432d7b42cf82732e39dcb574ce [id=2e83ee87482a4173b12fbfff30355138]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.113855 891 tablet_service.cc:1505] Processing CreateTablet for tablet 455e7d1197c144eda1cf1ef405aeb262 (DEFAULT_TABLE table=default.Table_5538c7d2137648efb81b8057751d9972 [id=d3fcc9cc54ba4938858eb27aa840f5d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.113952 890 tablet_service.cc:1505] Processing CreateTablet for tablet 5d2fc0a74684434d85b9922cb4e32fdc (DEFAULT_TABLE table=default.Table_ac6e92f3b1f540a7af72ae905e0481a7 [id=63269345f7e842dd9d4d250191e12c6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.115478 892 tablet_service.cc:1505] Processing CreateTablet for tablet 36db4ee7f5404cffaa3c14ad58e71701 (DEFAULT_TABLE table=default.Table_caf008ae2a414bdbbfdec276ff3725ea [id=45814f7e71644658870918b288682928]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:25.116796 525 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 67a8c9a1998b47b3b214a06ce62381e9 consensus state has no leader
I20260202 02:52:25.116261 1174 ts_tablet_manager.cc:1434] T f6746ccbf37e477196a9706396d1e3fd P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.021s user 0.001s sys 0.002s
I20260202 02:52:25.118191 1894 raft_consensus.cc:493] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:25.118364 1894 raft_consensus.cc:515] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:25.119031 1894 leader_election.cc:290] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:25.119881 1174 tablet_bootstrap.cc:492] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:25.119805 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a5622ef925bc46cca76f481ddfd12afd" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:25.120425 998 leader_election.cc:343] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:25.122941 525 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_aa12e8c2bff14a968a174b8879bba516" } new_table_name: "default.Table_6831562d21db463986dc58b0100e9792" modify_external_catalogs: true
I20260202 02:52:25.126000 1936 raft_consensus.cc:515] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.126617 1936 leader_election.cc:290] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:25.127537 917 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a5622ef925bc46cca76f481ddfd12afd" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:25.127810 917 raft_consensus.cc:2468] T a5622ef925bc46cca76f481ddfd12afd P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:25.128006 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 455e7d1197c144eda1cf1ef405aeb262. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.128237 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e94cacaa0714f92ad6a2b2ab6171f44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.128757 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 353bbad7f24f40e3b6316f619c749017. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.128899 999 leader_election.cc:304] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:25.129014 1174 tablet_bootstrap.cc:654] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:25.129230 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 470f9ce5e38f4d22b4850e986a4d5c52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.129369 1894 raft_consensus.cc:2804] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:25.129474 1894 raft_consensus.cc:493] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:25.129555 1894 raft_consensus.cc:3060] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:25.129666 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 940e656cd34e4e53896fda6d38132b6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.130086 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b34dc1cab5874fa6aee55d79c6e09ade. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.130441 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb145df67e1a44569a3dbc91c024b584. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.130796 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09a8aa4dd70b4fe2806293d2620e521c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.131151 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ab5c6fbec394b9bab322a197181a618. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.131520 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47e50559a7c04ae080fcc67342c631c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.132050 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d2fc0a74684434d85b9922cb4e32fdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.132113 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "91ec01d25f8f441cbe3783cacbee51db" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:25.132504 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36db4ee7f5404cffaa3c14ad58e71701. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.132936 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25f1d698c4ee4be7a33380184b5e2b4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.133378 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f227a89344b4aa6a62f6b58f679fbef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.133685 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ab5c6fbec394b9bab322a197181a618. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.133767 1182 ts_tablet_manager.cc:1929] T 68823eff1bc04108ba83487605aec0ea P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:25.133849 1182 log.cc:1199] T 68823eff1bc04108ba83487605aec0ea P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/68823eff1bc04108ba83487605aec0ea
I20260202 02:52:25.134147 1182 ts_tablet_manager.cc:1950] T 68823eff1bc04108ba83487605aec0ea P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:25.134207 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 353bbad7f24f40e3b6316f619c749017. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.134361 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09a8aa4dd70b4fe2806293d2620e521c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.134688 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb145df67e1a44569a3dbc91c024b584. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.135115 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 940e656cd34e4e53896fda6d38132b6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.135517 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b34dc1cab5874fa6aee55d79c6e09ade. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.135859 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e94cacaa0714f92ad6a2b2ab6171f44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.136243 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47e50559a7c04ae080fcc67342c631c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.136853 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 470f9ce5e38f4d22b4850e986a4d5c52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.137266 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f227a89344b4aa6a62f6b58f679fbef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.137634 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 455e7d1197c144eda1cf1ef405aeb262. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.137984 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36db4ee7f5404cffaa3c14ad58e71701. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.138394 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d2fc0a74684434d85b9922cb4e32fdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.138780 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25f1d698c4ee4be7a33380184b5e2b4d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:25.145329 857 leader_election.cc:343] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:25.146210 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09a8aa4dd70b4fe2806293d2620e521c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.146469 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 470f9ce5e38f4d22b4850e986a4d5c52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.146690 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f227a89344b4aa6a62f6b58f679fbef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.146907 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 353bbad7f24f40e3b6316f619c749017. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.147344 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b34dc1cab5874fa6aee55d79c6e09ade. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.147538 2022 tablet.cc:1722] T 622a113af04a41c6852c29fc8a8acd78 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:25.147751 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 940e656cd34e4e53896fda6d38132b6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.148111 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e94cacaa0714f92ad6a2b2ab6171f44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.148437 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47e50559a7c04ae080fcc67342c631c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.148885 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb145df67e1a44569a3dbc91c024b584. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.149256 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ab5c6fbec394b9bab322a197181a618. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.149709 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 455e7d1197c144eda1cf1ef405aeb262. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.150167 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36db4ee7f5404cffaa3c14ad58e71701. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:25.151945 998 consensus_peers.cc:597] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:25.152241 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25f1d698c4ee4be7a33380184b5e2b4d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:25.169790 859 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 94534.5us
I20260202 02:52:25.180676 885 tablet_service.cc:1505] Processing CreateTablet for tablet 8d9ffdb5f10e4510888a90c49cd779bd (DEFAULT_TABLE table=default.Table_346e800aad794a65acc77ee0bfb0b82c [id=0dcc50bca61b416ca6d7f7b2d3da8582]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.181181 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d9ffdb5f10e4510888a90c49cd779bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.181809 895 tablet_service.cc:1505] Processing CreateTablet for tablet 47edd9f65ef343b9a3be74c30708daca (DEFAULT_TABLE table=default.Table_1e4db591b5664638a2cd494429582cdf [id=cfef307c1e814a12a21a488477750337]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.182147 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47edd9f65ef343b9a3be74c30708daca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.183830 1173 tablet_bootstrap.cc:492] T 042d1ebf357b4abe8814a407891ce265 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:25.183990 1173 ts_tablet_manager.cc:1403] T 042d1ebf357b4abe8814a407891ce265 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.093s user 0.003s sys 0.000s
I20260202 02:52:25.184821 1173 raft_consensus.cc:359] T 042d1ebf357b4abe8814a407891ce265 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.185025 1173 raft_consensus.cc:385] T 042d1ebf357b4abe8814a407891ce265 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:25.185110 1173 raft_consensus.cc:740] T 042d1ebf357b4abe8814a407891ce265 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:25.185354 1173 consensus_queue.cc:260] T 042d1ebf357b4abe8814a407891ce265 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.185879 1173 ts_tablet_manager.cc:1434] T 042d1ebf357b4abe8814a407891ce265 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:25.186125 887 tablet_service.cc:1505] Processing CreateTablet for tablet 28f6161d0d0a4b41af4560f0f3bee0f7 (DEFAULT_TABLE table=default.Table_48faa250b04a4b61a37f532584081968 [id=a77fa0c30d27405294adaef676bc76f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.186504 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28f6161d0d0a4b41af4560f0f3bee0f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.186795 1173 tablet_bootstrap.cc:492] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:25.190330 752 tablet_service.cc:1505] Processing CreateTablet for tablet 8d9ffdb5f10e4510888a90c49cd779bd (DEFAULT_TABLE table=default.Table_346e800aad794a65acc77ee0bfb0b82c [id=0dcc50bca61b416ca6d7f7b2d3da8582]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.191164 756 tablet_service.cc:1505] Processing CreateTablet for tablet 47edd9f65ef343b9a3be74c30708daca (DEFAULT_TABLE table=default.Table_1e4db591b5664638a2cd494429582cdf [id=cfef307c1e814a12a21a488477750337]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.191906 740 tablet_service.cc:1505] Processing CreateTablet for tablet 28f6161d0d0a4b41af4560f0f3bee0f7 (DEFAULT_TABLE table=default.Table_48faa250b04a4b61a37f532584081968 [id=a77fa0c30d27405294adaef676bc76f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.196990 1039 tablet_service.cc:1505] Processing CreateTablet for tablet 8d9ffdb5f10e4510888a90c49cd779bd (DEFAULT_TABLE table=default.Table_346e800aad794a65acc77ee0bfb0b82c [id=0dcc50bca61b416ca6d7f7b2d3da8582]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.205650 747 tablet_service.cc:1505] Processing CreateTablet for tablet 3191a80374724effa8389adc842e3448 (DEFAULT_TABLE table=default.Table_82c14c20a7bf41b487efaff52ebc15a9 [id=7fbb33c105f149ef98505dcf622930c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.213265 1036 tablet_service.cc:1505] Processing CreateTablet for tablet 5d2fc0a74684434d85b9922cb4e32fdc (DEFAULT_TABLE table=default.Table_ac6e92f3b1f540a7af72ae905e0481a7 [id=63269345f7e842dd9d4d250191e12c6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.215561 1032 tablet_service.cc:1505] Processing CreateTablet for tablet 47edd9f65ef343b9a3be74c30708daca (DEFAULT_TABLE table=default.Table_1e4db591b5664638a2cd494429582cdf [id=cfef307c1e814a12a21a488477750337]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.215996 1033 tablet_service.cc:1505] Processing CreateTablet for tablet 28f6161d0d0a4b41af4560f0f3bee0f7 (DEFAULT_TABLE table=default.Table_48faa250b04a4b61a37f532584081968 [id=a77fa0c30d27405294adaef676bc76f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.216311 1173 tablet_bootstrap.cc:654] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:25.217669 1175 tablet_bootstrap.cc:492] T 3d1a7578e83447669e3c3972a426ad44 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:25.217826 1175 ts_tablet_manager.cc:1403] T 3d1a7578e83447669e3c3972a426ad44 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.147s user 0.000s sys 0.009s
I20260202 02:52:25.218564 1175 raft_consensus.cc:359] T 3d1a7578e83447669e3c3972a426ad44 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:25.218740 1175 raft_consensus.cc:385] T 3d1a7578e83447669e3c3972a426ad44 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:25.218788 1175 raft_consensus.cc:740] T 3d1a7578e83447669e3c3972a426ad44 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:25.218979 1175 consensus_queue.cc:260] T 3d1a7578e83447669e3c3972a426ad44 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:25.219471 1175 ts_tablet_manager.cc:1434] T 3d1a7578e83447669e3c3972a426ad44 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:25.219797 1175 tablet_bootstrap.cc:492] T d30557fbe9084cd3860820f69dda8752 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:25.222790 901 tablet_service.cc:1505] Processing CreateTablet for tablet 3191a80374724effa8389adc842e3448 (DEFAULT_TABLE table=default.Table_82c14c20a7bf41b487efaff52ebc15a9 [id=7fbb33c105f149ef98505dcf622930c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.222985 519 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet ccd6bc8bd0504bf18d108c73685ef9ff
I20260202 02:52:25.223470 519 catalog_manager.cc:6724] ReplaceTablet: tablet ccd6bc8bd0504bf18d108c73685ef9ff deleted and replaced by tablet 8ba28ab7d7ae44059006179067dff3c0
I20260202 02:52:25.223928 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_ba53f6b46a524a03832adc957671ae98"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:25.224542 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ba53f6b46a524a03832adc957671ae98 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:25.225627 759 tablet_service.cc:1552] Processing DeleteTablet for tablet ccd6bc8bd0504bf18d108c73685ef9ff with delete_type TABLET_DATA_DELETED (replaced by tablet 8ba28ab7d7ae44059006179067dff3c0) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:25.228739 524 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 361c0c3832614ec78695c28720b841e0
I20260202 02:52:25.229393 524 master_service.cc:954] ReplaceTablet: received request to replace tablet 3706562a22634eb6b72e36bb6a2d1ae5 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:25.230615 527 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 7f9dfac19c04475aadb45e95a6474eb4
I20260202 02:52:25.230989 1175 tablet_bootstrap.cc:654] T d30557fbe9084cd3860820f69dda8752 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:25.231123 759 tablet_service.cc:1552] Processing DeleteTablet for tablet 361c0c3832614ec78695c28720b841e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:25.231282 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_2cf49aa4b72d4448943e3ea50fa21c78"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:25.231709 759 tablet_service.cc:1552] Processing DeleteTablet for tablet 7f9dfac19c04475aadb45e95a6474eb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:42950
W20260202 02:52:25.231921 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2cf49aa4b72d4448943e3ea50fa21c78 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:25.232898 1182 tablet_replica.cc:333] T 9c1271ce0f194021b588b82ac9d3db79 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:25.233096 1182 raft_consensus.cc:2243] T 9c1271ce0f194021b588b82ac9d3db79 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:25.233244 1182 raft_consensus.cc:2272] T 9c1271ce0f194021b588b82ac9d3db79 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:25.233781 521 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet a0b5d82e5fb94b60b4aa324b978a5378
I20260202 02:52:25.233796 1182 ts_tablet_manager.cc:1916] T 9c1271ce0f194021b588b82ac9d3db79 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:25.234544 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_072c7718ffd04cdfaafcce94c169bcd8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:25.234659 759 tablet_service.cc:1552] Processing DeleteTablet for tablet a0b5d82e5fb94b60b4aa324b978a5378 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:42950
W20260202 02:52:25.235144 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_072c7718ffd04cdfaafcce94c169bcd8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:25.238651 1894 raft_consensus.cc:515] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:25.239423 1894 leader_election.cc:290] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:25.240518 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3191a80374724effa8389adc842e3448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.241425 528 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet ac2ab12f5040488aa306cb69b107b485
I20260202 02:52:25.241909 528 catalog_manager.cc:6724] ReplaceTablet: tablet ac2ab12f5040488aa306cb69b107b485 deleted and replaced by tablet 898778f8f0b548ec89cb1417d6f07926
I20260202 02:52:25.242205 1040 tablet_service.cc:1552] Processing DeleteTablet for tablet ccd6bc8bd0504bf18d108c73685ef9ff with delete_type TABLET_DATA_DELETED (replaced by tablet 8ba28ab7d7ae44059006179067dff3c0) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:25.242249 528 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_2acd399bc33146fe9f1b6e2a15e0b101" } modify_external_catalogs: true
I20260202 02:52:25.242419 528 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_2acd399bc33146fe9f1b6e2a15e0b101" } modify_external_catalogs: true
I20260202 02:52:25.242705 1174 tablet_bootstrap.cc:492] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:25.242871 1174 ts_tablet_manager.cc:1403] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.125s user 0.010s sys 0.006s
I20260202 02:52:25.243697 1174 raft_consensus.cc:359] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:25.243916 1174 raft_consensus.cc:385] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:25.244009 1174 raft_consensus.cc:740] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:25.244277 1174 consensus_queue.cc:260] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:25.244431 759 tablet_service.cc:1552] Processing DeleteTablet for tablet ac2ab12f5040488aa306cb69b107b485 with delete_type TABLET_DATA_DELETED (replaced by tablet 898778f8f0b548ec89cb1417d6f07926) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:25.245810 1174 ts_tablet_manager.cc:1434] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260202 02:52:25.247269 1174 tablet_bootstrap.cc:492] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:25.249087 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_d5007434a43d4bed87b905c21243d3aa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:25.249650 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d5007434a43d4bed87b905c21243d3aa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:25.249742 1174 tablet_bootstrap.cc:654] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:25.251042 525 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cef87d5f74f14a6d86150958de7b1ace consensus state has no leader
I20260202 02:52:25.251412 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_8a639c50c6674b2da7771e12fe2d7237"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:25.251981 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8a639c50c6674b2da7771e12fe2d7237 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:25.254271 520 master_service.cc:954] ReplaceTablet: received request to replace tablet 474df41e4fe24a818fa45a4ce84d5474 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:25.257658 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a5622ef925bc46cca76f481ddfd12afd" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:25.257789 759 tablet_service.cc:1552] Processing DeleteTablet for tablet 94bada52db6a47179415d302b541d01f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:42950
W20260202 02:52:25.260066 423 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 45834.1us
I20260202 02:52:25.261760 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 68823eff1bc04108ba83487605aec0ea (table default.Table_d4ad0ff5676e4af2a187497eda81d7a2 [id=0769ae34ed3d479a9f7010b34c35a110]) successfully deleted
I20260202 02:52:25.262780 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 7f9dfac19c04475aadb45e95a6474eb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:25.263028 891 tablet_service.cc:1552] Processing DeleteTablet for tablet a0b5d82e5fb94b60b4aa324b978a5378 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:25.263206 889 tablet_service.cc:1552] Processing DeleteTablet for tablet ac2ab12f5040488aa306cb69b107b485 with delete_type TABLET_DATA_DELETED (replaced by tablet 898778f8f0b548ec89cb1417d6f07926) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:25.267606 523 master_service.cc:954] ReplaceTablet: received request to replace tablet a0b5d82e5fb94b60b4aa324b978a5378 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:25.268378 523 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_14f3398803354519b92b5291be072517" } new_table_name: "default.Table_dfebb268090044a0bc4b4b7a830a9535" modify_external_catalogs: true
I20260202 02:52:25.269064 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 361c0c3832614ec78695c28720b841e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:25.271322 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_ce37f495e88145148c5f83a15c901826"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:25.271917 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ce37f495e88145148c5f83a15c901826 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:25.273764 917 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a5622ef925bc46cca76f481ddfd12afd" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:25.273989 917 raft_consensus.cc:3060] T a5622ef925bc46cca76f481ddfd12afd P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:25.275931 524 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 3706562a22634eb6b72e36bb6a2d1ae5
I20260202 02:52:25.276607 524 catalog_manager.cc:6724] ReplaceTablet: tablet 3706562a22634eb6b72e36bb6a2d1ae5 deleted and replaced by tablet 87ff5fbc95b84171a94fe1a83a67a7ba
I20260202 02:52:25.279158 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_3c414f5adb634d8eb3c5341307bad9fa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:25.279780 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3c414f5adb634d8eb3c5341307bad9fa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:25.288125 524 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_90344989f4234a11a30653bdc34fe899" } new_table_name: "default.Table_c33bb328c09b4b6ab9974ec216487b8a" modify_external_catalogs: true
I20260202 02:52:25.289435 746 tablet_service.cc:1552] Processing DeleteTablet for tablet 3706562a22634eb6b72e36bb6a2d1ae5 with delete_type TABLET_DATA_DELETED (replaced by tablet 87ff5fbc95b84171a94fe1a83a67a7ba) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:25.292913 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_642b9e6132634a6cb5876a88fb02e9a1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:25.293576 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_642b9e6132634a6cb5876a88fb02e9a1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:25.295923 2015 tablet.cc:1722] T 757492288d7c44fb91b25b7e6e21e01b P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:25.296734 1600 raft_consensus.cc:493] T f6746ccbf37e477196a9706396d1e3fd P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260202 02:52:25.296882 998 leader_election.cc:343] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:25.297828 1600 raft_consensus.cc:515] T f6746ccbf37e477196a9706396d1e3fd P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.298525 1600 leader_election.cc:290] T f6746ccbf37e477196a9706396d1e3fd P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:25.299269 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 3706562a22634eb6b72e36bb6a2d1ae5 with delete_type TABLET_DATA_DELETED (replaced by tablet 87ff5fbc95b84171a94fe1a83a67a7ba) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:25.300580 528 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 7921daa000f54c8dbed04473e21cf0a5
I20260202 02:52:25.301512 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_5963d74e3b6143cdb2eaca2cb99a58df"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:25.302075 1865 raft_consensus.cc:493] T 8774c6bc0fe9414ea2f17ed859d9392a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:25.302227 1865 raft_consensus.cc:515] T 8774c6bc0fe9414ea2f17ed859d9392a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.302829 1865 leader_election.cc:290] T 8774c6bc0fe9414ea2f17ed859d9392a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:25.302904 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5963d74e3b6143cdb2eaca2cb99a58df in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:25.304723 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f6746ccbf37e477196a9706396d1e3fd" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:25.305378 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 361c0c3832614ec78695c28720b841e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:25.306339 1037 tablet_service.cc:1552] Processing DeleteTablet for tablet 7f9dfac19c04475aadb45e95a6474eb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:25.308521 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 7921daa000f54c8dbed04473e21cf0a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:25 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:25.310251 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f6746ccbf37e477196a9706396d1e3fd" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:25.310501 923 raft_consensus.cc:2393] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f442ef85029461091de847118c34c0d in current term 1: Already voted for candidate b1f05820b6034228bdad884cc348716d in this term.
I20260202 02:52:25.313426 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet a0b5d82e5fb94b60b4aa324b978a5378 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:25.314685 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "91ec01d25f8f441cbe3783cacbee51db" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:25.314906 1059 raft_consensus.cc:3060] T 91ec01d25f8f441cbe3783cacbee51db P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:25.316926 525 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_2e2049e67b8943acab5a0db3deb1535a" } modify_external_catalogs: true
I20260202 02:52:25.317152 525 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_2e2049e67b8943acab5a0db3deb1535a" } modify_external_catalogs: true
I20260202 02:52:25.319103 1882 ts_tablet_manager.cc:1929] T 526f2c7b95184a0987e7aa94efc8475a P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:25.319239 1882 log.cc:1199] T 526f2c7b95184a0987e7aa94efc8475a P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/526f2c7b95184a0987e7aa94efc8475a
W20260202 02:52:25.319239 523 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b427c7e897c64977b924e7d586420e69 consensus state has no leader
I20260202 02:52:25.319629 1882 ts_tablet_manager.cc:1950] T 526f2c7b95184a0987e7aa94efc8475a P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:25.319625 520 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 474df41e4fe24a818fa45a4ce84d5474
I20260202 02:52:25.320094 520 catalog_manager.cc:6724] ReplaceTablet: tablet 474df41e4fe24a818fa45a4ce84d5474 deleted and replaced by tablet f30b698c177c48ee9a4e577552bf7eee
I20260202 02:52:25.322510 1059 raft_consensus.cc:2468] T 91ec01d25f8f441cbe3783cacbee51db P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
I20260202 02:52:25.323127 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 474df41e4fe24a818fa45a4ce84d5474 with delete_type TABLET_DATA_DELETED (replaced by tablet f30b698c177c48ee9a4e577552bf7eee) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:25.324549 917 raft_consensus.cc:2468] T a5622ef925bc46cca76f481ddfd12afd P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:25.325690 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8774c6bc0fe9414ea2f17ed859d9392a" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:25.326519 857 leader_election.cc:343] T 8774c6bc0fe9414ea2f17ed859d9392a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:25.327291 2018 tablet.cc:1722] T 757492288d7c44fb91b25b7e6e21e01b P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:25.327842 519 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_8cae34d2c6f1443ba07e3f2f8abd1989" } new_table_name: "default.Table_a15c97bb32284dcfb36cdc47921b2346" modify_external_catalogs: true
I20260202 02:52:25.330114 901 tablet_service.cc:1552] Processing DeleteTablet for tablet ccd6bc8bd0504bf18d108c73685ef9ff with delete_type TABLET_DATA_DELETED (replaced by tablet 8ba28ab7d7ae44059006179067dff3c0) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:25.331812 859 leader_election.cc:304] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:25.332774 1865 raft_consensus.cc:2804] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:25.332928 1865 raft_consensus.cc:697] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:25.333199 1865 consensus_queue.cc:237] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.337021 887 tablet_service.cc:1505] Processing CreateTablet for tablet de320a4afaa14ffd9d6123f2285e79fa (DEFAULT_TABLE table=default.Table_62e7c5405f2b46248650ac463717478b [id=0e3edffd39bd460b8ca0d02a983b76b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.343402 901 tablet_service.cc:1505] Processing CreateTablet for tablet 67495e8e525542fb976f675b71512ed2 (DEFAULT_TABLE table=default.Table_d1dc608853d84069a5d8fdfcc68cc10a [id=b2578b3303564a9ea04600020b7cefed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.347329 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d9ffdb5f10e4510888a90c49cd779bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.347782 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3191a80374724effa8389adc842e3448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.348382 1031 tablet_service.cc:1505] Processing CreateTablet for tablet de320a4afaa14ffd9d6123f2285e79fa (DEFAULT_TABLE table=default.Table_62e7c5405f2b46248650ac463717478b [id=0e3edffd39bd460b8ca0d02a983b76b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.348488 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28f6161d0d0a4b41af4560f0f3bee0f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.348667 1040 tablet_service.cc:1505] Processing CreateTablet for tablet 67495e8e525542fb976f675b71512ed2 (DEFAULT_TABLE table=default.Table_d1dc608853d84069a5d8fdfcc68cc10a [id=b2578b3303564a9ea04600020b7cefed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.348940 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47edd9f65ef343b9a3be74c30708daca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.352458 1037 tablet_service.cc:1552] Processing DeleteTablet for tablet ac2ab12f5040488aa306cb69b107b485 with delete_type TABLET_DATA_DELETED (replaced by tablet 898778f8f0b548ec89cb1417d6f07926) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:25.353024 714 leader_election.cc:304] T f6746ccbf37e477196a9706396d1e3fd P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:25.355042 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8774c6bc0fe9414ea2f17ed859d9392a" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:25.359071 999 leader_election.cc:304] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:25.360260 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet 3706562a22634eb6b72e36bb6a2d1ae5 with delete_type TABLET_DATA_DELETED (replaced by tablet 87ff5fbc95b84171a94fe1a83a67a7ba) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:25.360397 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 474df41e4fe24a818fa45a4ce84d5474 with delete_type TABLET_DATA_DELETED (replaced by tablet f30b698c177c48ee9a4e577552bf7eee) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:25.360580 1175 tablet_bootstrap.cc:492] T d30557fbe9084cd3860820f69dda8752 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:25.360569 1764 raft_consensus.cc:2804] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:25.360847 1175 ts_tablet_manager.cc:1403] T d30557fbe9084cd3860820f69dda8752 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.141s user 0.022s sys 0.015s
I20260202 02:52:25.360805 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 3191a80374724effa8389adc842e3448 (DEFAULT_TABLE table=default.Table_82c14c20a7bf41b487efaff52ebc15a9 [id=7fbb33c105f149ef98505dcf622930c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.362382 1764 raft_consensus.cc:697] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:25.362746 1764 consensus_queue.cc:237] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:25.363117 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d9ffdb5f10e4510888a90c49cd779bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.363243 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 474df41e4fe24a818fa45a4ce84d5474 with delete_type TABLET_DATA_DELETED (replaced by tablet f30b698c177c48ee9a4e577552bf7eee) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:25.363747 2022 tablet.cc:1722] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:25.364881 1175 raft_consensus.cc:359] T d30557fbe9084cd3860820f69dda8752 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:25.365063 1175 raft_consensus.cc:385] T d30557fbe9084cd3860820f69dda8752 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:25.365114 1175 raft_consensus.cc:740] T d30557fbe9084cd3860820f69dda8752 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:25.365304 1175 consensus_queue.cc:260] T d30557fbe9084cd3860820f69dda8752 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:25.366153 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de320a4afaa14ffd9d6123f2285e79fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.370626 757 tablet_service.cc:1505] Processing CreateTablet for tablet 67495e8e525542fb976f675b71512ed2 (DEFAULT_TABLE table=default.Table_d1dc608853d84069a5d8fdfcc68cc10a [id=b2578b3303564a9ea04600020b7cefed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.371500 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 374) took 2069 ms. Trace:
I20260202 02:52:25.372766 1175 ts_tablet_manager.cc:1434] T d30557fbe9084cd3860820f69dda8752 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.012s user 0.002s sys 0.000s
I20260202 02:52:25.372859 1600 raft_consensus.cc:3060] T f6746ccbf37e477196a9706396d1e3fd P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:25.371596 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 7921daa000f54c8dbed04473e21cf0a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:25 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:25.373699 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3191a80374724effa8389adc842e3448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.374631 750 tablet_service.cc:1552] Processing DeleteTablet for tablet 7921daa000f54c8dbed04473e21cf0a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:25 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:25.376785 1175 tablet_bootstrap.cc:492] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:25.379005 1175 tablet_bootstrap.cc:654] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:25.382683 1721 raft_consensus.cc:493] T 59990791123c4d4eb620090a1e8e31e1 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:25.386099 1865 raft_consensus.cc:493] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:25.386380 1865 raft_consensus.cc:515] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.387105 1865 leader_election.cc:290] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
W20260202 02:52:25.391256 859 leader_election.cc:343] T 8774c6bc0fe9414ea2f17ed859d9392a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:25.392148 527 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_7913f8fb6eac487a971545a11cf257a9 [id=b40664207c364307ba6f5ade490a21d6] alter complete (version 1)
I20260202 02:52:25.391745 859 leader_election.cc:304] T 8774c6bc0fe9414ea2f17ed859d9392a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:25.392917 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cb634d4bae554ef3b39eeaa9958adc6c" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:25.393360 1865 raft_consensus.cc:2749] T 8774c6bc0fe9414ea2f17ed859d9392a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:25.393806 857 leader_election.cc:343] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:25.393261 1174 tablet_bootstrap.cc:492] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
W20260202 02:52:25.393976 715 consensus_peers.cc:597] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 526f2c7b95184a0987e7aa94efc8475a. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:25.401157 1175 tablet_bootstrap.cc:492] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
W20260202 02:52:25.401014 515 consensus_peers.cc:597] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b -> Peer 7dab2abef14445d688ce1335b1f9b2f8 (127.0.104.190:33981): Couldn't send request to peer 7dab2abef14445d688ce1335b1f9b2f8. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.190:33981: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260202 02:52:25.401302 1175 ts_tablet_manager.cc:1403] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.002s
I20260202 02:52:25.402192 1175 raft_consensus.cc:359] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:25.402377 1175 raft_consensus.cc:385] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:25.402428 1175 raft_consensus.cc:740] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:25.402652 1175 consensus_queue.cc:260] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:25.403956 527 catalog_manager.cc:5654] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130). New cstate: current_term: 1 leader_uuid: "1521a6d262b7451bbef8debe970dd6b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: HEALTHY } } }
I20260202 02:52:25.404676 1865 raft_consensus.cc:493] T 38301e13e17c41fcae912c4b38c26a45 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:25.404808 1865 raft_consensus.cc:515] T 38301e13e17c41fcae912c4b38c26a45 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.404989 527 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 91ec01d25f8f441cbe3783cacbee51db (table default.Table_882b22c2c2b0446c83239be15da80b95 [id=255f4d94812f45d0b4423150098460ff]). Expected version 1 got 0
I20260202 02:52:25.405856 1865 leader_election.cc:290] T 38301e13e17c41fcae912c4b38c26a45 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:25.407999 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de320a4afaa14ffd9d6123f2285e79fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.409147 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67495e8e525542fb976f675b71512ed2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.410228 1182 ts_tablet_manager.cc:1929] T 9c1271ce0f194021b588b82ac9d3db79 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:25.410321 1182 log.cc:1199] T 9c1271ce0f194021b588b82ac9d3db79 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/9c1271ce0f194021b588b82ac9d3db79
I20260202 02:52:25.412503 1182 ts_tablet_manager.cc:1950] T 9c1271ce0f194021b588b82ac9d3db79 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:25.413379 1174 ts_tablet_manager.cc:1403] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.149s user 0.009s sys 0.021s
I20260202 02:52:25.416460 1894 consensus_queue.cc:1048] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.051s
I20260202 02:52:25.417251 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67495e8e525542fb976f675b71512ed2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.417287 1180 ts_tablet_manager.cc:1929] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:25.417464 1180 log.cc:1199] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/f93e8d3c4faf48eaa98c3cbf88d585e2
I20260202 02:52:25.417533 750 tablet_service.cc:1505] Processing CreateTablet for tablet de320a4afaa14ffd9d6123f2285e79fa (DEFAULT_TABLE table=default.Table_62e7c5405f2b46248650ac463717478b [id=0e3edffd39bd460b8ca0d02a983b76b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:25.417943 1180 ts_tablet_manager.cc:1950] T f93e8d3c4faf48eaa98c3cbf88d585e2 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:25.417979 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de320a4afaa14ffd9d6123f2285e79fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.418306 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 301) took 5122 ms. Trace:
I20260202 02:52:25.418429 1180 rpcz_store.cc:276] 0202 02:52:20.296003 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:20.297126 (+ 1123us) service_pool.cc:225] Handling call
0202 02:52:25.049641 (+4752515us) ts_tablet_manager.cc:1151] Deleting tablet f93e8d3c4faf48eaa98c3cbf88d585e2
0202 02:52:25.049651 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:25.127784 (+ 78133us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:25.406987 (+279203us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:25.418261 (+ 11274us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4752261}
I20260202 02:52:25.418780 1180 tablet_replica.cc:333] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:25.419003 1180 raft_consensus.cc:2243] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:25.419574 1180 raft_consensus.cc:2272] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:25.420370 1180 ts_tablet_manager.cc:1916] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:25.420302 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet f93e8d3c4faf48eaa98c3cbf88d585e2 (table default.Table_2ebdd65f856c4fd1881a573bcd3c46d4 [id=84a13cdc74684fc1ade2eb2d0a7f3c86]) successfully deleted
I20260202 02:52:25.421880 1043 tablet_service.cc:1552] Processing DeleteTablet for tablet 79f694b740954d429145fba9157338ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:21 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:25.406716 1175 ts_tablet_manager.cc:1434] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260202 02:52:25.424876 1175 tablet_bootstrap.cc:492] T fe9ea4c0502f4976b440465992f2ea2c P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:25.426744 1175 tablet_bootstrap.cc:654] T fe9ea4c0502f4976b440465992f2ea2c P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:25.427616 1182 tablet_replica.cc:333] T a5622ef925bc46cca76f481ddfd12afd P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:25.427779 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 9c1271ce0f194021b588b82ac9d3db79 (table default.Table_733044f900d44218a06c976c8701f122 [id=9f912bc02bec49a8b79e4e7e921595b9]) successfully deleted
I20260202 02:52:25.427817 1182 raft_consensus.cc:2243] T a5622ef925bc46cca76f481ddfd12afd P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:25.427969 1182 raft_consensus.cc:2272] T a5622ef925bc46cca76f481ddfd12afd P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:25.428030 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "38301e13e17c41fcae912c4b38c26a45" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:25.428515 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 1f54c1dfe0a942b3bafd85522953a032 because the tablet was not found. No further retry: Not found: Tablet not found: 1f54c1dfe0a942b3bafd85522953a032
I20260202 02:52:25.428578 1182 ts_tablet_manager.cc:1916] T a5622ef925bc46cca76f481ddfd12afd P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:25.428507 1600 raft_consensus.cc:2749] T f6746ccbf37e477196a9706396d1e3fd P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:25.429144 857 leader_election.cc:343] T 38301e13e17c41fcae912c4b38c26a45 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:25.429517 1882 rpcz_store.cc:276] 0202 02:52:23.290092 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.290783 (+ 691us) service_pool.cc:225] Handling call
0202 02:52:25.041547 (+1750764us) ts_tablet_manager.cc:1151] Deleting tablet 526f2c7b95184a0987e7aa94efc8475a
0202 02:52:25.041560 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:25.146242 (+104682us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:25.318933 (+172691us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:25.359399 (+ 40466us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1750483}
I20260202 02:52:25.429941 1882 tablet_replica.cc:333] T eb0ac55fbb534b29a86d6b3e8aa72ebc P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:25.430078 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67495e8e525542fb976f675b71512ed2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.430306 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28f6161d0d0a4b41af4560f0f3bee0f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.430470 1721 raft_consensus.cc:515] T 59990791123c4d4eb620090a1e8e31e1 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.430523 1174 raft_consensus.cc:359] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:25.430689 1174 raft_consensus.cc:385] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:25.430759 1174 raft_consensus.cc:740] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:25.431197 1721 leader_election.cc:290] T 59990791123c4d4eb620090a1e8e31e1 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:25.431139 1174 consensus_queue.cc:260] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:25.431826 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47edd9f65ef343b9a3be74c30708daca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.432026 1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d2fc0a74684434d85b9922cb4e32fdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:25.432765 1174 ts_tablet_manager.cc:1434] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.019s user 0.001s sys 0.001s
I20260202 02:52:25.433027 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "59990791123c4d4eb620090a1e8e31e1" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:25.433681 998 leader_election.cc:343] T 59990791123c4d4eb620090a1e8e31e1 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:25.436033 1860 consensus_queue.cc:1048] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.066s
I20260202 02:52:25.440997 1175 tablet_bootstrap.cc:492] T fe9ea4c0502f4976b440465992f2ea2c P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:25.441160 1175 ts_tablet_manager.cc:1403] T fe9ea4c0502f4976b440465992f2ea2c P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.000s
I20260202 02:52:25.441447 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cb634d4bae554ef3b39eeaa9958adc6c" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:25.441635 1894 consensus_queue.cc:1048] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:25.441852 1175 raft_consensus.cc:359] T fe9ea4c0502f4976b440465992f2ea2c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:25.442008 1175 raft_consensus.cc:385] T fe9ea4c0502f4976b440465992f2ea2c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:25.442052 1175 raft_consensus.cc:740] T fe9ea4c0502f4976b440465992f2ea2c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:25.442236 1175 consensus_queue.cc:260] T fe9ea4c0502f4976b440465992f2ea2c P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:25.444746 1175 ts_tablet_manager.cc:1434] T fe9ea4c0502f4976b440465992f2ea2c P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260202 02:52:25.445044 1175 tablet_bootstrap.cc:492] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:25.447563 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet a5622ef925bc46cca76f481ddfd12afd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:21 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:25.449425 1180 ts_tablet_manager.cc:1929] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:25.449646 1175 tablet_bootstrap.cc:654] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:25.449679 1180 log.cc:1199] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/c0937b6910534ddc8a2217f0cbd539c5
I20260202 02:52:25.450244 1180 ts_tablet_manager.cc:1950] T c0937b6910534ddc8a2217f0cbd539c5 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
W20260202 02:52:25.451685 999 leader_election.cc:343] T 59990791123c4d4eb620090a1e8e31e1 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 59990791123c4d4eb620090a1e8e31e1
I20260202 02:52:25.451856 999 leader_election.cc:304] T 59990791123c4d4eb620090a1e8e31e1 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:25.452577 1860 raft_consensus.cc:2749] T 59990791123c4d4eb620090a1e8e31e1 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:25.452291 1882 raft_consensus.cc:2243] T eb0ac55fbb534b29a86d6b3e8aa72ebc P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:25.455123 1882 raft_consensus.cc:2272] T eb0ac55fbb534b29a86d6b3e8aa72ebc P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:25.455770 1882 ts_tablet_manager.cc:1916] T eb0ac55fbb534b29a86d6b3e8aa72ebc P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:25.459151 1182 ts_tablet_manager.cc:1929] T a5622ef925bc46cca76f481ddfd12afd P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:25.459414 1182 log.cc:1199] T a5622ef925bc46cca76f481ddfd12afd P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/a5622ef925bc46cca76f481ddfd12afd
I20260202 02:52:25.459828 1182 ts_tablet_manager.cc:1950] T a5622ef925bc46cca76f481ddfd12afd P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:25.460561 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 526f2c7b95184a0987e7aa94efc8475a (table default.Table_0842fd6a82e145e7a98b3fed2c0c6993 [id=4e076b2023be4c888ac1c27babde1607]) successfully deleted
W20260202 02:52:25.461894 859 leader_election.cc:343] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:25.462507 1182 tablet_replica.cc:333] T a1f39e5b02f747819e636cddb4292eb6 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:25.462615 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet a5622ef925bc46cca76f481ddfd12afd (table default.Table_3d26f7a34d0c48379e3cebda66e1ef13 [id=45e6d83589fa493195b06f633e295b00]) successfully deleted
I20260202 02:52:25.462706 1182 raft_consensus.cc:2243] T a1f39e5b02f747819e636cddb4292eb6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:25.462844 1182 raft_consensus.cc:2272] T a1f39e5b02f747819e636cddb4292eb6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:25.463203 859 leader_election.cc:304] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:25.463753 1865 raft_consensus.cc:2749] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:25.463366 1182 ts_tablet_manager.cc:1916] T a1f39e5b02f747819e636cddb4292eb6 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:25.471494 521 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ca7d5182d8e340adbb15c7c1457862d7" } modify_external_catalogs: true
I20260202 02:52:25.471750 521 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ca7d5182d8e340adbb15c7c1457862d7" } modify_external_catalogs: true
I20260202 02:52:25.472659 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_8a25e41d22cf497aad652a6171bd142c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:25.472862 526 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_d6ba351d68ec4a10939a09519b7d4b67" } modify_external_catalogs: true
I20260202 02:52:25.473016 526 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_d6ba351d68ec4a10939a09519b7d4b67" } modify_external_catalogs: true
W20260202 02:52:25.473320 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8a25e41d22cf497aad652a6171bd142c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:25.473552 1865 raft_consensus.cc:493] T 13cd1383348b458894fdecc22bf24b74 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:25.473675 1865 raft_consensus.cc:515] T 13cd1383348b458894fdecc22bf24b74 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.474344 1865 leader_election.cc:290] T 13cd1383348b458894fdecc22bf24b74 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:25.475283 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "13cd1383348b458894fdecc22bf24b74" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:25.475907 857 leader_election.cc:343] T 13cd1383348b458894fdecc22bf24b74 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:25.477542 523 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_499eb7d77ae34d88ad956bf4a5c55648" } new_table_name: "default.Table_0cd8f9947fbd430f8992cc1543aed9f7" modify_external_catalogs: true
I20260202 02:52:25.477486 520 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_8c47663f416b467cad37110bd09d85ad" } new_table_name: "default.Table_03cb720f804b44e2b29f8cc47336b2db" modify_external_catalogs: true
I20260202 02:52:25.482066 1175 tablet_bootstrap.cc:492] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:25.483757 1175 ts_tablet_manager.cc:1403] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.039s user 0.003s sys 0.001s
I20260202 02:52:25.483734 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_a115f2d803494367ad12eaafeb1cdf2a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:25.484431 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a115f2d803494367ad12eaafeb1cdf2a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:25.484627 1175 raft_consensus.cc:359] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:25.484805 1175 raft_consensus.cc:385] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:25.484853 1175 raft_consensus.cc:740] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:25.486640 528 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_db26223b0d604aeabd9431ef5f27da88" } modify_external_catalogs: true
I20260202 02:52:25.487061 528 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_db26223b0d604aeabd9431ef5f27da88" } modify_external_catalogs: true
I20260202 02:52:25.488372 1182 ts_tablet_manager.cc:1929] T a1f39e5b02f747819e636cddb4292eb6 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:25.488500 1182 log.cc:1199] T a1f39e5b02f747819e636cddb4292eb6 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/a1f39e5b02f747819e636cddb4292eb6
I20260202 02:52:25.488809 1182 ts_tablet_manager.cc:1950] T a1f39e5b02f747819e636cddb4292eb6 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:25.489410 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet a1f39e5b02f747819e636cddb4292eb6 (table default.Table_61c2ba441aa746edbb73f8b5de4204ba [id=abceac85d3524c96834546f67f41f933]) successfully deleted
I20260202 02:52:25.490056 1175 consensus_queue.cc:260] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:25.491158 1175 ts_tablet_manager.cc:1434] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260202 02:52:25.491432 1182 tablet_replica.cc:333] T 13cd1383348b458894fdecc22bf24b74 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:25.491768 1182 raft_consensus.cc:2243] T 13cd1383348b458894fdecc22bf24b74 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:25.491921 1175 tablet_bootstrap.cc:492] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:25.492216 1182 raft_consensus.cc:2272] T 13cd1383348b458894fdecc22bf24b74 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:25.493034 1182 ts_tablet_manager.cc:1916] T 13cd1383348b458894fdecc22bf24b74 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:25.500891 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "38301e13e17c41fcae912c4b38c26a45" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:25.509689 859 leader_election.cc:343] T 38301e13e17c41fcae912c4b38c26a45 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:25.510499 859 leader_election.cc:304] T 38301e13e17c41fcae912c4b38c26a45 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:25.510784 1175 tablet_bootstrap.cc:654] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:25.511096 1936 raft_consensus.cc:2749] T 38301e13e17c41fcae912c4b38c26a45 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:25.512310 2032 log.cc:927] Time spent T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d: Append to log took a long time: real 0.053s user 0.000s sys 0.000s
I20260202 02:52:25.515437 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 302) took 5201 ms. Trace:
I20260202 02:52:25.515566 1180 rpcz_store.cc:276] 0202 02:52:20.313569 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:20.314716 (+ 1147us) service_pool.cc:225] Handling call
0202 02:52:25.418704 (+5103988us) ts_tablet_manager.cc:1151] Deleting tablet c0937b6910534ddc8a2217f0cbd539c5
0202 02:52:25.418722 (+ 18us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:25.445494 (+ 26772us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:25.449278 (+ 3784us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:25.515363 (+ 66085us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5103646}
I20260202 02:52:25.516130 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet c0937b6910534ddc8a2217f0cbd539c5 (table default.Table_f48ed1c7e02349adad7ce22c0e51ed81 [id=46b5d47377474ff9bac2acdb26b08c65]) successfully deleted
I20260202 02:52:25.516547 1180 tablet_replica.cc:333] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:25.516757 1180 raft_consensus.cc:2243] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:25.517462 1180 raft_consensus.cc:2272] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:25.522625 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "13cd1383348b458894fdecc22bf24b74" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:25.525085 859 leader_election.cc:343] T 13cd1383348b458894fdecc22bf24b74 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:25.525333 859 leader_election.cc:304] T 13cd1383348b458894fdecc22bf24b74 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
W20260202 02:52:25.525560 859 raft_consensus.cc:2686] T 13cd1383348b458894fdecc22bf24b74 P 1521a6d262b7451bbef8debe970dd6b5: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260202 02:52:25.532912 1174 tablet_bootstrap.cc:492] T 25765fb687294e9db1289f76530dd877 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:25.535981 1174 tablet_bootstrap.cc:654] T 25765fb687294e9db1289f76530dd877 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:25.539299 1894 raft_consensus.cc:493] T 042d1ebf357b4abe8814a407891ce265 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:25.539482 1894 raft_consensus.cc:515] T 042d1ebf357b4abe8814a407891ce265 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.539883 1180 ts_tablet_manager.cc:1916] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:25.541030 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "042d1ebf357b4abe8814a407891ce265" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:25.542631 1894 leader_election.cc:290] T 042d1ebf357b4abe8814a407891ce265 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:25.543272 998 leader_election.cc:343] T 042d1ebf357b4abe8814a407891ce265 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:25.543640 999 leader_election.cc:343] T 042d1ebf357b4abe8814a407891ce265 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 042d1ebf357b4abe8814a407891ce265
I20260202 02:52:25.543780 999 leader_election.cc:304] T 042d1ebf357b4abe8814a407891ce265 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:25.544411 1894 raft_consensus.cc:2749] T 042d1ebf357b4abe8814a407891ce265 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:25.566540 1175 tablet_bootstrap.cc:492] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:25.566932 1175 ts_tablet_manager.cc:1403] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.075s user 0.004s sys 0.003s
I20260202 02:52:25.569420 1175 raft_consensus.cc:359] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:25.569645 1175 raft_consensus.cc:385] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:25.569712 1175 raft_consensus.cc:740] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:25.569926 1175 consensus_queue.cc:260] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:25.575755 1175 ts_tablet_manager.cc:1434] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260202 02:52:25.576155 1175 tablet_bootstrap.cc:492] T 75569511179b41038e521fa3346a92ae P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:25.576422 1882 ts_tablet_manager.cc:1929] T eb0ac55fbb534b29a86d6b3e8aa72ebc P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:25.576690 1882 log.cc:1199] T eb0ac55fbb534b29a86d6b3e8aa72ebc P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/eb0ac55fbb534b29a86d6b3e8aa72ebc
I20260202 02:52:25.577219 1882 ts_tablet_manager.cc:1950] T eb0ac55fbb534b29a86d6b3e8aa72ebc P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:25.578135 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 375) took 2287 ms. Trace:
I20260202 02:52:25.585830 1936 raft_consensus.cc:493] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:25.586007 1936 raft_consensus.cc:515] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:25.586501 1175 tablet_bootstrap.cc:654] T 75569511179b41038e521fa3346a92ae P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:25.586647 1936 leader_election.cc:290] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:25.584729 2038 raft_consensus.cc:493] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:25.589726 2038 raft_consensus.cc:515] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:25.590492 2038 leader_election.cc:290] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:25.588939 1058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "874ca4ba4ebc4721bb4b60cba8e01841" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:25.589535 1173 tablet_bootstrap.cc:492] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:25.591614 1173 ts_tablet_manager.cc:1403] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.405s user 0.005s sys 0.000s
W20260202 02:52:25.591606 859 leader_election.cc:343] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:25.592569 1882 rpcz_store.cc:276] 0202 02:52:23.290180 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.291099 (+ 919us) service_pool.cc:225] Handling call
0202 02:52:25.429873 (+2138774us) ts_tablet_manager.cc:1151] Deleting tablet eb0ac55fbb534b29a86d6b3e8aa72ebc
0202 02:52:25.429885 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:25.547311 (+117426us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:25.575765 (+ 28454us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:25.578076 (+ 2311us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":13652,"tablet-delete.queue_time_us":2138615}
I20260202 02:52:25.592743 917 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "20ea39bc5d3d46f5a4ba427277d6c6a3" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:25.592927 917 raft_consensus.cc:2468] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:25.593165 1173 raft_consensus.cc:359] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.593353 1173 raft_consensus.cc:385] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:25.593411 1173 raft_consensus.cc:740] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:25.593629 1173 consensus_queue.cc:260] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.593822 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet eb0ac55fbb534b29a86d6b3e8aa72ebc (table default.Table_8f23e3356aa14793a1aad274d2b845eb [id=21c284c2be45496ca760633f24f090d2]) successfully deleted
I20260202 02:52:25.594094 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 370) took 2348 ms. Trace:
I20260202 02:52:25.594190 1882 rpcz_store.cc:276] 0202 02:52:23.245585 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.245660 (+ 75us) service_pool.cc:225] Handling call
0202 02:52:25.593888 (+2348228us) ts_tablet_manager.cc:1151] Deleting tablet dada209d982d48f3b74a49905f19c5b6
0202 02:52:25.593901 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:25.594039 (+ 138us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2279384}
I20260202 02:52:25.594543 999 leader_election.cc:304] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
W20260202 02:52:25.594941 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet dada209d982d48f3b74a49905f19c5b6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dada209d982d48f3b74a49905f19c5b6 already in progress: creating tablet
I20260202 02:52:25.595619 2028 tablet.cc:1722] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260202 02:52:25.597402 1882 tablet_replica.cc:333] T 75b97d7e848241cf8d894b0ae92a8f8f P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:25.597658 1882 raft_consensus.cc:2243] T 75b97d7e848241cf8d894b0ae92a8f8f P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:25.598456 1764 raft_consensus.cc:2804] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:25.598567 1764 raft_consensus.cc:493] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:25.598673 1764 raft_consensus.cc:3060] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:25.599184 1173 ts_tablet_manager.cc:1434] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260202 02:52:25.599470 1882 raft_consensus.cc:2272] T 75b97d7e848241cf8d894b0ae92a8f8f P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:25.600364 1173 tablet_bootstrap.cc:492] T 68823eff1bc04108ba83487605aec0ea P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:25.601567 1764 raft_consensus.cc:515] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:25.602242 1882 ts_tablet_manager.cc:1916] T 75b97d7e848241cf8d894b0ae92a8f8f P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:25.602454 2022 tablet.cc:1722] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260202 02:52:25.602689 1764 leader_election.cc:290] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:25.603075 1175 tablet_bootstrap.cc:492] T 75569511179b41038e521fa3346a92ae P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:25.603215 1175 ts_tablet_manager.cc:1403] T 75569511179b41038e521fa3346a92ae P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.027s user 0.006s sys 0.000s
I20260202 02:52:25.603994 1175 raft_consensus.cc:359] T 75569511179b41038e521fa3346a92ae P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:25.604163 1175 raft_consensus.cc:385] T 75569511179b41038e521fa3346a92ae P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:25.604229 1175 raft_consensus.cc:740] T 75569511179b41038e521fa3346a92ae P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:25.604486 1175 consensus_queue.cc:260] T 75569511179b41038e521fa3346a92ae P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:25.604830 2015 tablet.cc:1722] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260202 02:52:25.605690 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "20ea39bc5d3d46f5a4ba427277d6c6a3" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:25.605907 1173 tablet_bootstrap.cc:654] T 68823eff1bc04108ba83487605aec0ea P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:25.605939 923 raft_consensus.cc:3060] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:25.607187 1175 ts_tablet_manager.cc:1434] T 75569511179b41038e521fa3346a92ae P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260202 02:52:25.607870 1175 tablet_bootstrap.cc:492] T 02b434fa058b45529ff3938b053263c6 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:25.611819 525 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 93f958924885457ab15f2aac5509e0aa
I20260202 02:52:25.612704 1032 tablet_service.cc:1552] Processing DeleteTablet for tablet 93f958924885457ab15f2aac5509e0aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:25 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:25.613294 1175 tablet_bootstrap.cc:654] T 02b434fa058b45529ff3938b053263c6 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:25.614578 525 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 25765fb687294e9db1289f76530dd877 (table default.Table_8c47663f416b467cad37110bd09d85ad [id=ea0c52a72da8422ba14345fbfc242d27]). Expected version 1 got 0
I20260202 02:52:25.615561 901 tablet_service.cc:1552] Processing DeleteTablet for tablet 93f958924885457ab15f2aac5509e0aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:25 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:25.620509 1882 ts_tablet_manager.cc:1929] T 75b97d7e848241cf8d894b0ae92a8f8f P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:25.625813 1182 ts_tablet_manager.cc:1929] T 13cd1383348b458894fdecc22bf24b74 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260202 02:52:25.626963 999 consensus_peers.cc:597] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1f54c1dfe0a942b3bafd85522953a032. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:25.628595 1882 log.cc:1199] T 75b97d7e848241cf8d894b0ae92a8f8f P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/75b97d7e848241cf8d894b0ae92a8f8f
I20260202 02:52:25.628949 1182 log.cc:1199] T 13cd1383348b458894fdecc22bf24b74 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/13cd1383348b458894fdecc22bf24b74
I20260202 02:52:25.629473 1882 ts_tablet_manager.cc:1950] T 75b97d7e848241cf8d894b0ae92a8f8f P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:25.629951 1182 ts_tablet_manager.cc:1950] T 13cd1383348b458894fdecc22bf24b74 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:25.631944 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 13cd1383348b458894fdecc22bf24b74 (table default.Table_f92d6341db3143e490f6d64418ef6ff2 [id=ebc82aae55584b8ba2ba5c4223d9e258]) successfully deleted
I20260202 02:52:25.632535 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "20ea39bc5d3d46f5a4ba427277d6c6a3" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:25.632938 1175 tablet_bootstrap.cc:492] T 02b434fa058b45529ff3938b053263c6 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:25.633075 1175 ts_tablet_manager.cc:1403] T 02b434fa058b45529ff3938b053263c6 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.000s
I20260202 02:52:25.633188 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "874ca4ba4ebc4721bb4b60cba8e01841" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:25.633901 857 leader_election.cc:343] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:25.634018 1175 raft_consensus.cc:359] T 02b434fa058b45529ff3938b053263c6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:25.634192 1175 raft_consensus.cc:385] T 02b434fa058b45529ff3938b053263c6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:25.634238 1175 raft_consensus.cc:740] T 02b434fa058b45529ff3938b053263c6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:25.634408 1175 consensus_queue.cc:260] T 02b434fa058b45529ff3938b053263c6 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:25.634887 1175 ts_tablet_manager.cc:1434] T 02b434fa058b45529ff3938b053263c6 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:25.634050 857 leader_election.cc:304] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:25.635176 1175 tablet_bootstrap.cc:492] T fe53a8bdbd13417f9ce195c54a52a93c P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:25.635394 1936 raft_consensus.cc:2749] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:25.637109 998 leader_election.cc:343] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:25.637545 1175 tablet_bootstrap.cc:654] T fe53a8bdbd13417f9ce195c54a52a93c P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:25.640420 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "20ea39bc5d3d46f5a4ba427277d6c6a3" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
W20260202 02:52:25.642338 998 leader_election.cc:343] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:25.643774 1182 tablet_replica.cc:333] T c25e329023c04f609d8a743b4f7a95a4 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:25.643981 1182 raft_consensus.cc:2243] T c25e329023c04f609d8a743b4f7a95a4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:25.644140 1182 raft_consensus.cc:2272] T c25e329023c04f609d8a743b4f7a95a4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:25.644724 1182 ts_tablet_manager.cc:1916] T c25e329023c04f609d8a743b4f7a95a4 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:25.645731 1860 raft_consensus.cc:493] T 79f694b740954d429145fba9157338ca P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:25.645975 1860 raft_consensus.cc:515] T 79f694b740954d429145fba9157338ca P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.647169 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79f694b740954d429145fba9157338ca" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:25.647876 998 leader_election.cc:343] T 79f694b740954d429145fba9157338ca P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:25.647964 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 373) took 2357 ms. Trace:
I20260202 02:52:25.648070 1882 rpcz_store.cc:276] 0202 02:52:23.290008 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.290760 (+ 752us) service_pool.cc:225] Handling call
0202 02:52:25.597270 (+2306510us) ts_tablet_manager.cc:1151] Deleting tablet 75b97d7e848241cf8d894b0ae92a8f8f
0202 02:52:25.597282 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:25.617392 (+ 20110us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:25.620357 (+ 2965us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:25.647921 (+ 27564us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1562,"tablet-delete.queue_time_us":2266225}
I20260202 02:52:25.649617 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 75b97d7e848241cf8d894b0ae92a8f8f (table default.Table_5e366ca4b9744597add5ca93f7fe7850 [id=f2e865926cf5453292c34daa4af8be2a]) successfully deleted
I20260202 02:52:25.650059 1882 tablet_replica.cc:333] T ffd80badce1048f3ac95366cad05ae53 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:25.659929 1860 leader_election.cc:290] T 79f694b740954d429145fba9157338ca P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:25.660089 999 leader_election.cc:343] T 79f694b740954d429145fba9157338ca P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 79f694b740954d429145fba9157338ca
I20260202 02:52:25.660259 999 leader_election.cc:304] T 79f694b740954d429145fba9157338ca P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:25.660511 750 tablet_service.cc:1552] Processing DeleteTablet for tablet 93f958924885457ab15f2aac5509e0aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:25 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:25.659988 1882 raft_consensus.cc:2243] T ffd80badce1048f3ac95366cad05ae53 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:25.661032 1882 raft_consensus.cc:2272] T ffd80badce1048f3ac95366cad05ae53 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:25.661487 2019 raft_consensus.cc:493] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:25.661685 2019 raft_consensus.cc:515] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:25.663112 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26b890ed87c3471b9fc0b172c90e0a11" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:25.663798 857 leader_election.cc:343] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:25.664062 2019 leader_election.cc:290] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:25.664310 2038 raft_consensus.cc:2749] T 79f694b740954d429145fba9157338ca P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:25.664361 1860 raft_consensus.cc:493] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:25.664649 1058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26b890ed87c3471b9fc0b172c90e0a11" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:25.664876 1860 raft_consensus.cc:515] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:25.665241 859 leader_election.cc:343] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:25.665336 859 leader_election.cc:304] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:25.665766 2019 raft_consensus.cc:2749] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:25.666018 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8774c6bc0fe9414ea2f17ed859d9392a" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:25.666652 998 leader_election.cc:343] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:25.667186 1860 leader_election.cc:290] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:25.667459 921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8774c6bc0fe9414ea2f17ed859d9392a" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:25.667642 921 raft_consensus.cc:2468] T 8774c6bc0fe9414ea2f17ed859d9392a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:25.667855 1182 ts_tablet_manager.cc:1929] T c25e329023c04f609d8a743b4f7a95a4 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:25.667963 1182 log.cc:1199] T c25e329023c04f609d8a743b4f7a95a4 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/c25e329023c04f609d8a743b4f7a95a4
I20260202 02:52:25.668056 999 leader_election.cc:304] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:25.668275 1182 ts_tablet_manager.cc:1950] T c25e329023c04f609d8a743b4f7a95a4 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:25.668849 1860 raft_consensus.cc:2804] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:25.668977 1860 raft_consensus.cc:493] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:25.669083 1860 raft_consensus.cc:3060] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:25.671516 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet c25e329023c04f609d8a743b4f7a95a4 (table default.Table_f034976ca8c24ddd84eba68e745b6821 [id=81e199e091b149a0a103eb3b8a232983]) successfully deleted
I20260202 02:52:25.672345 1860 raft_consensus.cc:515] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.673372 1182 tablet_replica.cc:333] T 8774c6bc0fe9414ea2f17ed859d9392a P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:25.673673 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8774c6bc0fe9414ea2f17ed859d9392a" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
W20260202 02:52:25.674558 998 leader_election.cc:343] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:25.674587 1182 raft_consensus.cc:2243] T 8774c6bc0fe9414ea2f17ed859d9392a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:25.664314 1882 ts_tablet_manager.cc:1916] T ffd80badce1048f3ac95366cad05ae53 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:25.674785 1182 raft_consensus.cc:2272] T 8774c6bc0fe9414ea2f17ed859d9392a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:25.675341 1182 ts_tablet_manager.cc:1916] T 8774c6bc0fe9414ea2f17ed859d9392a P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:25.680315 923 raft_consensus.cc:2468] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:25.680931 999 leader_election.cc:304] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:25.681473 2038 raft_consensus.cc:2804] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:25.681622 2038 raft_consensus.cc:697] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:25.681911 2038 consensus_queue.cc:237] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
W20260202 02:52:25.682843 519 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 79c73f4a92044bafb5b8384229cb3a4d consensus state has no leader
I20260202 02:52:25.690367 1174 tablet_bootstrap.cc:492] T 25765fb687294e9db1289f76530dd877 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:25.690549 1174 ts_tablet_manager.cc:1403] T 25765fb687294e9db1289f76530dd877 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.158s user 0.002s sys 0.002s
I20260202 02:52:25.692108 1174 raft_consensus.cc:359] T 25765fb687294e9db1289f76530dd877 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.692345 1174 raft_consensus.cc:385] T 25765fb687294e9db1289f76530dd877 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:25.692430 1174 raft_consensus.cc:740] T 25765fb687294e9db1289f76530dd877 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:25.692602 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_d46939f310bc46c3a2e99a85374c4b5e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:25.692687 1174 consensus_queue.cc:260] T 25765fb687294e9db1289f76530dd877 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:25.693182 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d46939f310bc46c3a2e99a85374c4b5e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:25.693229 1174 ts_tablet_manager.cc:1434] T 25765fb687294e9db1289f76530dd877 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260202 02:52:25.693526 1174 tablet_bootstrap.cc:492] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:25.699257 519 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet 757492288d7c44fb91b25b7e6e21e01b (table default.Table_7c01ea0d668a490099dbd42db17162ec [id=02e9e243f710428091de2f9497532c37]). Expected version 1 got 0
I20260202 02:52:25.701489 901 tablet_service.cc:1552] Processing DeleteTablet for tablet d30557fbe9084cd3860820f69dda8752 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:25.703784 756 tablet_service.cc:1552] Processing DeleteTablet for tablet f93e8d3c4faf48eaa98c3cbf88d585e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:25.704093 1174 tablet_bootstrap.cc:654] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:25.705696 1182 ts_tablet_manager.cc:1929] T 8774c6bc0fe9414ea2f17ed859d9392a P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:25.707295 1182 log.cc:1199] T 8774c6bc0fe9414ea2f17ed859d9392a P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/8774c6bc0fe9414ea2f17ed859d9392a
I20260202 02:52:25.707615 1182 ts_tablet_manager.cc:1950] T 8774c6bc0fe9414ea2f17ed859d9392a P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:25.711079 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 8774c6bc0fe9414ea2f17ed859d9392a (table default.Table_0eb450e3edbb4a4ba0906567f831830c [id=8918ec9e2cdf4f2995b2b4baf21297d7]) successfully deleted
W20260202 02:52:25.712649 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet a5f864fd8a3d4c4a83e7f68abb8a0a83 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a5f864fd8a3d4c4a83e7f68abb8a0a83 already in progress: creating tablet
I20260202 02:52:25.714331 1182 tablet_replica.cc:333] T 38301e13e17c41fcae912c4b38c26a45 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:25.714547 1182 raft_consensus.cc:2243] T 38301e13e17c41fcae912c4b38c26a45 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:25.716238 1182 raft_consensus.cc:2272] T 38301e13e17c41fcae912c4b38c26a45 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:25.720773 1182 ts_tablet_manager.cc:1916] T 38301e13e17c41fcae912c4b38c26a45 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:25.725729 1174 tablet_bootstrap.cc:492] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:25.725950 1174 ts_tablet_manager.cc:1403] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.032s user 0.002s sys 0.003s
I20260202 02:52:25.728073 1174 raft_consensus.cc:359] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:25.728308 1174 raft_consensus.cc:385] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:25.728425 1174 raft_consensus.cc:740] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:25.728682 1174 consensus_queue.cc:260] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:25.729208 1174 ts_tablet_manager.cc:1434] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20260202 02:52:25.729558 1174 tablet_bootstrap.cc:492] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:25.731916 1174 tablet_bootstrap.cc:654] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:25.734665 2038 raft_consensus.cc:493] T 2b8ba9dd450948ada395207ae1c5b325 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:25.734985 2038 raft_consensus.cc:515] T 2b8ba9dd450948ada395207ae1c5b325 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.736390 2047 raft_consensus.cc:493] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:25.736557 2047 raft_consensus.cc:515] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:25.737228 2047 leader_election.cc:290] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
W20260202 02:52:25.738816 714 leader_election.cc:343] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: d64f42d187bc4c9f9381c931449ec0cc
I20260202 02:52:25.740058 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2b8ba9dd450948ada395207ae1c5b325" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:25.740728 998 leader_election.cc:343] T 2b8ba9dd450948ada395207ae1c5b325 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:25.741973 2038 leader_election.cc:290] T 2b8ba9dd450948ada395207ae1c5b325 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:25.742653 999 leader_election.cc:343] T 2b8ba9dd450948ada395207ae1c5b325 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 2b8ba9dd450948ada395207ae1c5b325
I20260202 02:52:25.742787 999 leader_election.cc:304] T 2b8ba9dd450948ada395207ae1c5b325 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:25.746969 1182 ts_tablet_manager.cc:1929] T 38301e13e17c41fcae912c4b38c26a45 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:25.747100 1182 log.cc:1199] T 38301e13e17c41fcae912c4b38c26a45 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/38301e13e17c41fcae912c4b38c26a45
I20260202 02:52:25.747370 1182 ts_tablet_manager.cc:1950] T 38301e13e17c41fcae912c4b38c26a45 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:25.752493 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d64f42d187bc4c9f9381c931449ec0cc" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:25.752770 1062 raft_consensus.cc:2468] T d64f42d187bc4c9f9381c931449ec0cc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:25.753465 715 leader_election.cc:304] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:25.753595 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 38301e13e17c41fcae912c4b38c26a45 (table default.Table_80912cc76f934f139da7f866546ae97a [id=2f3bf7e0cf4c41ac8a2554360b57214a]) successfully deleted
I20260202 02:52:25.753932 2047 raft_consensus.cc:2804] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:25.754053 2047 raft_consensus.cc:493] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:25.754128 2047 raft_consensus.cc:3060] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:25.754611 523 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fbcb4a522df648ba9542250450c08f1b consensus state has no leader
I20260202 02:52:25.755065 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_d816f250a1204733bad1ba0967a17297"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:25.755654 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d816f250a1204733bad1ba0967a17297 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:25.755970 1182 tablet_replica.cc:333] T 06b80a1c9251420cbc16fc95ae98aca9 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:25.756170 1182 raft_consensus.cc:2243] T 06b80a1c9251420cbc16fc95ae98aca9 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:25.756309 1182 raft_consensus.cc:2272] T 06b80a1c9251420cbc16fc95ae98aca9 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:25.756872 1182 ts_tablet_manager.cc:1916] T 06b80a1c9251420cbc16fc95ae98aca9 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:25.757117 2047 raft_consensus.cc:515] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:25.758319 2047 leader_election.cc:290] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:25.759059 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d64f42d187bc4c9f9381c931449ec0cc" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:25.759303 1062 raft_consensus.cc:3060] T d64f42d187bc4c9f9381c931449ec0cc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:25.760403 524 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet 25765fb687294e9db1289f76530dd877 (table default.Table_8c47663f416b467cad37110bd09d85ad [id=ea0c52a72da8422ba14345fbfc242d27]). Expected version 1 got 0
I20260202 02:52:25.761947 1062 raft_consensus.cc:2468] T d64f42d187bc4c9f9381c931449ec0cc P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:25.762962 715 leader_election.cc:304] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:25.763445 2047 raft_consensus.cc:2804] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:25.763644 2047 raft_consensus.cc:697] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:25.764117 2047 consensus_queue.cc:237] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:25.769457 1277 raft_consensus.cc:493] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:25.769649 1174 tablet_bootstrap.cc:492] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:25.769701 1277 raft_consensus.cc:515] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.769847 1174 ts_tablet_manager.cc:1403] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.040s user 0.005s sys 0.003s
I20260202 02:52:25.770372 1277 leader_election.cc:290] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:25.770669 1174 raft_consensus.cc:359] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.770851 1174 raft_consensus.cc:385] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:25.770931 1174 raft_consensus.cc:740] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:25.771121 1174 consensus_queue.cc:260] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.771713 1174 ts_tablet_manager.cc:1434] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260202 02:52:25.772047 1174 tablet_bootstrap.cc:492] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:25.772464 1721 raft_consensus.cc:2749] T 2b8ba9dd450948ada395207ae1c5b325 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:25.778668 1182 ts_tablet_manager.cc:1929] T 06b80a1c9251420cbc16fc95ae98aca9 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:25.778831 1182 log.cc:1199] T 06b80a1c9251420cbc16fc95ae98aca9 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/06b80a1c9251420cbc16fc95ae98aca9
I20260202 02:52:25.779106 1182 ts_tablet_manager.cc:1950] T 06b80a1c9251420cbc16fc95ae98aca9 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:25.780469 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 06b80a1c9251420cbc16fc95ae98aca9 (table default.Table_84d6693620cc468e82d5cb1ec6cc69ea [id=f6625b21b6064828b11acdfdb694cd71]) successfully deleted
I20260202 02:52:25.782300 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "94bada52db6a47179415d302b541d01f" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:25.784147 1062 raft_consensus.cc:2393] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f442ef85029461091de847118c34c0d in current term 1: Already voted for candidate b1f05820b6034228bdad884cc348716d in this term.
W20260202 02:52:25.785575 714 leader_election.cc:343] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 94bada52db6a47179415d302b541d01f
I20260202 02:52:25.785756 714 leader_election.cc:304] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:25.786079 1277 raft_consensus.cc:3060] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:25.788678 1174 tablet_bootstrap.cc:654] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:25.792351 1277 raft_consensus.cc:2749] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:25.793211 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 7f9dfac19c04475aadb45e95a6474eb4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7f9dfac19c04475aadb45e95a6474eb4 already in progress: creating tablet
W20260202 02:52:25.794232 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet a0b5d82e5fb94b60b4aa324b978a5378 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a0b5d82e5fb94b60b4aa324b978a5378 already in progress: creating tablet
W20260202 02:52:25.794956 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet ac2ab12f5040488aa306cb69b107b485 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ac2ab12f5040488aa306cb69b107b485 already in progress: creating tablet
W20260202 02:52:25.796082 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 361c0c3832614ec78695c28720b841e0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 361c0c3832614ec78695c28720b841e0 already in progress: creating tablet
W20260202 02:52:25.797089 714 leader_election.cc:343] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: d64f42d187bc4c9f9381c931449ec0cc
W20260202 02:52:25.797873 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 3706562a22634eb6b72e36bb6a2d1ae5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3706562a22634eb6b72e36bb6a2d1ae5 already in progress: creating tablet
W20260202 02:52:25.798553 999 leader_election.cc:343] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 8774c6bc0fe9414ea2f17ed859d9392a
I20260202 02:52:25.798689 999 leader_election.cc:304] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
W20260202 02:52:25.798902 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 7921daa000f54c8dbed04473e21cf0a5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7921daa000f54c8dbed04473e21cf0a5 already in progress: creating tablet
I20260202 02:52:25.799083 2038 raft_consensus.cc:2749] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260202 02:52:25.799618 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 474df41e4fe24a818fa45a4ce84d5474 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 474df41e4fe24a818fa45a4ce84d5474 already in progress: creating tablet
I20260202 02:52:25.799520 1860 leader_election.cc:290] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:25.800159 1175 tablet_bootstrap.cc:492] T fe53a8bdbd13417f9ce195c54a52a93c P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
W20260202 02:52:25.800873 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet ccd6bc8bd0504bf18d108c73685ef9ff because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ccd6bc8bd0504bf18d108c73685ef9ff already in progress: creating tablet
I20260202 02:52:25.801836 1173 tablet_bootstrap.cc:492] T 68823eff1bc04108ba83487605aec0ea P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:25.800316 1175 ts_tablet_manager.cc:1403] T fe53a8bdbd13417f9ce195c54a52a93c P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.165s user 0.011s sys 0.031s
I20260202 02:52:25.803054 1175 raft_consensus.cc:359] T fe53a8bdbd13417f9ce195c54a52a93c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:25.803772 1173 ts_tablet_manager.cc:1403] T 68823eff1bc04108ba83487605aec0ea P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.204s user 0.020s sys 0.026s
W20260202 02:52:25.804004 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 93f958924885457ab15f2aac5509e0aa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 93f958924885457ab15f2aac5509e0aa already in progress: creating tablet
I20260202 02:52:25.804862 1175 raft_consensus.cc:385] T fe53a8bdbd13417f9ce195c54a52a93c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:25.804952 1175 raft_consensus.cc:740] T fe53a8bdbd13417f9ce195c54a52a93c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:25.803561 1182 tablet_replica.cc:333] T d30557fbe9084cd3860820f69dda8752 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:25.805256 1182 raft_consensus.cc:2243] T d30557fbe9084cd3860820f69dda8752 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:25.805416 1182 raft_consensus.cc:2272] T d30557fbe9084cd3860820f69dda8752 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:25.805617 1180 ts_tablet_manager.cc:1929] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260202 02:52:25.805837 1173 raft_consensus.cc:359] T 68823eff1bc04108ba83487605aec0ea P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.806782 1182 ts_tablet_manager.cc:1916] T d30557fbe9084cd3860820f69dda8752 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:25.808009 1180 log.cc:1199] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/ff6f5710de4b4c48948aacdf5be29e8b
I20260202 02:52:25.806952 1175 consensus_queue.cc:260] T fe53a8bdbd13417f9ce195c54a52a93c P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:25.810212 1175 ts_tablet_manager.cc:1434] T fe53a8bdbd13417f9ce195c54a52a93c P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260202 02:52:25.810518 1175 tablet_bootstrap.cc:492] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:25.811996 1173 raft_consensus.cc:385] T 68823eff1bc04108ba83487605aec0ea P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:25.812216 1173 raft_consensus.cc:740] T 68823eff1bc04108ba83487605aec0ea P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:25.812572 1173 consensus_queue.cc:260] T 68823eff1bc04108ba83487605aec0ea P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.813228 1173 ts_tablet_manager.cc:1434] T 68823eff1bc04108ba83487605aec0ea P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260202 02:52:25.817229 1173 tablet_bootstrap.cc:492] T c25e329023c04f609d8a743b4f7a95a4 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:25.821211 522 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ce37f495e88145148c5f83a15c901826" } modify_external_catalogs: true
I20260202 02:52:25.821437 522 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ce37f495e88145148c5f83a15c901826" } modify_external_catalogs: true
I20260202 02:52:25.823568 1175 tablet_bootstrap.cc:654] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:25.825743 1180 ts_tablet_manager.cc:1950] T ff6f5710de4b4c48948aacdf5be29e8b P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:25.826741 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 303) took 5427 ms. Trace:
I20260202 02:52:25.826876 1180 rpcz_store.cc:276] 0202 02:52:20.399550 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:20.399610 (+ 60us) service_pool.cc:225] Handling call
0202 02:52:25.516428 (+5116818us) ts_tablet_manager.cc:1151] Deleting tablet ff6f5710de4b4c48948aacdf5be29e8b
0202 02:52:25.516438 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:25.630993 (+114555us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:25.805434 (+174441us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:25.826693 (+ 21259us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5116584}
I20260202 02:52:25.827565 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet ff6f5710de4b4c48948aacdf5be29e8b (table default.Table_ad048ed673a5401b8ffa739c0f03b545 [id=f2276e1c2ca04a70804053da61dd3447]) successfully deleted
I20260202 02:52:25.831303 1182 ts_tablet_manager.cc:1929] T d30557fbe9084cd3860820f69dda8752 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:25.837145 1182 log.cc:1199] T d30557fbe9084cd3860820f69dda8752 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/d30557fbe9084cd3860820f69dda8752
I20260202 02:52:25.837707 1182 ts_tablet_manager.cc:1950] T d30557fbe9084cd3860820f69dda8752 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:25.835592 1175 tablet_bootstrap.cc:492] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:25.839241 1175 ts_tablet_manager.cc:1403] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.029s user 0.003s sys 0.002s
I20260202 02:52:25.839197 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet d30557fbe9084cd3860820f69dda8752 (table default.Table_0bf3a06700844105a02b98edd499e75a [id=966fa513d6e947cdb895edc05d26ff65]) successfully deleted
I20260202 02:52:25.840138 1175 raft_consensus.cc:359] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:25.840479 1175 raft_consensus.cc:385] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:25.840608 1175 raft_consensus.cc:740] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:25.841188 1175 consensus_queue.cc:260] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:25.841853 1173 tablet_bootstrap.cc:654] T c25e329023c04f609d8a743b4f7a95a4 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:25.845399 1277 consensus_queue.cc:1048] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:25.846323 1175 ts_tablet_manager.cc:1434] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260202 02:52:25.852469 1173 tablet_bootstrap.cc:492] T c25e329023c04f609d8a743b4f7a95a4 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:25.852826 1173 ts_tablet_manager.cc:1403] T c25e329023c04f609d8a743b4f7a95a4 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.036s user 0.006s sys 0.000s
I20260202 02:52:25.853740 1173 raft_consensus.cc:359] T c25e329023c04f609d8a743b4f7a95a4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.854048 1173 raft_consensus.cc:385] T c25e329023c04f609d8a743b4f7a95a4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:25.854161 1173 raft_consensus.cc:740] T c25e329023c04f609d8a743b4f7a95a4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:25.854486 1173 consensus_queue.cc:260] T c25e329023c04f609d8a743b4f7a95a4 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.854996 1175 tablet_bootstrap.cc:492] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:25.856567 1173 ts_tablet_manager.cc:1434] T c25e329023c04f609d8a743b4f7a95a4 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260202 02:52:25.859002 1175 tablet_bootstrap.cc:654] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:25.860232 1173 tablet_bootstrap.cc:492] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:25.862578 1173 tablet_bootstrap.cc:654] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:25.865614 1850 raft_consensus.cc:493] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:25.865844 1850 raft_consensus.cc:515] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.866508 1850 leader_election.cc:290] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:25.867345 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6154460838eb4e508f3646d3e3e4cf21" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:25.867595 1062 raft_consensus.cc:2468] T 6154460838eb4e508f3646d3e3e4cf21 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
W20260202 02:52:25.867619 714 leader_election.cc:343] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 6154460838eb4e508f3646d3e3e4cf21
I20260202 02:52:25.868209 715 leader_election.cc:304] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:25.868741 1850 raft_consensus.cc:2804] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:25.868850 1850 raft_consensus.cc:493] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:25.868928 1850 raft_consensus.cc:3060] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:25.869153 512 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet cef87d5f74f14a6d86150958de7b1ace (table default.Table_6831562d21db463986dc58b0100e9792 [id=50e58db611f247ee9dcd5b31776ace05], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet cef87d5f74f14a6d86150958de7b1ace consensus state has no leader
W20260202 02:52:25.870872 714 consensus_peers.cc:597] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 929eb48881b141718873d0e359e768a8. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:25.872192 1850 raft_consensus.cc:515] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.873612 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6154460838eb4e508f3646d3e3e4cf21" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:25.873863 1062 raft_consensus.cc:3060] T 6154460838eb4e508f3646d3e3e4cf21 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:25.876812 1062 raft_consensus.cc:2468] T 6154460838eb4e508f3646d3e3e4cf21 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:25.877269 1850 leader_election.cc:290] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:25.877810 715 leader_election.cc:304] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:25.878051 1173 tablet_bootstrap.cc:492] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
W20260202 02:52:25.878127 714 leader_election.cc:343] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 6154460838eb4e508f3646d3e3e4cf21
I20260202 02:52:25.878190 1173 ts_tablet_manager.cc:1403] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.002s
I20260202 02:52:25.878491 1850 raft_consensus.cc:2804] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:25.878700 1850 raft_consensus.cc:697] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:25.878963 1173 raft_consensus.cc:359] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.879252 1173 raft_consensus.cc:385] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:25.879401 1173 raft_consensus.cc:740] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:25.879482 1850 consensus_queue.cc:237] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.879717 1173 consensus_queue.cc:260] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.881525 1173 ts_tablet_manager.cc:1434] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260202 02:52:25.882388 1173 tablet_bootstrap.cc:492] T a1f39e5b02f747819e636cddb4292eb6 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:25.885237 1173 tablet_bootstrap.cc:654] T a1f39e5b02f747819e636cddb4292eb6 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:25.899550 528 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 47e50559a7c04ae080fcc67342c631c9
I20260202 02:52:25.900647 1032 tablet_service.cc:1552] Processing DeleteTablet for tablet 47e50559a7c04ae080fcc67342c631c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:25 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:25.901007 901 tablet_service.cc:1552] Processing DeleteTablet for tablet 47e50559a7c04ae080fcc67342c631c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:25 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:25.901556 750 tablet_service.cc:1552] Processing DeleteTablet for tablet 47e50559a7c04ae080fcc67342c631c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:25 UTC) from {username='slave'} at 127.0.0.1:42950
W20260202 02:52:25.901966 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 47e50559a7c04ae080fcc67342c631c9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 47e50559a7c04ae080fcc67342c631c9 already in progress: creating tablet
I20260202 02:52:25.902082 1173 tablet_bootstrap.cc:492] T a1f39e5b02f747819e636cddb4292eb6 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:25.902247 1173 ts_tablet_manager.cc:1403] T a1f39e5b02f747819e636cddb4292eb6 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.008s
I20260202 02:52:25.903072 1173 raft_consensus.cc:359] T a1f39e5b02f747819e636cddb4292eb6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.903350 1173 raft_consensus.cc:385] T a1f39e5b02f747819e636cddb4292eb6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:25.903551 1173 raft_consensus.cc:740] T a1f39e5b02f747819e636cddb4292eb6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:25.903885 1173 consensus_queue.cc:260] T a1f39e5b02f747819e636cddb4292eb6 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.904510 1173 ts_tablet_manager.cc:1434] T a1f39e5b02f747819e636cddb4292eb6 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:25.904877 1173 tablet_bootstrap.cc:492] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:25.906823 1173 tablet_bootstrap.cc:654] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:25.907931 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_61d946ae91ff440c9960b0136900fa6e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:25.908664 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_61d946ae91ff440c9960b0136900fa6e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:25.910822 2019 raft_consensus.cc:493] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:25.910982 2019 raft_consensus.cc:515] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:25.911603 2019 leader_election.cc:290] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:25.912747 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "16b47a2831f54d809ed0b16deaec7e3d" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:25.913692 859 leader_election.cc:343] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:25.913755 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "16b47a2831f54d809ed0b16deaec7e3d" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:25.914402 857 leader_election.cc:343] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:25.914521 857 leader_election.cc:304] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:25.915544 2019 raft_consensus.cc:2749] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:25.926704 1173 tablet_bootstrap.cc:492] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:25.927013 1173 ts_tablet_manager.cc:1403] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.022s user 0.009s sys 0.010s
I20260202 02:52:25.928166 1173 raft_consensus.cc:359] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.928489 1173 raft_consensus.cc:385] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:25.928623 1173 raft_consensus.cc:740] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:25.928961 1173 consensus_queue.cc:260] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.930045 1173 ts_tablet_manager.cc:1434] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:25.930856 1173 tablet_bootstrap.cc:492] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:25.887173 1882 ts_tablet_manager.cc:1929] T ffd80badce1048f3ac95366cad05ae53 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:25.890425 2038 raft_consensus.cc:493] T 68823eff1bc04108ba83487605aec0ea P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:25.932543 2038 raft_consensus.cc:515] T 68823eff1bc04108ba83487605aec0ea P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.932976 1174 tablet_bootstrap.cc:492] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:25.933143 1174 ts_tablet_manager.cc:1403] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.161s user 0.002s sys 0.042s
I20260202 02:52:25.933199 2038 leader_election.cc:290] T 68823eff1bc04108ba83487605aec0ea P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:25.933315 1173 tablet_bootstrap.cc:654] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:25.933972 1174 raft_consensus.cc:359] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.934224 1174 raft_consensus.cc:385] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:25.934314 1174 raft_consensus.cc:740] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:25.934296 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "68823eff1bc04108ba83487605aec0ea" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:25.934616 1174 consensus_queue.cc:260] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:25.934803 999 leader_election.cc:343] T 68823eff1bc04108ba83487605aec0ea P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 68823eff1bc04108ba83487605aec0ea
I20260202 02:52:25.935176 1174 ts_tablet_manager.cc:1434] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20260202 02:52:25.935354 998 leader_election.cc:343] T 68823eff1bc04108ba83487605aec0ea P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:25.935470 998 leader_election.cc:304] T 68823eff1bc04108ba83487605aec0ea P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:25.935523 1174 tablet_bootstrap.cc:492] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:25.935953 2038 raft_consensus.cc:2749] T 68823eff1bc04108ba83487605aec0ea P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:25.936316 1882 log.cc:1199] T ffd80badce1048f3ac95366cad05ae53 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/ffd80badce1048f3ac95366cad05ae53
I20260202 02:52:25.936811 1882 ts_tablet_manager.cc:1950] T ffd80badce1048f3ac95366cad05ae53 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:25.937386 1174 tablet_bootstrap.cc:654] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:25.938567 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 377) took 2648 ms. Trace:
I20260202 02:52:25.938745 1882 rpcz_store.cc:276] 0202 02:52:23.290443 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.291282 (+ 839us) service_pool.cc:225] Handling call
0202 02:52:25.649723 (+2358441us) ts_tablet_manager.cc:1151] Deleting tablet ffd80badce1048f3ac95366cad05ae53
0202 02:52:25.649998 (+ 275us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:25.677873 (+ 27875us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:25.887017 (+209144us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:25.938497 (+ 51480us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2307868}
I20260202 02:52:25.939374 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet ffd80badce1048f3ac95366cad05ae53 (table default.Table_38c5bca682e343c1be1ebdc253c9e848 [id=6f44bec7f8ea4d2abed8c0fd603c6569]) successfully deleted
I20260202 02:52:25.939447 1882 tablet_replica.cc:333] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:25.939735 1882 raft_consensus.cc:2243] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:25.940173 1882 raft_consensus.cc:2272] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:25.940929 1882 ts_tablet_manager.cc:1916] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:25.941797 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_569ab9ec7c9c4870b31eba3616ee73d3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:25.942492 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_569ab9ec7c9c4870b31eba3616ee73d3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:25.948190 1174 tablet_bootstrap.cc:492] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:25.827384 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 304) took 5367 ms. Trace:
I20260202 02:52:25.948933 1180 rpcz_store.cc:276] 0202 02:52:20.460404 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:20.460455 (+ 51us) service_pool.cc:225] Handling call
0202 02:52:25.827190 (+5366735us) ts_tablet_manager.cc:1151] Deleting tablet 06b80a1c9251420cbc16fc95ae98aca9
0202 02:52:25.827202 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:25.827339 (+ 137us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5366483}
I20260202 02:52:25.949096 1174 ts_tablet_manager.cc:1403] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.001s
I20260202 02:52:25.950086 1180 tablet_replica.cc:333] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:25.950340 1180 raft_consensus.cc:2243] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:25.950531 1180 raft_consensus.cc:2272] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:25.950671 1174 raft_consensus.cc:359] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.950878 1174 raft_consensus.cc:385] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:25.950980 1174 raft_consensus.cc:740] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:25.951125 1180 ts_tablet_manager.cc:1916] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:25.952558 1174 consensus_queue.cc:260] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:25.953765 1174 ts_tablet_manager.cc:1434] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260202 02:52:25.954883 1174 tablet_bootstrap.cc:492] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:25.957330 523 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_6de529f984b247d79857c52705d4974a" } new_table_name: "default.Table_bed5f0c218b24f529e48138901d1afa0" modify_external_catalogs: true
I20260202 02:52:25.958664 1174 tablet_bootstrap.cc:654] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:25.962172 522 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet dab32469bd3b426fa63b35689d5bed7b
I20260202 02:52:25.973685 2047 raft_consensus.cc:493] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:25.973958 2047 raft_consensus.cc:515] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:25.974737 2047 leader_election.cc:290] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
W20260202 02:52:25.976693 714 leader_election.cc:343] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: d67e91a022f1401ea724dff898c3b81b
I20260202 02:52:25.977957 1175 tablet_bootstrap.cc:492] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:25.978188 1175 ts_tablet_manager.cc:1403] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.124s user 0.027s sys 0.030s
I20260202 02:52:25.979077 1175 raft_consensus.cc:359] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:25.979482 1175 raft_consensus.cc:385] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:25.979589 1175 raft_consensus.cc:740] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:25.979923 1175 consensus_queue.cc:260] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:25.983917 1175 ts_tablet_manager.cc:1434] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260202 02:52:25.984875 1175 tablet_bootstrap.cc:492] T 26eeef6eebbf4f2cb44f69255fb37594 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:25.986966 1175 tablet_bootstrap.cc:654] T 26eeef6eebbf4f2cb44f69255fb37594 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:25.988384 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_cbbe5e139f7e4e058d9e05816e811c98"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:25.989051 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cbbe5e139f7e4e058d9e05816e811c98 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:25.990381 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_d24b7b40906a4dfba59c9907d7b3cd1a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:25.990955 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d24b7b40906a4dfba59c9907d7b3cd1a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:25.951176 1173 tablet_bootstrap.cc:492] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
W20260202 02:52:25.965484 998 consensus_peers.cc:597] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f93e8d3c4faf48eaa98c3cbf88d585e2. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:25.991448 1173 ts_tablet_manager.cc:1403] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.061s user 0.010s sys 0.004s
I20260202 02:52:25.976145 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d67e91a022f1401ea724dff898c3b81b" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:25.991689 1062 raft_consensus.cc:2468] T d67e91a022f1401ea724dff898c3b81b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:25.992421 1173 raft_consensus.cc:359] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:25.993153 1173 raft_consensus.cc:385] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:25.993261 1173 raft_consensus.cc:740] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:25.993235 715 leader_election.cc:304] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:25.993525 1173 consensus_queue.cc:260] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:25.993755 2047 raft_consensus.cc:2804] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:25.993844 2047 raft_consensus.cc:493] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:25.993937 2047 raft_consensus.cc:3060] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:25.994067 1173 ts_tablet_manager.cc:1434] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260202 02:52:25.994386 1173 tablet_bootstrap.cc:492] T 38301e13e17c41fcae912c4b38c26a45 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:25.996119 1173 tablet_bootstrap.cc:654] T 38301e13e17c41fcae912c4b38c26a45 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:26.003327 2047 raft_consensus.cc:515] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:26.004827 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d67e91a022f1401ea724dff898c3b81b" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
W20260202 02:52:26.004913 714 leader_election.cc:343] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: d67e91a022f1401ea724dff898c3b81b
I20260202 02:52:26.005105 1062 raft_consensus.cc:3060] T d67e91a022f1401ea724dff898c3b81b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:26.005729 527 master_service.cc:954] ReplaceTablet: received request to replace tablet f6c2bc531d194a4b8f3062c9b40e3da6 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:26.008208 1062 raft_consensus.cc:2468] T d67e91a022f1401ea724dff898c3b81b P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:26.008538 2019 raft_consensus.cc:493] T fe9ea4c0502f4976b440465992f2ea2c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:26.008692 2019 raft_consensus.cc:515] T fe9ea4c0502f4976b440465992f2ea2c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:26.009364 2019 leader_election.cc:290] T fe9ea4c0502f4976b440465992f2ea2c P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:26.010025 2047 leader_election.cc:290] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:26.010203 1058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fe9ea4c0502f4976b440465992f2ea2c" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:26.010603 715 leader_election.cc:304] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:26.010671 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fe9ea4c0502f4976b440465992f2ea2c" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:26.011031 2047 raft_consensus.cc:2804] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:26.011291 2047 raft_consensus.cc:697] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
W20260202 02:52:26.011641 857 leader_election.cc:343] T fe9ea4c0502f4976b440465992f2ea2c P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:26.012091 859 leader_election.cc:343] T fe9ea4c0502f4976b440465992f2ea2c P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:26.012025 2047 consensus_queue.cc:237] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:26.012210 1173 tablet_bootstrap.cc:492] T 38301e13e17c41fcae912c4b38c26a45 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:26.012185 859 leader_election.cc:304] T fe9ea4c0502f4976b440465992f2ea2c P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:26.012372 1173 ts_tablet_manager.cc:1403] T 38301e13e17c41fcae912c4b38c26a45 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.008s
I20260202 02:52:26.012698 2019 raft_consensus.cc:2749] T fe9ea4c0502f4976b440465992f2ea2c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:26.013291 1173 raft_consensus.cc:359] T 38301e13e17c41fcae912c4b38c26a45 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.013494 1173 raft_consensus.cc:385] T 38301e13e17c41fcae912c4b38c26a45 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:26.013582 1173 raft_consensus.cc:740] T 38301e13e17c41fcae912c4b38c26a45 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:26.013825 1173 consensus_queue.cc:260] T 38301e13e17c41fcae912c4b38c26a45 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.014945 523 master_service.cc:954] ReplaceTablet: received request to replace tablet 5e20842de8cb4355b732685605c88834 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:26.017143 1174 tablet_bootstrap.cc:492] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:26.017341 1174 ts_tablet_manager.cc:1403] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.063s user 0.011s sys 0.011s
I20260202 02:52:26.019539 1174 raft_consensus.cc:359] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.019806 1174 raft_consensus.cc:385] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:26.019873 1174 raft_consensus.cc:740] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:26.021203 1173 ts_tablet_manager.cc:1434] T 38301e13e17c41fcae912c4b38c26a45 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260202 02:52:26.021617 1174 consensus_queue.cc:260] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.024435 1174 ts_tablet_manager.cc:1434] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.007s user 0.002s sys 0.001s
I20260202 02:52:26.024987 1173 tablet_bootstrap.cc:492] T 08d8d781bb4e4acf915f11afb4fb2e22 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:26.026295 1174 tablet_bootstrap.cc:492] T a5622ef925bc46cca76f481ddfd12afd P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:26.027917 1174 tablet_bootstrap.cc:654] T a5622ef925bc46cca76f481ddfd12afd P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:26.034438 519 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 16b47a2831f54d809ed0b16deaec7e3d (table default.Table_e4018d1c57714a24be5d3a20c1ccaef4 [id=1e291b35a6764638acfb781bb4e6adb3]). Expected version 2 got 0
I20260202 02:52:26.034768 519 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 26b890ed87c3471b9fc0b172c90e0a11 (table default.Table_044ef6a2b1af4e3cab3c8cf00505c6a6 [id=d2e4220a0cbf443e8380433da4f064f8]). Expected version 1 got 0
I20260202 02:52:26.034977 519 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 25765fb687294e9db1289f76530dd877 (table default.Table_8c47663f416b467cad37110bd09d85ad [id=ea0c52a72da8422ba14345fbfc242d27]). Expected version 1 got 0
I20260202 02:52:26.037084 524 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_7c01ea0d668a490099dbd42db17162ec [id=02e9e243f710428091de2f9497532c37] alter complete (version 1)
I20260202 02:52:26.038259 520 master_service.cc:954] ReplaceTablet: received request to replace tablet 8ab5c6fbec394b9bab322a197181a618 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:26.041085 901 tablet_service.cc:1505] Processing CreateTablet for tablet 7665dbea16d14f619af57ea056639eef (DEFAULT_TABLE table=default.Table_d5007434a43d4bed87b905c21243d3aa [id=06b0c5ef08054a7db456c82c61327fda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.041538 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7665dbea16d14f619af57ea056639eef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.042271 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 8cf6a52eea054ab688410e2dd029abf3 (DEFAULT_TABLE table=default.Table_072c7718ffd04cdfaafcce94c169bcd8 [id=96366ac25e1e4d3a9a9455a98dd2d252]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.042363 891 tablet_service.cc:1505] Processing CreateTablet for tablet a472b3b95de1476c99c74dcdddbd022d (DEFAULT_TABLE table=default.Table_bcaaa936a01e4aafbb3dc42f01abff56 [id=f3cca9e684db48a08ed7a40cf8683cce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.042691 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a472b3b95de1476c99c74dcdddbd022d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.042681 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cf6a52eea054ab688410e2dd029abf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.043390 1032 tablet_service.cc:1505] Processing CreateTablet for tablet 7665dbea16d14f619af57ea056639eef (DEFAULT_TABLE table=default.Table_d5007434a43d4bed87b905c21243d3aa [id=06b0c5ef08054a7db456c82c61327fda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.043748 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7665dbea16d14f619af57ea056639eef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.044639 887 tablet_service.cc:1505] Processing CreateTablet for tablet 8cf6a52eea054ab688410e2dd029abf3 (DEFAULT_TABLE table=default.Table_072c7718ffd04cdfaafcce94c169bcd8 [id=96366ac25e1e4d3a9a9455a98dd2d252]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.044981 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cf6a52eea054ab688410e2dd029abf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.042271 1039 tablet_service.cc:1505] Processing CreateTablet for tablet b8f2b34b4f1e4ff2b2be4ff40aeb0c66 (DEFAULT_TABLE table=default.Table_0ee7c010ec144c2e9d633447e4d84dcf [id=812637d0b1da4789850022a699e6bbd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.047359 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8f2b34b4f1e4ff2b2be4ff40aeb0c66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.041117 889 tablet_service.cc:1505] Processing CreateTablet for tablet b8f2b34b4f1e4ff2b2be4ff40aeb0c66 (DEFAULT_TABLE table=default.Table_0ee7c010ec144c2e9d633447e4d84dcf [id=812637d0b1da4789850022a699e6bbd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.048451 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8f2b34b4f1e4ff2b2be4ff40aeb0c66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.049662 750 tablet_service.cc:1505] Processing CreateTablet for tablet 7665dbea16d14f619af57ea056639eef (DEFAULT_TABLE table=default.Table_d5007434a43d4bed87b905c21243d3aa [id=06b0c5ef08054a7db456c82c61327fda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.050066 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7665dbea16d14f619af57ea056639eef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.052685 756 tablet_service.cc:1505] Processing CreateTablet for tablet 8cf6a52eea054ab688410e2dd029abf3 (DEFAULT_TABLE table=default.Table_072c7718ffd04cdfaafcce94c169bcd8 [id=96366ac25e1e4d3a9a9455a98dd2d252]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.053102 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cf6a52eea054ab688410e2dd029abf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.054332 1277 raft_consensus.cc:493] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:26.054510 1277 raft_consensus.cc:515] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:26.055204 1277 leader_election.cc:290] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:26.055763 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3711910e050c4515bed4294d1271d5a9" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:26.055971 1062 raft_consensus.cc:2468] T 3711910e050c4515bed4294d1271d5a9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:26.056463 715 leader_election.cc:304] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:26.056684 747 tablet_service.cc:1505] Processing CreateTablet for tablet b8f2b34b4f1e4ff2b2be4ff40aeb0c66 (DEFAULT_TABLE table=default.Table_0ee7c010ec144c2e9d633447e4d84dcf [id=812637d0b1da4789850022a699e6bbd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.056806 1277 raft_consensus.cc:2804] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:26.056887 1277 raft_consensus.cc:493] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:26.057009 1277 raft_consensus.cc:3060] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:26.057118 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8f2b34b4f1e4ff2b2be4ff40aeb0c66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.060303 1277 raft_consensus.cc:515] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:26.060720 752 tablet_service.cc:1505] Processing CreateTablet for tablet a472b3b95de1476c99c74dcdddbd022d (DEFAULT_TABLE table=default.Table_bcaaa936a01e4aafbb3dc42f01abff56 [id=f3cca9e684db48a08ed7a40cf8683cce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.062844 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a472b3b95de1476c99c74dcdddbd022d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.063235 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3711910e050c4515bed4294d1271d5a9" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:26.063438 1062 raft_consensus.cc:3060] T 3711910e050c4515bed4294d1271d5a9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:26.063931 1882 ts_tablet_manager.cc:1929] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:26.064078 1882 log.cc:1199] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/eaa47f3388144e8b852804c167f232ce
I20260202 02:52:26.064628 1882 ts_tablet_manager.cc:1950] T eaa47f3388144e8b852804c167f232ce P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:26.066251 1062 raft_consensus.cc:2468] T 3711910e050c4515bed4294d1271d5a9 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:26.068542 525 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_39f7c709fcc14c2d9c0449cb64b980d0" } new_table_name: "default.Table_c65ca0837da14b6baeb2f46f36018548" modify_external_catalogs: true
I20260202 02:52:26.070122 1936 raft_consensus.cc:493] T 3d1a7578e83447669e3c3972a426ad44 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:26.070276 1936 raft_consensus.cc:515] T 3d1a7578e83447669e3c3972a426ad44 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:26.070861 1936 leader_election.cc:290] T 3d1a7578e83447669e3c3972a426ad44 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:26.071282 1860 raft_consensus.cc:493] T 80d443bdb66d4cc68ae283e08c410fe6 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:26.071424 715 leader_election.cc:304] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:26.071480 1860 raft_consensus.cc:515] T 80d443bdb66d4cc68ae283e08c410fe6 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.072077 1860 leader_election.cc:290] T 80d443bdb66d4cc68ae283e08c410fe6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:26.072453 2042 raft_consensus.cc:2804] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:26.072597 2042 raft_consensus.cc:697] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:26.073283 2042 consensus_queue.cc:237] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
W20260202 02:52:26.074811 998 leader_election.cc:343] T 80d443bdb66d4cc68ae283e08c410fe6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Not found: Tablet not found: 80d443bdb66d4cc68ae283e08c410fe6
W20260202 02:52:26.075220 714 leader_election.cc:343] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 3711910e050c4515bed4294d1271d5a9
W20260202 02:52:26.075855 714 leader_election.cc:343] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 3711910e050c4515bed4294d1271d5a9
I20260202 02:52:26.076074 1850 raft_consensus.cc:493] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
I20260202 02:52:26.076258 1850 raft_consensus.cc:515] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.076997 1850 leader_election.cc:290] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:26.078874 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3d1a7578e83447669e3c3972a426ad44" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:26.081552 999 leader_election.cc:343] T 80d443bdb66d4cc68ae283e08c410fe6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 80d443bdb66d4cc68ae283e08c410fe6
W20260202 02:52:26.081553 857 leader_election.cc:343] T 3d1a7578e83447669e3c3972a426ad44 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:26.081475 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_9121c17ae28a45f2a1ce941e70260558"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:26.081741 999 leader_election.cc:304] T 80d443bdb66d4cc68ae283e08c410fe6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
W20260202 02:52:26.082141 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9121c17ae28a45f2a1ce941e70260558 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:26.082252 1860 raft_consensus.cc:2749] T 80d443bdb66d4cc68ae283e08c410fe6 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260202 02:52:26.082145 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_d9237a0573e5464a995da7c7bafc08a6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:26.082751 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d9237a0573e5464a995da7c7bafc08a6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:26.083992 1180 ts_tablet_manager.cc:1929] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:26.084259 1180 log.cc:1199] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/9884c90cf1d245b2a7eaaf00d71b1c3e
I20260202 02:52:26.084913 1180 ts_tablet_manager.cc:1950] T 9884c90cf1d245b2a7eaaf00d71b1c3e P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
W20260202 02:52:26.085597 714 leader_election.cc:343] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 938813ee0a8a40b6bf0e537e3ec1a0d4
I20260202 02:52:26.085973 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 309) took 5491 ms. Trace:
I20260202 02:52:26.086125 1180 rpcz_store.cc:276] 0202 02:52:20.594139 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:20.594350 (+ 211us) service_pool.cc:225] Handling call
0202 02:52:25.949416 (+5355066us) ts_tablet_manager.cc:1151] Deleting tablet 9884c90cf1d245b2a7eaaf00d71b1c3e
0202 02:52:25.949431 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:26.001003 (+ 51572us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:26.081024 (+ 80021us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:26.085896 (+ 4872us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5354781}
I20260202 02:52:26.087026 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 9884c90cf1d245b2a7eaaf00d71b1c3e (table default.Table_22dd5a0be7944811bf358ec584461157 [id=7c8c316a6dd74622a257e70c3299ea1b]) successfully deleted
I20260202 02:52:26.087633 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 310) took 5463 ms. Trace:
I20260202 02:52:26.087822 1180 rpcz_store.cc:276] 0202 02:52:20.624595 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:20.624845 (+ 250us) service_pool.cc:225] Handling call
0202 02:52:26.087414 (+5462569us) ts_tablet_manager.cc:1151] Deleting tablet 072bb7f0e4ab4935be49461bc8a3c163
0202 02:52:26.087426 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:26.087570 (+ 144us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5454980}
W20260202 02:52:26.088426 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 072bb7f0e4ab4935be49461bc8a3c163 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 072bb7f0e4ab4935be49461bc8a3c163 already in progress: creating tablet
I20260202 02:52:26.089087 1033 tablet_service.cc:1505] Processing CreateTablet for tablet a472b3b95de1476c99c74dcdddbd022d (DEFAULT_TABLE table=default.Table_bcaaa936a01e4aafbb3dc42f01abff56 [id=f3cca9e684db48a08ed7a40cf8683cce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.090210 887 tablet_service.cc:1505] Processing CreateTablet for tablet 7b45a2e6b4b14d3fa96a7131afb16208 (DEFAULT_TABLE table=default.Table_378dba7a0b4f47c586bcdd3f53712ceb [id=11ccce1bf8564d90a479bee20b1d7bb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.091583 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b45a2e6b4b14d3fa96a7131afb16208. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.093941 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 7b45a2e6b4b14d3fa96a7131afb16208 (DEFAULT_TABLE table=default.Table_378dba7a0b4f47c586bcdd3f53712ceb [id=11ccce1bf8564d90a479bee20b1d7bb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.101543 1860 raft_consensus.cc:493] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
I20260202 02:52:26.101786 1860 raft_consensus.cc:515] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.102463 1860 leader_election.cc:290] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:26.103618 999 leader_election.cc:343] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 938813ee0a8a40b6bf0e537e3ec1a0d4
I20260202 02:52:26.104489 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3d1a7578e83447669e3c3972a426ad44" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:26.104838 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "938813ee0a8a40b6bf0e537e3ec1a0d4" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:26.105100 1060 raft_consensus.cc:2468] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:26.105244 2015 tablet.cc:1722] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260202 02:52:26.106019 859 leader_election.cc:343] T 3d1a7578e83447669e3c3972a426ad44 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:26.106133 859 leader_election.cc:304] T 3d1a7578e83447669e3c3972a426ad44 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:26.106487 2019 raft_consensus.cc:2749] T 3d1a7578e83447669e3c3972a426ad44 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:26.094280 745 tablet_service.cc:1505] Processing CreateTablet for tablet 7b45a2e6b4b14d3fa96a7131afb16208 (DEFAULT_TABLE table=default.Table_378dba7a0b4f47c586bcdd3f53712ceb [id=11ccce1bf8564d90a479bee20b1d7bb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.107411 901 tablet_service.cc:1505] Processing CreateTablet for tablet e99a0003b6b74b99bf2b834c3690c1bb (DEFAULT_TABLE table=default.Table_df0389a8470d4dcfa6d9b90366867bae [id=305ae4f445fe4461aaed24b6dba424e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.107543 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b45a2e6b4b14d3fa96a7131afb16208. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.107762 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e99a0003b6b74b99bf2b834c3690c1bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.103446 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "938813ee0a8a40b6bf0e537e3ec1a0d4" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:26.109055 779 raft_consensus.cc:2468] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:26.114495 740 tablet_service.cc:1505] Processing CreateTablet for tablet e99a0003b6b74b99bf2b834c3690c1bb (DEFAULT_TABLE table=default.Table_df0389a8470d4dcfa6d9b90366867bae [id=305ae4f445fe4461aaed24b6dba424e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.115036 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e99a0003b6b74b99bf2b834c3690c1bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.115430 2019 raft_consensus.cc:493] T 75569511179b41038e521fa3346a92ae P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:26.115569 2019 raft_consensus.cc:515] T 75569511179b41038e521fa3346a92ae P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:26.116178 2019 leader_election.cc:290] T 75569511179b41038e521fa3346a92ae P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:26.105836 715 leader_election.cc:304] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:26.118495 1972 raft_consensus.cc:2804] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260202 02:52:26.118602 1972 raft_consensus.cc:493] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
I20260202 02:52:26.118702 1972 raft_consensus.cc:3060] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:26.119871 523 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 5e20842de8cb4355b732685605c88834
I20260202 02:52:26.120388 523 catalog_manager.cc:6724] ReplaceTablet: tablet 5e20842de8cb4355b732685605c88834 deleted and replaced by tablet 34a01a4ce1be4ad08d911763de158073
I20260202 02:52:26.124614 521 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 0ddc726efd3f49489092919c9c6e01fb
I20260202 02:52:26.125353 521 master_service.cc:954] ReplaceTablet: received request to replace tablet 2616e1bbce4049dbb584550ccc150288 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:26.126854 1972 raft_consensus.cc:515] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.127712 1972 leader_election.cc:290] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:26.129815 523 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 28a900bcaa8b41718a285e595886d805 (table default.Table_911bb209b4554ee8b9bc09edaf488d63 [id=cd07f9746ec94d9ebe956a7ca0d19b1c]). Expected version 1 got 0
I20260202 02:52:26.130709 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 5e20842de8cb4355b732685605c88834 with delete_type TABLET_DATA_DELETED (replaced by tablet 34a01a4ce1be4ad08d911763de158073) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:26.131013 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 0ddc726efd3f49489092919c9c6e01fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:25 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:26.091814 1277 leader_election.cc:290] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:26.132553 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "75569511179b41038e521fa3346a92ae" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:26.133159 526 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 1116872cc25a4ab5b1a3a6bbba1f0da7
I20260202 02:52:26.133788 526 master_service.cc:954] ReplaceTablet: received request to replace tablet 3191a80374724effa8389adc842e3448 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:26.135140 523 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet cb634d4bae554ef3b39eeaa9958adc6c (table default.Table_e5f4ac82d1ef4b85999695997eb9df2a [id=c97e407e519d4a538860f66931f6aefd]). Expected version 2 got 0
I20260202 02:52:26.135666 523 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 25765fb687294e9db1289f76530dd877 (table default.Table_03cb720f804b44e2b29f8cc47336b2db [id=ea0c52a72da8422ba14345fbfc242d27]). Expected version 2 got 0
I20260202 02:52:26.135912 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 1116872cc25a4ab5b1a3a6bbba1f0da7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:25 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:26.136806 523 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 4e79cd48fefb4ce29ef48beacfc4d26f (table default.Table_bed5f0c218b24f529e48138901d1afa0 [id=2a625eeef55b43f19cd4bbeb61e85bca]). Expected version 1 got 0
W20260202 02:52:26.136893 857 leader_election.cc:343] T 75569511179b41038e521fa3346a92ae P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:26.137266 714 leader_election.cc:343] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 938813ee0a8a40b6bf0e537e3ec1a0d4
I20260202 02:52:26.138880 523 catalog_manager.cc:5654] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d reported cstate change: term changed from 0 to 1, leader changed from <none> to b1f05820b6034228bdad884cc348716d (127.0.104.131). New cstate: current_term: 1 leader_uuid: "b1f05820b6034228bdad884cc348716d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: HEALTHY } } }
W20260202 02:52:26.140084 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 1116872cc25a4ab5b1a3a6bbba1f0da7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1116872cc25a4ab5b1a3a6bbba1f0da7 already in progress: creating tablet
I20260202 02:52:26.141811 901 tablet_service.cc:1552] Processing DeleteTablet for tablet 0ddc726efd3f49489092919c9c6e01fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:25 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:26.141913 523 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 20ea39bc5d3d46f5a4ba427277d6c6a3 (table default.Table_3b335a0806154f98a907f551596ba353 [id=0da4db61b3fe4cf4846edd5530b9327f]). Expected version 1 got 0
I20260202 02:52:26.143357 745 tablet_service.cc:1552] Processing DeleteTablet for tablet 1116872cc25a4ab5b1a3a6bbba1f0da7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:25 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:26.145582 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 5e20842de8cb4355b732685605c88834 with delete_type TABLET_DATA_DELETED (replaced by tablet 34a01a4ce1be4ad08d911763de158073) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:26.145542 745 tablet_service.cc:1505] Processing CreateTablet for tablet ff5966cfdcdd49adaaa3dbc994f82e57 (DEFAULT_TABLE table=default.Table_39f7c709fcc14c2d9c0449cb64b980d0 [id=3ce5b518d0b0428f9624bc81c3b5c36b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.146054 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff5966cfdcdd49adaaa3dbc994f82e57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.148167 527 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet f6c2bc531d194a4b8f3062c9b40e3da6
I20260202 02:52:26.148500 519 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_c33bb328c09b4b6ab9974ec216487b8a [id=fa95e04db2d443b5972de98c7c0e6457] alter complete (version 4)
I20260202 02:52:26.148715 527 catalog_manager.cc:6724] ReplaceTablet: tablet f6c2bc531d194a4b8f3062c9b40e3da6 deleted and replaced by tablet 29cde4226db6428987d4a39cde804a5c
I20260202 02:52:26.149067 527 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_e8e9b4cdaf434b6cbbfed1a65bff1fc0" } modify_external_catalogs: true
I20260202 02:52:26.149276 527 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_e8e9b4cdaf434b6cbbfed1a65bff1fc0" } modify_external_catalogs: true
I20260202 02:52:26.150875 519 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_f93837f1f1614be4bb992e92e5237ea2" } new_table_name: "default.Table_6bfcc07665864f5dbbabd5685876b6a1" modify_external_catalogs: true
I20260202 02:52:26.090102 1180 tablet_replica.cc:333] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d: stopping tablet replica
W20260202 02:52:26.152113 525 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ff5966cfdcdd49adaaa3dbc994f82e57 consensus state has no leader
I20260202 02:52:26.152236 1180 raft_consensus.cc:2243] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:26.152628 1180 pending_rounds.cc:70] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d: Trying to abort 1 pending ops.
I20260202 02:52:26.152570 525 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_48faa250b04a4b61a37f532584081968" } new_table_name: "default.Table_b5dcfcce852a48a0b1fbd3deed34f4d2" modify_external_catalogs: true
I20260202 02:52:26.152691 1180 pending_rounds.cc:77] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923036160380928 op_type: NO_OP noop_request { }
I20260202 02:52:26.152828 1180 raft_consensus.cc:2889] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:26.152926 1180 raft_consensus.cc:2272] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:26.151517 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_190bbbb8a6a6454cb95f513c1c72b8fd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:26.106917 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 378) took 2816 ms. Trace:
I20260202 02:52:26.153923 1882 rpcz_store.cc:276] 0202 02:52:23.290544 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.291341 (+ 797us) service_pool.cc:225] Handling call
0202 02:52:25.939357 (+2648016us) ts_tablet_manager.cc:1151] Deleting tablet eaa47f3388144e8b852804c167f232ce
0202 02:52:25.939368 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:26.059869 (+120501us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:26.063776 (+ 3907us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:26.106873 (+ 43097us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2597269}
W20260202 02:52:26.154235 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_190bbbb8a6a6454cb95f513c1c72b8fd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:26.154511 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet eaa47f3388144e8b852804c167f232ce (table default.Table_87277c5d80184738b96b773f026b32a5 [id=52687dbd0e554cfb81d88e8fcdd4dd5d]) successfully deleted
I20260202 02:52:26.155390 1882 tablet_replica.cc:333] T 80d443bdb66d4cc68ae283e08c410fe6 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:26.155627 1882 raft_consensus.cc:2243] T 80d443bdb66d4cc68ae283e08c410fe6 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:26.155813 1882 raft_consensus.cc:2272] T 80d443bdb66d4cc68ae283e08c410fe6 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:26.108639 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a472b3b95de1476c99c74dcdddbd022d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.156411 1882 ts_tablet_manager.cc:1916] T 80d443bdb66d4cc68ae283e08c410fe6 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:26.158847 520 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 8ab5c6fbec394b9bab322a197181a618
I20260202 02:52:26.159369 520 catalog_manager.cc:6724] ReplaceTablet: tablet 8ab5c6fbec394b9bab322a197181a618 deleted and replaced by tablet 83d198d90a204c6396b569fc1b65abef
I20260202 02:52:26.160300 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 8ab5c6fbec394b9bab322a197181a618 with delete_type TABLET_DATA_DELETED (replaced by tablet 83d198d90a204c6396b569fc1b65abef) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:26.109592 998 leader_election.cc:304] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
W20260202 02:52:26.160974 998 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 51443.7us
I20260202 02:52:26.114216 1029 tablet_service.cc:1505] Processing CreateTablet for tablet e99a0003b6b74b99bf2b834c3690c1bb (DEFAULT_TABLE table=default.Table_df0389a8470d4dcfa6d9b90366867bae [id=305ae4f445fe4461aaed24b6dba424e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.161855 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e99a0003b6b74b99bf2b834c3690c1bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.162410 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_6959068a5e634039a454c64744f31522"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:26.163007 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6959068a5e634039a454c64744f31522 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:26.163012 1180 ts_tablet_manager.cc:1916] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:26.164709 1721 raft_consensus.cc:2804] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260202 02:52:26.164871 1721 raft_consensus.cc:493] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
I20260202 02:52:26.164961 1721 raft_consensus.cc:3060] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:26.165953 756 tablet_service.cc:1552] Processing DeleteTablet for tablet f6c2bc531d194a4b8f3062c9b40e3da6 with delete_type TABLET_DATA_DELETED (replaced by tablet 29cde4226db6428987d4a39cde804a5c) from {username='slave'} at 127.0.0.1:42950
W20260202 02:52:26.167375 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 0ddc726efd3f49489092919c9c6e01fb because the tablet was not found. No further retry: Not found: Tablet not found: 0ddc726efd3f49489092919c9c6e01fb
I20260202 02:52:26.167481 2038 raft_consensus.cc:493] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:26.167687 2038 raft_consensus.cc:515] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:26.167842 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b45a2e6b4b14d3fa96a7131afb16208. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:26.167836 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 5e20842de8cb4355b732685605c88834 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5e20842de8cb4355b732685605c88834 already in progress: creating tablet
I20260202 02:52:26.168533 901 tablet_service.cc:1552] Processing DeleteTablet for tablet f6c2bc531d194a4b8f3062c9b40e3da6 with delete_type TABLET_DATA_DELETED (replaced by tablet 29cde4226db6428987d4a39cde804a5c) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:26.168684 2038 leader_election.cc:290] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:26.168910 1721 raft_consensus.cc:515] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.169018 887 tablet_service.cc:1505] Processing CreateTablet for tablet ff5966cfdcdd49adaaa3dbc994f82e57 (DEFAULT_TABLE table=default.Table_39f7c709fcc14c2d9c0449cb64b980d0 [id=3ce5b518d0b0428f9624bc81c3b5c36b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.169495 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff5966cfdcdd49adaaa3dbc994f82e57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.169678 1721 leader_election.cc:290] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:26.172469 1036 tablet_service.cc:1505] Processing CreateTablet for tablet ff5966cfdcdd49adaaa3dbc994f82e57 (DEFAULT_TABLE table=default.Table_39f7c709fcc14c2d9c0449cb64b980d0 [id=3ce5b518d0b0428f9624bc81c3b5c36b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.172776 2082 tablet.cc:1722] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:26.172917 1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff5966cfdcdd49adaaa3dbc994f82e57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.174288 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "938813ee0a8a40b6bf0e537e3ec1a0d4" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:26.174554 775 raft_consensus.cc:2393] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b1f05820b6034228bdad884cc348716d in current term 2: Already voted for candidate 1f442ef85029461091de847118c34c0d in this term.
I20260202 02:52:26.174866 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "38fdbc6df7ec40b9afe4e0dfe0119e3e" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:26.176733 1028 tablet_service.cc:1552] Processing DeleteTablet for tablet f6c2bc531d194a4b8f3062c9b40e3da6 with delete_type TABLET_DATA_DELETED (replaced by tablet 29cde4226db6428987d4a39cde804a5c) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:26.177034 1034 tablet_service.cc:1552] Processing DeleteTablet for tablet 8ab5c6fbec394b9bab322a197181a618 with delete_type TABLET_DATA_DELETED (replaced by tablet 83d198d90a204c6396b569fc1b65abef) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:26.177620 998 leader_election.cc:343] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:26.116889 1058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "75569511179b41038e521fa3346a92ae" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:26.121097 1032 tablet_service.cc:1552] Processing DeleteTablet for tablet 5e20842de8cb4355b732685605c88834 with delete_type TABLET_DATA_DELETED (replaced by tablet 34a01a4ce1be4ad08d911763de158073) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:26.126402 1039 tablet_service.cc:1552] Processing DeleteTablet for tablet 0ddc726efd3f49489092919c9c6e01fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:25 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:26.128072 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "938813ee0a8a40b6bf0e537e3ec1a0d4" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:26.133844 1042 tablet_service.cc:1552] Processing DeleteTablet for tablet 1116872cc25a4ab5b1a3a6bbba1f0da7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:25 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:26.179574 1062 raft_consensus.cc:2393] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f442ef85029461091de847118c34c0d in current term 2: Already voted for candidate b1f05820b6034228bdad884cc348716d in this term.
W20260202 02:52:26.179950 859 leader_election.cc:343] T 75569511179b41038e521fa3346a92ae P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:26.180068 859 leader_election.cc:304] T 75569511179b41038e521fa3346a92ae P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:26.180151 715 leader_election.cc:304] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:26.180473 1936 raft_consensus.cc:2749] T 75569511179b41038e521fa3346a92ae P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:26.180670 1850 raft_consensus.cc:2749] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260202 02:52:26.184285 522 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_d040708acb92480496bac8b012890006" } modify_external_catalogs: true
I20260202 02:52:26.184547 522 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_d040708acb92480496bac8b012890006" } modify_external_catalogs: true
I20260202 02:52:26.185590 1036 tablet_service.cc:1505] Processing CreateTablet for tablet 8ae6390a856d4c838e0689cd0c6c5c92 (DEFAULT_TABLE table=default.Table_4ba0da0467a14feb8978f827568eb41f [id=d6643cdb0b874f32bef9da7fda1e582d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.186133 1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ae6390a856d4c838e0689cd0c6c5c92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.187395 747 tablet_service.cc:1505] Processing CreateTablet for tablet 8ae6390a856d4c838e0689cd0c6c5c92 (DEFAULT_TABLE table=default.Table_4ba0da0467a14feb8978f827568eb41f [id=d6643cdb0b874f32bef9da7fda1e582d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.187915 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ae6390a856d4c838e0689cd0c6c5c92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.192732 901 tablet_service.cc:1552] Processing DeleteTablet for tablet 75569511179b41038e521fa3346a92ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:26.192850 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 02b434fa058b45529ff3938b053263c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:26.193039 1182 tablet_replica.cc:333] T 75569511179b41038e521fa3346a92ae P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:26.193259 1182 raft_consensus.cc:2243] T 75569511179b41038e521fa3346a92ae P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:26.193415 1182 raft_consensus.cc:2272] T 75569511179b41038e521fa3346a92ae P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:26.193841 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 874ca4ba4ebc4721bb4b60cba8e01841 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:26.193991 1182 ts_tablet_manager.cc:1916] T 75569511179b41038e521fa3346a92ae P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:26.194545 891 tablet_service.cc:1552] Processing DeleteTablet for tablet fe9ea4c0502f4976b440465992f2ea2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:26.195089 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 8ab5c6fbec394b9bab322a197181a618 with delete_type TABLET_DATA_DELETED (replaced by tablet 83d198d90a204c6396b569fc1b65abef) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:26.195816 891 tablet_service.cc:1505] Processing CreateTablet for tablet 8ae6390a856d4c838e0689cd0c6c5c92 (DEFAULT_TABLE table=default.Table_4ba0da0467a14feb8978f827568eb41f [id=d6643cdb0b874f32bef9da7fda1e582d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.196215 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ae6390a856d4c838e0689cd0c6c5c92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.198336 1721 raft_consensus.cc:493] T c25e329023c04f609d8a743b4f7a95a4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:26.198510 1721 raft_consensus.cc:515] T c25e329023c04f609d8a743b4f7a95a4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.199182 1721 leader_election.cc:290] T c25e329023c04f609d8a743b4f7a95a4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:26.199512 774 raft_consensus.cc:3060] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:26.203073 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet f6c2bc531d194a4b8f3062c9b40e3da6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f6c2bc531d194a4b8f3062c9b40e3da6 already in progress: creating tablet
I20260202 02:52:26.203884 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c25e329023c04f609d8a743b4f7a95a4" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:26.206974 998 leader_election.cc:343] T c25e329023c04f609d8a743b4f7a95a4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:26.208166 521 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 2616e1bbce4049dbb584550ccc150288
I20260202 02:52:26.208716 521 catalog_manager.cc:6724] ReplaceTablet: tablet 2616e1bbce4049dbb584550ccc150288 deleted and replaced by tablet 8dd62de8e8234d6c9ce380b1aadc097d
I20260202 02:52:26.209329 526 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 3191a80374724effa8389adc842e3448
I20260202 02:52:26.210211 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 8b4dcfed8ec348e4b48a85788d728ca2 (DEFAULT_TABLE table=default.Table_72656c0c3e7f4157b3c57bcd71d097b7 [id=fb15f49f2a484130aa798949cb3f26e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.210480 526 catalog_manager.cc:6724] ReplaceTablet: tablet 3191a80374724effa8389adc842e3448 deleted and replaced by tablet 77f60925b6ca496d9b43502ed77bb55a
I20260202 02:52:26.210817 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b4dcfed8ec348e4b48a85788d728ca2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.212050 1042 tablet_service.cc:1552] Processing DeleteTablet for tablet 2616e1bbce4049dbb584550ccc150288 with delete_type TABLET_DATA_DELETED (replaced by tablet 8dd62de8e8234d6c9ce380b1aadc097d) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:26.212435 1039 tablet_service.cc:1552] Processing DeleteTablet for tablet 3191a80374724effa8389adc842e3448 with delete_type TABLET_DATA_DELETED (replaced by tablet 77f60925b6ca496d9b43502ed77bb55a) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:26.213590 527 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet ef073880898049e2be93d2cb6dc0c0bb
I20260202 02:52:26.214435 1039 tablet_service.cc:1552] Processing DeleteTablet for tablet ef073880898049e2be93d2cb6dc0c0bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:26.216109 1182 ts_tablet_manager.cc:1929] T 75569511179b41038e521fa3346a92ae P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:26.219795 740 tablet_service.cc:1552] Processing DeleteTablet for tablet ef073880898049e2be93d2cb6dc0c0bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:42950
W20260202 02:52:26.223059 999 leader_election.cc:343] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 38fdbc6df7ec40b9afe4e0dfe0119e3e
I20260202 02:52:26.223235 999 leader_election.cc:304] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
W20260202 02:52:26.223650 999 leader_election.cc:343] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 938813ee0a8a40b6bf0e537e3ec1a0d4
I20260202 02:52:26.223743 999 leader_election.cc:304] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
W20260202 02:52:26.224119 999 leader_election.cc:343] T c25e329023c04f609d8a743b4f7a95a4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: c25e329023c04f609d8a743b4f7a95a4
I20260202 02:52:26.224215 999 leader_election.cc:304] T c25e329023c04f609d8a743b4f7a95a4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:26.224586 2038 raft_consensus.cc:2749] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:26.224751 2038 raft_consensus.cc:2749] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260202 02:52:26.224895 2038 raft_consensus.cc:2749] T c25e329023c04f609d8a743b4f7a95a4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:26.224299 891 tablet_service.cc:1505] Processing CreateTablet for tablet 8b4dcfed8ec348e4b48a85788d728ca2 (DEFAULT_TABLE table=default.Table_72656c0c3e7f4157b3c57bcd71d097b7 [id=fb15f49f2a484130aa798949cb3f26e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.225304 1182 log.cc:1199] T 75569511179b41038e521fa3346a92ae P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/75569511179b41038e521fa3346a92ae
I20260202 02:52:26.225634 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b4dcfed8ec348e4b48a85788d728ca2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.225643 1182 ts_tablet_manager.cc:1950] T 75569511179b41038e521fa3346a92ae P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:26.227411 1182 tablet_replica.cc:333] T 02b434fa058b45529ff3938b053263c6 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:26.227607 1182 raft_consensus.cc:2243] T 02b434fa058b45529ff3938b053263c6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:26.227912 1182 raft_consensus.cc:2272] T 02b434fa058b45529ff3938b053263c6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:26.228011 752 tablet_service.cc:1552] Processing DeleteTablet for tablet 3191a80374724effa8389adc842e3448 with delete_type TABLET_DATA_DELETED (replaced by tablet 77f60925b6ca496d9b43502ed77bb55a) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:26.228447 1182 ts_tablet_manager.cc:1916] T 02b434fa058b45529ff3938b053263c6 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:26.234787 750 tablet_service.cc:1505] Processing CreateTablet for tablet 8b4dcfed8ec348e4b48a85788d728ca2 (DEFAULT_TABLE table=default.Table_72656c0c3e7f4157b3c57bcd71d097b7 [id=fb15f49f2a484130aa798949cb3f26e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.236496 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b4dcfed8ec348e4b48a85788d728ca2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.239740 1027 tablet_service.cc:1505] Processing CreateTablet for tablet b6e44b2a88e649599b398a7c9c4028c5 (DEFAULT_TABLE table=default.Table_6596d734c9de42ccbf1af1b077742369 [id=0583f0adc4aa46aa8d25f82f5539c669]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.240280 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6e44b2a88e649599b398a7c9c4028c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.242688 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 3191a80374724effa8389adc842e3448 with delete_type TABLET_DATA_DELETED (replaced by tablet 77f60925b6ca496d9b43502ed77bb55a) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:26.242746 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 2616e1bbce4049dbb584550ccc150288 with delete_type TABLET_DATA_DELETED (replaced by tablet 8dd62de8e8234d6c9ce380b1aadc097d) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:26.243295 887 tablet_service.cc:1552] Processing DeleteTablet for tablet ef073880898049e2be93d2cb6dc0c0bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:26.243446 889 tablet_service.cc:1505] Processing CreateTablet for tablet b6e44b2a88e649599b398a7c9c4028c5 (DEFAULT_TABLE table=default.Table_6596d734c9de42ccbf1af1b077742369 [id=0583f0adc4aa46aa8d25f82f5539c669]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.243844 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6e44b2a88e649599b398a7c9c4028c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.238461 752 tablet_service.cc:1505] Processing CreateTablet for tablet b6e44b2a88e649599b398a7c9c4028c5 (DEFAULT_TABLE table=default.Table_6596d734c9de42ccbf1af1b077742369 [id=0583f0adc4aa46aa8d25f82f5539c669]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.244674 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6e44b2a88e649599b398a7c9c4028c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.246500 740 tablet_service.cc:1505] Processing CreateTablet for tablet a806e1bd36564d659ff06cbe2016bb2a (DEFAULT_TABLE table=default.Table_ba53f6b46a524a03832adc957671ae98 [id=71a5a7d475ba498099f2bd7cc7289808]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.246920 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a806e1bd36564d659ff06cbe2016bb2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.247083 1033 tablet_service.cc:1505] Processing CreateTablet for tablet a806e1bd36564d659ff06cbe2016bb2a (DEFAULT_TABLE table=default.Table_ba53f6b46a524a03832adc957671ae98 [id=71a5a7d475ba498099f2bd7cc7289808]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.247387 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 75569511179b41038e521fa3346a92ae (table default.Table_6da292362a6c43738dbeb895750b918a [id=6d50800a3c8642c0992dd1a7f38cfd30]) successfully deleted
I20260202 02:52:26.247494 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a806e1bd36564d659ff06cbe2016bb2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.249433 887 tablet_service.cc:1505] Processing CreateTablet for tablet a806e1bd36564d659ff06cbe2016bb2a (DEFAULT_TABLE table=default.Table_ba53f6b46a524a03832adc957671ae98 [id=71a5a7d475ba498099f2bd7cc7289808]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.250129 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a806e1bd36564d659ff06cbe2016bb2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.251036 745 tablet_service.cc:1552] Processing DeleteTablet for tablet 2616e1bbce4049dbb584550ccc150288 with delete_type TABLET_DATA_DELETED (replaced by tablet 8dd62de8e8234d6c9ce380b1aadc097d) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:26.251382 2082 tablet.cc:1722] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:26.258565 2015 tablet.cc:1722] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:26.258577 1882 ts_tablet_manager.cc:1929] T 80d443bdb66d4cc68ae283e08c410fe6 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:26.258917 1882 log.cc:1199] T 80d443bdb66d4cc68ae283e08c410fe6 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/80d443bdb66d4cc68ae283e08c410fe6
I20260202 02:52:26.259366 1882 ts_tablet_manager.cc:1950] T 80d443bdb66d4cc68ae283e08c410fe6 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:26.260249 1173 tablet_bootstrap.cc:654] T 08d8d781bb4e4acf915f11afb4fb2e22 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:26.261327 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_5991ee9682fa4e3396ece0dbf283e777"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:26.261276 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_35e37309ffba469c975e911bffcff4ce"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:26.263329 519 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8b0368e03f644f1db28dbb57c95c48c1 consensus state has no leader
W20260202 02:52:26.263412 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_35e37309ffba469c975e911bffcff4ce in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:26.263288 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5991ee9682fa4e3396ece0dbf283e777 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:26.263700 519 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_4ba0da0467a14feb8978f827568eb41f" } modify_external_catalogs: true
I20260202 02:52:26.263898 519 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_4ba0da0467a14feb8978f827568eb41f" } modify_external_catalogs: true
I20260202 02:52:26.264801 1860 raft_consensus.cc:493] T a1f39e5b02f747819e636cddb4292eb6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:26.264997 1860 raft_consensus.cc:515] T a1f39e5b02f747819e636cddb4292eb6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.265017 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_5a24aceace304393ae51aa2018eab99f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:26.265551 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5a24aceace304393ae51aa2018eab99f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:26.265667 1860 leader_election.cc:290] T a1f39e5b02f747819e636cddb4292eb6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:26.267725 999 leader_election.cc:343] T a1f39e5b02f747819e636cddb4292eb6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: a1f39e5b02f747819e636cddb4292eb6
I20260202 02:52:26.268370 891 tablet_service.cc:1505] Processing CreateTablet for tablet 198432a3bb3a4090af38aace27a0b9c8 (DEFAULT_TABLE table=default.Table_8a639c50c6674b2da7771e12fe2d7237 [id=576ab2d635b74195835c1c2ee9464acd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.268518 740 tablet_service.cc:1505] Processing CreateTablet for tablet 198432a3bb3a4090af38aace27a0b9c8 (DEFAULT_TABLE table=default.Table_8a639c50c6674b2da7771e12fe2d7237 [id=576ab2d635b74195835c1c2ee9464acd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.269055 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 198432a3bb3a4090af38aace27a0b9c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.271385 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 198432a3bb3a4090af38aace27a0b9c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.271359 1039 tablet_service.cc:1505] Processing CreateTablet for tablet 198432a3bb3a4090af38aace27a0b9c8 (DEFAULT_TABLE table=default.Table_8a639c50c6674b2da7771e12fe2d7237 [id=576ab2d635b74195835c1c2ee9464acd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.271778 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 198432a3bb3a4090af38aace27a0b9c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.278297 889 tablet_service.cc:1505] Processing CreateTablet for tablet e07e87c0182b46a49b8ca64c072f4210 (DEFAULT_TABLE table=default.Table_82c4120b31ef4f7fbb290c51dc0f7866 [id=ef32a2eeb5f94bbc8e3ae63aabbc232f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.278744 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e07e87c0182b46a49b8ca64c072f4210. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.279546 901 tablet_service.cc:1505] Processing CreateTablet for tablet 0545aad1286948e1ac4aa3958bfada16 (DEFAULT_TABLE table=default.Table_2cf49aa4b72d4448943e3ea50fa21c78 [id=9aa9a92096d2492a84e98022117a91ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.279920 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0545aad1286948e1ac4aa3958bfada16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.279564 1036 tablet_service.cc:1505] Processing CreateTablet for tablet e07e87c0182b46a49b8ca64c072f4210 (DEFAULT_TABLE table=default.Table_82c4120b31ef4f7fbb290c51dc0f7866 [id=ef32a2eeb5f94bbc8e3ae63aabbc232f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.281143 1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e07e87c0182b46a49b8ca64c072f4210. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.287890 745 tablet_service.cc:1505] Processing CreateTablet for tablet 0545aad1286948e1ac4aa3958bfada16 (DEFAULT_TABLE table=default.Table_2cf49aa4b72d4448943e3ea50fa21c78 [id=9aa9a92096d2492a84e98022117a91ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.288970 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0545aad1286948e1ac4aa3958bfada16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.291169 1039 tablet_service.cc:1505] Processing CreateTablet for tablet 0545aad1286948e1ac4aa3958bfada16 (DEFAULT_TABLE table=default.Table_2cf49aa4b72d4448943e3ea50fa21c78 [id=9aa9a92096d2492a84e98022117a91ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.289536 1180 ts_tablet_manager.cc:1929] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:26.291648 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0545aad1286948e1ac4aa3958bfada16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.291785 1180 log.cc:1199] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/c63a670d5c5e489897bc6dbb142b844d
I20260202 02:52:26.292305 1180 ts_tablet_manager.cc:1950] T c63a670d5c5e489897bc6dbb142b844d P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:26.293115 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 314) took 5495 ms. Trace:
I20260202 02:52:26.293285 1180 rpcz_store.cc:276] 0202 02:52:20.797598 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:20.797693 (+ 95us) service_pool.cc:225] Handling call
0202 02:52:26.089998 (+5292305us) ts_tablet_manager.cc:1151] Deleting tablet c63a670d5c5e489897bc6dbb142b844d
0202 02:52:26.090009 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:26.191115 (+101106us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:26.289409 (+ 98294us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:26.293060 (+ 3651us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5292066}
I20260202 02:52:26.293730 756 tablet_service.cc:1505] Processing CreateTablet for tablet 7ac33134670747ee860b6d39dd7db547 (DEFAULT_TABLE table=default.Table_4799ebe642db4543be7712203e1fe532 [id=f06e78bd7d5c48dd8f456fdf60ae91c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.293913 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet c63a670d5c5e489897bc6dbb142b844d (table default.Table_12d68a317c8e4d809dbbe77c095a1f6e [id=7219522ffc3849faab94d754a42a8511]) successfully deleted
I20260202 02:52:26.294101 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ac33134670747ee860b6d39dd7db547. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:26.296690 525 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 28f6161d0d0a4b41af4560f0f3bee0f7 consensus state has no leader
I20260202 02:52:26.297071 522 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 7ea7a5d8a92b4ac0903483627258ae66
I20260202 02:52:26.297132 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_a7d94a8b6efd406c845b24e72a7d4d25"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:26.297809 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a7d94a8b6efd406c845b24e72a7d4d25 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:26.297866 522 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_91411bac2a2a45a395b81a4dab2617f7" } new_table_name: "default.Table_12c352712c374a45b7de7a0e634a1327" modify_external_catalogs: true
I20260202 02:52:26.299067 528 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_03cb720f804b44e2b29f8cc47336b2db [id=ea0c52a72da8422ba14345fbfc242d27] alter complete (version 2)
I20260202 02:52:26.299392 519 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 8ae6390a856d4c838e0689cd0c6c5c92
I20260202 02:52:26.299857 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_df0a1db6c8b048938a0bbd425a0a65dd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:26.300110 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_9d2dd70237e04edbbc5c56011ed511ce"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:26.300501 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_df0a1db6c8b048938a0bbd425a0a65dd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:26.300592 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9d2dd70237e04edbbc5c56011ed511ce in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:26.300824 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 7ea7a5d8a92b4ac0903483627258ae66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:26.300930 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 8ae6390a856d4c838e0689cd0c6c5c92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:26.300829 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a1f39e5b02f747819e636cddb4292eb6" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:26.301661 998 leader_election.cc:343] T a1f39e5b02f747819e636cddb4292eb6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:26.301819 998 leader_election.cc:304] T a1f39e5b02f747819e636cddb4292eb6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:26.302291 1860 raft_consensus.cc:2749] T a1f39e5b02f747819e636cddb4292eb6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:26.302709 1180 tablet_replica.cc:333] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:26.302668 747 tablet_service.cc:1505] Processing CreateTablet for tablet e07e87c0182b46a49b8ca64c072f4210 (DEFAULT_TABLE table=default.Table_82c4120b31ef4f7fbb290c51dc0f7866 [id=ef32a2eeb5f94bbc8e3ae63aabbc232f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.303011 1180 raft_consensus.cc:2243] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:26.303210 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e07e87c0182b46a49b8ca64c072f4210. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.303290 1180 pending_rounds.cc:70] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d: Trying to abort 1 pending ops.
I20260202 02:52:26.303354 1180 pending_rounds.cc:77] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923055918657536 op_type: NO_OP noop_request { }
I20260202 02:52:26.303479 1180 raft_consensus.cc:2889] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:26.304131 1180 raft_consensus.cc:2272] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:26.260237 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 379) took 2969 ms. Trace:
I20260202 02:52:26.304455 1882 rpcz_store.cc:276] 0202 02:52:23.290625 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.291445 (+ 820us) service_pool.cc:225] Handling call
0202 02:52:26.155299 (+2863854us) ts_tablet_manager.cc:1151] Deleting tablet 80d443bdb66d4cc68ae283e08c410fe6
0202 02:52:26.155320 (+ 21us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:26.183035 (+ 27715us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:26.258432 (+ 75397us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:26.260160 (+ 1728us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2813051}
I20260202 02:52:26.299396 524 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_a5d6ab55014a4c068dceeb6a781bf3b1" } new_table_name: "default.Table_dd4c50f48678452cb127469d5a7bf548" modify_external_catalogs: true
I20260202 02:52:26.305214 889 tablet_service.cc:1505] Processing CreateTablet for tablet 7ac33134670747ee860b6d39dd7db547 (DEFAULT_TABLE table=default.Table_4799ebe642db4543be7712203e1fe532 [id=f06e78bd7d5c48dd8f456fdf60ae91c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.305624 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ac33134670747ee860b6d39dd7db547. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.308177 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 80d443bdb66d4cc68ae283e08c410fe6 (table default.Table_78e75899f83c46f09c66dc84192f5438 [id=0bd47d48c02944a19ce210fc83794879]) successfully deleted
I20260202 02:52:26.308926 1173 tablet_bootstrap.cc:492] T 08d8d781bb4e4acf915f11afb4fb2e22 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:26.309127 1173 ts_tablet_manager.cc:1403] T 08d8d781bb4e4acf915f11afb4fb2e22 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.285s user 0.002s sys 0.001s
I20260202 02:52:26.309554 1042 tablet_service.cc:1552] Processing DeleteTablet for tablet 38fdbc6df7ec40b9afe4e0dfe0119e3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:26.309836 1032 tablet_service.cc:1552] Processing DeleteTablet for tablet 38301e13e17c41fcae912c4b38c26a45 with delete_type TABLET_DATA_DELETED (replaced by tablet 79badde1001b432b95f58b5161ace15c) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:26.309947 1173 raft_consensus.cc:359] T 08d8d781bb4e4acf915f11afb4fb2e22 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.310166 1173 raft_consensus.cc:385] T 08d8d781bb4e4acf915f11afb4fb2e22 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:26.310204 1028 tablet_service.cc:1552] Processing DeleteTablet for tablet 8774c6bc0fe9414ea2f17ed859d9392a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:26.310247 1173 raft_consensus.cc:740] T 08d8d781bb4e4acf915f11afb4fb2e22 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:26.310041 1034 tablet_service.cc:1552] Processing DeleteTablet for tablet 68823eff1bc04108ba83487605aec0ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:21 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:26.310525 1173 consensus_queue.cc:260] T 08d8d781bb4e4acf915f11afb4fb2e22 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.310869 1936 raft_consensus.cc:493] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:26.311009 1936 raft_consensus.cc:515] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:26.311143 1173 ts_tablet_manager.cc:1434] T 08d8d781bb4e4acf915f11afb4fb2e22 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:26.311558 1936 leader_election.cc:290] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:26.314707 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "026e6ee0b3ae4d938fdf1550044ed1fa" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:26.315052 1173 tablet_bootstrap.cc:492] T 9c1271ce0f194021b588b82ac9d3db79 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:26.316932 1173 tablet_bootstrap.cc:654] T 9c1271ce0f194021b588b82ac9d3db79 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:26.318404 859 leader_election.cc:343] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:26.318490 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 8ae6390a856d4c838e0689cd0c6c5c92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:26.319257 885 tablet_service.cc:1552] Processing DeleteTablet for tablet fe53a8bdbd13417f9ce195c54a52a93c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:21 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:26.319619 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 026e6ee0b3ae4d938fdf1550044ed1fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:26.319932 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 8ae6390a856d4c838e0689cd0c6c5c92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:26.320020 901 tablet_service.cc:1552] Processing DeleteTablet for tablet 7ea7a5d8a92b4ac0903483627258ae66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:26.321462 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "026e6ee0b3ae4d938fdf1550044ed1fa" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:26.321995 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 7ac33134670747ee860b6d39dd7db547 (DEFAULT_TABLE table=default.Table_4799ebe642db4543be7712203e1fe532 [id=f06e78bd7d5c48dd8f456fdf60ae91c8]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:26.322361 857 leader_election.cc:343] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:26.322551 857 leader_election.cc:304] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:26.322852 1936 raft_consensus.cc:2749] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:26.323056 857 consensus_peers.cc:597] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5 -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:26.324108 757 tablet_service.cc:1552] Processing DeleteTablet for tablet c63a670d5c5e489897bc6dbb142b844d with delete_type TABLET_DATA_DELETED (replaced by tablet 939fbcaf75c7439d9208bbbf83a72717) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:26.324924 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 2f0e8c5e2fd44370ae150a206f4e6cbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:26.325630 757 tablet_service.cc:1552] Processing DeleteTablet for tablet d67e91a022f1401ea724dff898c3b81b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:17 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:26.326575 526 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_d6bec77a74b1431fb1c008f55f482b3a" } modify_external_catalogs: true
I20260202 02:52:26.326802 526 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_d6bec77a74b1431fb1c008f55f482b3a" } modify_external_catalogs: true
I20260202 02:52:26.327471 757 tablet_service.cc:1505] Processing CreateTablet for tablet f30b698c177c48ee9a4e577552bf7eee (DEFAULT_TABLE table=default.Table_96e21d4e03c941129f0581aa558481c0 [id=14bbe505bc8b4a4bbde264a2c0dcdf50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.327894 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f30b698c177c48ee9a4e577552bf7eee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.328188 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 59990791123c4d4eb620090a1e8e31e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:26.328491 1882 tablet_replica.cc:333] T d64f42d187bc4c9f9381c931449ec0cc P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:26.328691 1882 raft_consensus.cc:2243] T d64f42d187bc4c9f9381c931449ec0cc P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:26.328887 1882 raft_consensus.cc:2272] T d64f42d187bc4c9f9381c931449ec0cc P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:26.329421 1882 ts_tablet_manager.cc:1916] T d64f42d187bc4c9f9381c931449ec0cc P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:26.333197 1180 ts_tablet_manager.cc:1916] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:26.335233 527 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_cc8d0231630743f180159c4d3a4efab8" } modify_external_catalogs: true
I20260202 02:52:26.335444 527 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_cc8d0231630743f180159c4d3a4efab8" } modify_external_catalogs: true
W20260202 02:52:26.337374 999 consensus_peers.cc:597] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a5622ef925bc46cca76f481ddfd12afd. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:26.338284 1034 tablet_service.cc:1505] Processing CreateTablet for tablet f30b698c177c48ee9a4e577552bf7eee (DEFAULT_TABLE table=default.Table_96e21d4e03c941129f0581aa558481c0 [id=14bbe505bc8b4a4bbde264a2c0dcdf50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.342000 1033 tablet_service.cc:1552] Processing DeleteTablet for tablet 7ea7a5d8a92b4ac0903483627258ae66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:26.343454 901 tablet_service.cc:1505] Processing CreateTablet for tablet f30b698c177c48ee9a4e577552bf7eee (DEFAULT_TABLE table=default.Table_96e21d4e03c941129f0581aa558481c0 [id=14bbe505bc8b4a4bbde264a2c0dcdf50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.343878 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f30b698c177c48ee9a4e577552bf7eee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.344126 757 tablet_service.cc:1505] Processing CreateTablet for tablet 898778f8f0b548ec89cb1417d6f07926 (DEFAULT_TABLE table=default.Table_6cf38c67711747acb1b5be35632b7a68 [id=640ad152bb4243cda5a7e7242996e2fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.344543 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 898778f8f0b548ec89cb1417d6f07926. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.344926 1033 tablet_service.cc:1505] Processing CreateTablet for tablet 898778f8f0b548ec89cb1417d6f07926 (DEFAULT_TABLE table=default.Table_6cf38c67711747acb1b5be35632b7a68 [id=640ad152bb4243cda5a7e7242996e2fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.346032 2089 tablet.cc:1722] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:26.346761 891 tablet_service.cc:1505] Processing CreateTablet for tablet 898778f8f0b548ec89cb1417d6f07926 (DEFAULT_TABLE table=default.Table_6cf38c67711747acb1b5be35632b7a68 [id=640ad152bb4243cda5a7e7242996e2fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.347112 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 898778f8f0b548ec89cb1417d6f07926. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:26.348937 998 consensus_peers.cc:597] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:26.355089 1182 ts_tablet_manager.cc:1929] T 02b434fa058b45529ff3938b053263c6 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:26.355297 1182 log.cc:1199] T 02b434fa058b45529ff3938b053263c6 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/02b434fa058b45529ff3938b053263c6
I20260202 02:52:26.355671 1182 ts_tablet_manager.cc:1950] T 02b434fa058b45529ff3938b053263c6 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:26.356400 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 02b434fa058b45529ff3938b053263c6 (table default.Table_6133c617ff8d40eca5cbde96533acd82 [id=5a502f86d98b42c6a3e274de5bf08544]) successfully deleted
I20260202 02:52:26.359935 887 tablet_service.cc:1505] Processing CreateTablet for tablet 87ff5fbc95b84171a94fe1a83a67a7ba (DEFAULT_TABLE table=default.Table_1cc0209a40ad471fb1c41012a476df79 [id=b941bc2edc03444394d1ba8a596a0d27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.360492 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87ff5fbc95b84171a94fe1a83a67a7ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.361158 1174 tablet_bootstrap.cc:492] T a5622ef925bc46cca76f481ddfd12afd P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:26.361312 1174 ts_tablet_manager.cc:1403] T a5622ef925bc46cca76f481ddfd12afd P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.337s user 0.004s sys 0.020s
I20260202 02:52:26.362067 1174 raft_consensus.cc:359] T a5622ef925bc46cca76f481ddfd12afd P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:26.362254 1174 raft_consensus.cc:385] T a5622ef925bc46cca76f481ddfd12afd P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:26.362339 1174 raft_consensus.cc:740] T a5622ef925bc46cca76f481ddfd12afd P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:26.364954 1174 consensus_queue.cc:260] T a5622ef925bc46cca76f481ddfd12afd P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:26.365738 1174 ts_tablet_manager.cc:1434] T a5622ef925bc46cca76f481ddfd12afd P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260202 02:52:26.365883 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f30b698c177c48ee9a4e577552bf7eee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.366146 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ac33134670747ee860b6d39dd7db547. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.368150 519 master_service.cc:954] ReplaceTablet: received request to replace tablet 761f5a51335a4de5977f5f7fef0d4260 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:26.371305 1174 tablet_bootstrap.cc:492] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:26.372438 2047 raft_consensus.cc:493] T 25765fb687294e9db1289f76530dd877 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:26.372629 2047 raft_consensus.cc:515] T 25765fb687294e9db1289f76530dd877 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.376629 2047 leader_election.cc:290] T 25765fb687294e9db1289f76530dd877 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:26.377775 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "25765fb687294e9db1289f76530dd877" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:26.378710 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_fca1d0422c4d49b080e0203a229298a7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:26.378826 528 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_6bfcc07665864f5dbbabd5685876b6a1" } new_table_name: "default.Table_f8d8615b34774b738faeb83039f9601a" modify_external_catalogs: true
W20260202 02:52:26.379449 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fca1d0422c4d49b080e0203a229298a7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:26.379951 1174 tablet_bootstrap.cc:654] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:26.385776 1173 tablet_bootstrap.cc:492] T 9c1271ce0f194021b588b82ac9d3db79 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:26.385962 1173 ts_tablet_manager.cc:1403] T 9c1271ce0f194021b588b82ac9d3db79 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.071s user 0.003s sys 0.000s
I20260202 02:52:26.386834 1173 raft_consensus.cc:359] T 9c1271ce0f194021b588b82ac9d3db79 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.387039 1173 raft_consensus.cc:385] T 9c1271ce0f194021b588b82ac9d3db79 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:26.387138 1173 raft_consensus.cc:740] T 9c1271ce0f194021b588b82ac9d3db79 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:26.387405 1173 consensus_queue.cc:260] T 9c1271ce0f194021b588b82ac9d3db79 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.388160 1173 ts_tablet_manager.cc:1434] T 9c1271ce0f194021b588b82ac9d3db79 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260202 02:52:26.389575 524 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3857680ccbdc4dd4ad8247da96254434 consensus state has no leader
I20260202 02:52:26.390823 1936 raft_consensus.cc:493] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:26.390990 1936 raft_consensus.cc:515] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:26.391636 1936 leader_election.cc:290] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:26.393178 921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "25765fb687294e9db1289f76530dd877" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:26.393934 714 leader_election.cc:304] T 25765fb687294e9db1289f76530dd877 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:26.394102 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "be993c1512b04df4b92cab3195241d84" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:26.394522 2047 raft_consensus.cc:2749] T 25765fb687294e9db1289f76530dd877 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:26.394548 857 leader_election.cc:343] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:26.394852 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 898778f8f0b548ec89cb1417d6f07926. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.403723 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_a2f4e72d62b34f0abf8290dd08bc279a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:26.404462 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a2f4e72d62b34f0abf8290dd08bc279a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:26.405803 522 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet af5aa68e47bb420eac300754d274fa73 consensus state has no leader
I20260202 02:52:26.408420 1174 tablet_bootstrap.cc:492] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:26.408605 1174 ts_tablet_manager.cc:1403] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.037s user 0.001s sys 0.003s
I20260202 02:52:26.409119 750 tablet_service.cc:1505] Processing CreateTablet for tablet 87ff5fbc95b84171a94fe1a83a67a7ba (DEFAULT_TABLE table=default.Table_1cc0209a40ad471fb1c41012a476df79 [id=b941bc2edc03444394d1ba8a596a0d27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.409453 1174 raft_consensus.cc:359] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.409660 1180 ts_tablet_manager.cc:1929] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:26.410526 1175 tablet_bootstrap.cc:492] T 26eeef6eebbf4f2cb44f69255fb37594 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:26.410679 1175 ts_tablet_manager.cc:1403] T 26eeef6eebbf4f2cb44f69255fb37594 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.426s user 0.007s sys 0.032s
I20260202 02:52:26.411758 1039 tablet_service.cc:1505] Processing CreateTablet for tablet 87ff5fbc95b84171a94fe1a83a67a7ba (DEFAULT_TABLE table=default.Table_1cc0209a40ad471fb1c41012a476df79 [id=b941bc2edc03444394d1ba8a596a0d27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.412114 1175 raft_consensus.cc:359] T 26eeef6eebbf4f2cb44f69255fb37594 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:26.412382 1175 raft_consensus.cc:385] T 26eeef6eebbf4f2cb44f69255fb37594 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:26.412489 1175 raft_consensus.cc:740] T 26eeef6eebbf4f2cb44f69255fb37594 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:26.412807 522 master_service.cc:954] ReplaceTablet: received request to replace tablet 23671408fcbc454cbd918328015cf6e6 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:26.412806 1175 consensus_queue.cc:260] T 26eeef6eebbf4f2cb44f69255fb37594 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:26.414054 1175 ts_tablet_manager.cc:1434] T 26eeef6eebbf4f2cb44f69255fb37594 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260202 02:52:26.414510 526 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 7e94cacaa0714f92ad6a2b2ab6171f44
I20260202 02:52:26.414638 523 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_bed5f0c218b24f529e48138901d1afa0 [id=2a625eeef55b43f19cd4bbeb61e85bca] alter complete (version 1)
I20260202 02:52:26.416700 1860 raft_consensus.cc:493] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:26.416891 1860 raft_consensus.cc:515] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.417568 1860 leader_election.cc:290] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:26.420779 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cb634d4bae554ef3b39eeaa9958adc6c" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:26.421438 998 leader_election.cc:343] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:26.421897 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet 7e94cacaa0714f92ad6a2b2ab6171f44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:26.421912 1180 log.cc:1199] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/3711910e050c4515bed4294d1271d5a9
I20260202 02:52:26.422410 1180 ts_tablet_manager.cc:1950] T 3711910e050c4515bed4294d1271d5a9 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:26.423045 747 tablet_service.cc:1552] Processing DeleteTablet for tablet 7e94cacaa0714f92ad6a2b2ab6171f44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:26.423274 1175 tablet_bootstrap.cc:492] T 33150e1456144b8e9c64cc2a2ac6dcca P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:26.425030 1175 tablet_bootstrap.cc:654] T 33150e1456144b8e9c64cc2a2ac6dcca P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:26.426370 745 tablet_service.cc:1505] Processing CreateTablet for tablet 0ddc726efd3f49489092919c9c6e01fb (DEFAULT_TABLE table=default.Table_ca7d5182d8e340adbb15c7c1457862d7 [id=20cee21f21b14221983488b4c6ea2dc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.409660 1174 raft_consensus.cc:385] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:26.428301 1174 raft_consensus.cc:740] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:26.423286 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 311) took 5798 ms. Trace:
I20260202 02:52:26.428586 1180 rpcz_store.cc:276] 0202 02:52:20.624794 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:20.626809 (+ 2015us) service_pool.cc:225] Handling call
0202 02:52:26.302612 (+5675803us) ts_tablet_manager.cc:1151] Deleting tablet 3711910e050c4515bed4294d1271d5a9
0202 02:52:26.302623 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:26.341216 (+ 38593us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:26.409519 (+ 68303us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:26.423235 (+ 13716us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5504307}
I20260202 02:52:26.428694 1174 consensus_queue.cc:260] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.429091 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 3711910e050c4515bed4294d1271d5a9 (table default.Table_ab29f752f1e140da9038d9e11ee61f39 [id=9ef504467a544f299b48499e49adf27f]) successfully deleted
I20260202 02:52:26.430629 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 315) took 5632 ms. Trace:
I20260202 02:52:26.430742 1180 rpcz_store.cc:276] 0202 02:52:20.797708 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:20.797765 (+ 57us) service_pool.cc:225] Handling call
0202 02:52:26.430438 (+5632673us) ts_tablet_manager.cc:1151] Deleting tablet d0671586e45e45ffb7fd9e1be6e2bbf8
0202 02:52:26.430452 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:26.430561 (+ 109us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5628609}
W20260202 02:52:26.431524 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet d0671586e45e45ffb7fd9e1be6e2bbf8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d0671586e45e45ffb7fd9e1be6e2bbf8 already in progress: creating tablet
I20260202 02:52:26.433779 527 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 2f664f6119ba4d28b5ef1c212e611866
I20260202 02:52:26.434731 519 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 761f5a51335a4de5977f5f7fef0d4260
I20260202 02:52:26.434734 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet 2f664f6119ba4d28b5ef1c212e611866 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:26.435207 519 catalog_manager.cc:6724] ReplaceTablet: tablet 761f5a51335a4de5977f5f7fef0d4260 deleted and replaced by tablet 118753e8cb344b47a35a745aa8fed265
I20260202 02:52:26.436350 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet 761f5a51335a4de5977f5f7fef0d4260 with delete_type TABLET_DATA_DELETED (replaced by tablet 118753e8cb344b47a35a745aa8fed265) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:26.436959 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 761f5a51335a4de5977f5f7fef0d4260 with delete_type TABLET_DATA_DELETED (replaced by tablet 118753e8cb344b47a35a745aa8fed265) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:26.437613 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 26eeef6eebbf4f2cb44f69255fb37594 with delete_type TABLET_DATA_DELETED (replaced by tablet dc4c3f52f4f44a84b0d0e3c9a4a49f09) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:26.439872 901 tablet_service.cc:1552] Processing DeleteTablet for tablet 7e94cacaa0714f92ad6a2b2ab6171f44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:26.439981 525 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_a9b816e5b5b84151b840bfad315deb70" } modify_external_catalogs: true
I20260202 02:52:26.440203 525 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_a9b816e5b5b84151b840bfad315deb70" } modify_external_catalogs: true
I20260202 02:52:26.440249 1174 ts_tablet_manager.cc:1434] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.032s user 0.002s sys 0.000s
I20260202 02:52:26.442662 757 tablet_service.cc:1552] Processing DeleteTablet for tablet d0671586e45e45ffb7fd9e1be6e2bbf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:26.442879 1058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "be993c1512b04df4b92cab3195241d84" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:26.445269 998 consensus_peers.cc:597] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9884c90cf1d245b2a7eaaf00d71b1c3e. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:26.445329 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 761f5a51335a4de5977f5f7fef0d4260 with delete_type TABLET_DATA_DELETED (replaced by tablet 118753e8cb344b47a35a745aa8fed265) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:26.442688 747 tablet_service.cc:1552] Processing DeleteTablet for tablet 2f664f6119ba4d28b5ef1c212e611866 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:42950
W20260202 02:52:26.446218 859 leader_election.cc:343] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:26.446346 859 leader_election.cc:304] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:26.446442 921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cb634d4bae554ef3b39eeaa9958adc6c" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:26.446620 921 raft_consensus.cc:2468] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:26.446822 2019 raft_consensus.cc:2749] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:26.447482 999 leader_election.cc:304] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:26.447924 1860 raft_consensus.cc:2804] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:26.448030 1860 raft_consensus.cc:493] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:26.448120 1860 raft_consensus.cc:3060] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:26.449591 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87ff5fbc95b84171a94fe1a83a67a7ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.450095 901 tablet_service.cc:1552] Processing DeleteTablet for tablet 2f664f6119ba4d28b5ef1c212e611866 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:26.450899 1860 raft_consensus.cc:515] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.451768 1860 leader_election.cc:290] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:26.451995 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 312) took 5655 ms. Trace:
I20260202 02:52:26.452204 1180 rpcz_store.cc:276] 0202 02:52:20.796649 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:20.796727 (+ 78us) service_pool.cc:225] Handling call
0202 02:52:26.451676 (+5654949us) ts_tablet_manager.cc:1151] Deleting tablet 38fdbc6df7ec40b9afe4e0dfe0119e3e
0202 02:52:26.451691 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:26.451835 (+ 144us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5640273}
I20260202 02:52:26.453027 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cb634d4bae554ef3b39eeaa9958adc6c" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
W20260202 02:52:26.453145 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 38fdbc6df7ec40b9afe4e0dfe0119e3e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 38fdbc6df7ec40b9afe4e0dfe0119e3e already in progress: creating tablet
I20260202 02:52:26.453321 1028 tablet_service.cc:1552] Processing DeleteTablet for tablet 9c1271ce0f194021b588b82ac9d3db79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:26.453603 998 leader_election.cc:343] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:26.454022 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 320) took 5622 ms. Trace:
I20260202 02:52:26.455135 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet 08d8d781bb4e4acf915f11afb4fb2e22 with delete_type TABLET_DATA_DELETED (replaced by tablet 7921daa000f54c8dbed04473e21cf0a5) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:26.455024 1180 rpcz_store.cc:276] 0202 02:52:20.831908 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:20.832076 (+ 168us) service_pool.cc:225] Handling call
0202 02:52:26.453849 (+5621773us) ts_tablet_manager.cc:1151] Deleting tablet 026e6ee0b3ae4d938fdf1550044ed1fa
0202 02:52:26.453861 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:26.453960 (+ 99us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5562923}
W20260202 02:52:26.455667 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 026e6ee0b3ae4d938fdf1550044ed1fa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 026e6ee0b3ae4d938fdf1550044ed1fa already in progress: creating tablet
I20260202 02:52:26.456370 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 330) took 4553 ms. Trace:
I20260202 02:52:26.455433 1173 tablet_bootstrap.cc:492] T af5aa68e47bb420eac300754d274fa73 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:26.459199 921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cb634d4bae554ef3b39eeaa9958adc6c" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:26.459381 921 raft_consensus.cc:3060] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:26.460875 889 tablet_service.cc:1505] Processing CreateTablet for tablet 0ddc726efd3f49489092919c9c6e01fb (DEFAULT_TABLE table=default.Table_ca7d5182d8e340adbb15c7c1457862d7 [id=20cee21f21b14221983488b4c6ea2dc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.461290 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ddc726efd3f49489092919c9c6e01fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.462093 921 raft_consensus.cc:2468] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:26.466711 521 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_85ea5ebb89ae4a8fb42f2f5c96ab588c" } new_table_name: "default.Table_11c571ac785448ca9a174c50d15ec37f" modify_external_catalogs: true
I20260202 02:52:26.468693 1036 tablet_service.cc:1505] Processing CreateTablet for tablet 0ddc726efd3f49489092919c9c6e01fb (DEFAULT_TABLE table=default.Table_ca7d5182d8e340adbb15c7c1457862d7 [id=20cee21f21b14221983488b4c6ea2dc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.469259 1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ddc726efd3f49489092919c9c6e01fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.472225 1180 rpcz_store.cc:276] 0202 02:52:21.902421 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:21.903181 (+ 760us) service_pool.cc:225] Handling call
0202 02:52:26.456166 (+4552985us) ts_tablet_manager.cc:1151] Deleting tablet d30557fbe9084cd3860820f69dda8752
0202 02:52:26.456177 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:26.456276 (+ 99us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4546230}
I20260202 02:52:26.472852 1173 tablet_bootstrap.cc:654] T af5aa68e47bb420eac300754d274fa73 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:26.472853 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet d30557fbe9084cd3860820f69dda8752 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d30557fbe9084cd3860820f69dda8752 already in progress: creating tablet
I20260202 02:52:26.473356 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 354) took 4418 ms. Trace:
I20260202 02:52:26.473351 999 leader_election.cc:304] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:26.473462 1180 rpcz_store.cc:276] 0202 02:52:22.054625 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:22.103419 (+ 48794us) service_pool.cc:225] Handling call
0202 02:52:26.473169 (+4369750us) ts_tablet_manager.cc:1151] Deleting tablet 2f5aae089fcc492f9ce55f84674a452c
0202 02:52:26.473183 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:26.473305 (+ 122us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4366333}
W20260202 02:52:26.473893 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 2f5aae089fcc492f9ce55f84674a452c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2f5aae089fcc492f9ce55f84674a452c already in progress: creating tablet
I20260202 02:52:26.474009 1860 raft_consensus.cc:2804] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:26.466724 1174 tablet_bootstrap.cc:492] T 79f694b740954d429145fba9157338ca P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:26.474378 1860 raft_consensus.cc:697] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:26.474787 1860 consensus_queue.cc:237] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.475032 1180 tablet_replica.cc:333] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:26.475759 1180 raft_consensus.cc:2243] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:26.476879 1180 pending_rounds.cc:70] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d: Trying to abort 1 pending ops.
I20260202 02:52:26.477162 1180 pending_rounds.cc:77] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923055127285760 op_type: NO_OP noop_request { }
I20260202 02:52:26.477567 1721 consensus_queue.cc:1048] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:26.477623 1180 raft_consensus.cc:2889] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:26.478025 1180 raft_consensus.cc:2272] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:26.480916 1180 ts_tablet_manager.cc:1916] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:26.483510 527 catalog_manager.cc:5654] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d reported cstate change: term changed from 0 to 1, leader changed from <none> to b1f05820b6034228bdad884cc348716d (127.0.104.131). New cstate: current_term: 1 leader_uuid: "b1f05820b6034228bdad884cc348716d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } }
I20260202 02:52:26.484376 527 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet cb634d4bae554ef3b39eeaa9958adc6c (table default.Table_e5f4ac82d1ef4b85999695997eb9df2a [id=c97e407e519d4a538860f66931f6aefd]). Expected version 2 got 0
I20260202 02:52:26.487007 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87ff5fbc95b84171a94fe1a83a67a7ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.487601 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ddc726efd3f49489092919c9c6e01fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.489269 1882 ts_tablet_manager.cc:1929] T d64f42d187bc4c9f9381c931449ec0cc P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:26.489419 1882 log.cc:1199] T d64f42d187bc4c9f9381c931449ec0cc P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/d64f42d187bc4c9f9381c931449ec0cc
I20260202 02:52:26.489796 1882 ts_tablet_manager.cc:1950] T d64f42d187bc4c9f9381c931449ec0cc P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:26.490581 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 372) took 3200 ms. Trace:
I20260202 02:52:26.490718 1882 rpcz_store.cc:276] 0202 02:52:23.289866 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.290721 (+ 855us) service_pool.cc:225] Handling call
0202 02:52:26.311301 (+3020580us) ts_tablet_manager.cc:1151] Deleting tablet d64f42d187bc4c9f9381c931449ec0cc
0202 02:52:26.328357 (+ 17056us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:26.365902 (+ 37545us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:26.489095 (+123193us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:26.490538 (+ 1443us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2968599}
I20260202 02:52:26.491344 1182 tablet_replica.cc:333] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:26.491552 1182 raft_consensus.cc:2243] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:26.491721 1182 raft_consensus.cc:2272] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:26.493427 1174 tablet_bootstrap.cc:654] T 79f694b740954d429145fba9157338ca P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:26.499033 887 tablet_service.cc:1505] Processing CreateTablet for tablet 0d4efc9e96f54e03bc1a990de90ac0db (DEFAULT_TABLE table=default.Table_6f5a23937e4f4572b861c8889b453872 [id=f4f53cb760134f9198764d93c13a68b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.499547 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d4efc9e96f54e03bc1a990de90ac0db. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:26.509493 528 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8b0368e03f644f1db28dbb57c95c48c1 consensus state has no leader
I20260202 02:52:26.511119 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_0205a93a018940bbbd0bcf218e404fb6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:26.511621 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_ffaecf51f55d45a684b870bda106663a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:26.511948 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0205a93a018940bbbd0bcf218e404fb6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:26.512130 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ffaecf51f55d45a684b870bda106663a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:26.511230 519 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_f034976ca8c24ddd84eba68e745b6821" } modify_external_catalogs: true
I20260202 02:52:26.512620 519 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_f034976ca8c24ddd84eba68e745b6821" } modify_external_catalogs: true
I20260202 02:52:26.513032 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_cccb5565674d4157a97fa0a565bfeea2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:26.513561 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cccb5565674d4157a97fa0a565bfeea2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:26.514061 528 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_5ecb81038c9f48438e74fe874fc83e74" } modify_external_catalogs: true
I20260202 02:52:26.514242 528 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_5ecb81038c9f48438e74fe874fc83e74" } modify_external_catalogs: true
I20260202 02:52:26.515486 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_a9f32b021bdd40d58ea29090ef488d6b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:26.516003 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a9f32b021bdd40d58ea29090ef488d6b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:26.517136 522 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 23671408fcbc454cbd918328015cf6e6
I20260202 02:52:26.517606 522 catalog_manager.cc:6724] ReplaceTablet: tablet 23671408fcbc454cbd918328015cf6e6 deleted and replaced by tablet 32fb916ae82145f195cc8b951069e5da
I20260202 02:52:26.517982 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_64dd9d5476434c7baef533ca4bc498e2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:26.518524 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_64dd9d5476434c7baef533ca4bc498e2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:26.521793 525 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet dc90eeb79ded4739883ad98928476680
I20260202 02:52:26.523684 1039 tablet_service.cc:1552] Processing DeleteTablet for tablet dc90eeb79ded4739883ad98928476680 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:26.523684 740 tablet_service.cc:1552] Processing DeleteTablet for tablet dc90eeb79ded4739883ad98928476680 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:26.524971 519 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 0a5bbb994cfa40f0add3163816b65269
I20260202 02:52:26.530865 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_0601f3c5b49446b3a7ec44443f89a38c" } new_table_name: "default.Table_5a9beb18046647e182f792cc1ce30192" modify_external_catalogs: true
W20260202 02:52:26.531520 521 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 83d198d90a204c6396b569fc1b65abef has no consensus state
I20260202 02:52:26.533257 740 tablet_service.cc:1552] Processing DeleteTablet for tablet 0a5bbb994cfa40f0add3163816b65269 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:26.533285 1039 tablet_service.cc:1552] Processing DeleteTablet for tablet 0a5bbb994cfa40f0add3163816b65269 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:26.536998 901 tablet_service.cc:1552] Processing DeleteTablet for tablet 23671408fcbc454cbd918328015cf6e6 with delete_type TABLET_DATA_DELETED (replaced by tablet 32fb916ae82145f195cc8b951069e5da) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:26.537127 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 23671408fcbc454cbd918328015cf6e6 with delete_type TABLET_DATA_DELETED (replaced by tablet 32fb916ae82145f195cc8b951069e5da) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:26.539179 740 tablet_service.cc:1505] Processing CreateTablet for tablet 0d4efc9e96f54e03bc1a990de90ac0db (DEFAULT_TABLE table=default.Table_6f5a23937e4f4572b861c8889b453872 [id=f4f53cb760134f9198764d93c13a68b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.539755 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d4efc9e96f54e03bc1a990de90ac0db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.542570 885 tablet_service.cc:1552] Processing DeleteTablet for tablet dc90eeb79ded4739883ad98928476680 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:26.543258 1182 ts_tablet_manager.cc:1916] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:26.545285 528 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 8ba28ab7d7ae44059006179067dff3c0
I20260202 02:52:26.547197 747 tablet_service.cc:1552] Processing DeleteTablet for tablet 8ba28ab7d7ae44059006179067dff3c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:26.547806 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 8ba28ab7d7ae44059006179067dff3c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:26.552394 745 tablet_service.cc:1552] Processing DeleteTablet for tablet 23671408fcbc454cbd918328015cf6e6 with delete_type TABLET_DATA_DELETED (replaced by tablet 32fb916ae82145f195cc8b951069e5da) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:26.555496 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_e6661a10889e498eaf10149c7998f713"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:26.556485 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e6661a10889e498eaf10149c7998f713 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:26.558902 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 0a5bbb994cfa40f0add3163816b65269 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:26.563397 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 0d4efc9e96f54e03bc1a990de90ac0db (DEFAULT_TABLE table=default.Table_6f5a23937e4f4572b861c8889b453872 [id=f4f53cb760134f9198764d93c13a68b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.568133 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_a7f3566b065c4bfbafe1349368f05d99"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:26.568794 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a7f3566b065c4bfbafe1349368f05d99 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:26.570418 1173 tablet_bootstrap.cc:492] T af5aa68e47bb420eac300754d274fa73 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:26.572041 1173 ts_tablet_manager.cc:1403] T af5aa68e47bb420eac300754d274fa73 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.118s user 0.002s sys 0.002s
I20260202 02:52:26.572501 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 8ba28ab7d7ae44059006179067dff3c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:26.574165 1173 raft_consensus.cc:359] T af5aa68e47bb420eac300754d274fa73 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.583107 1173 raft_consensus.cc:385] T af5aa68e47bb420eac300754d274fa73 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:26.583797 1173 raft_consensus.cc:740] T af5aa68e47bb420eac300754d274fa73 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:26.585237 1173 consensus_queue.cc:260] T af5aa68e47bb420eac300754d274fa73 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.586869 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_a115529cbd054b83bc0b16a843f8017d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:26.587594 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a115529cbd054b83bc0b16a843f8017d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:26.588012 1174 tablet_bootstrap.cc:492] T 79f694b740954d429145fba9157338ca P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:26.588235 1174 ts_tablet_manager.cc:1403] T 79f694b740954d429145fba9157338ca P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.122s user 0.007s sys 0.000s
I20260202 02:52:26.589460 1174 raft_consensus.cc:359] T 79f694b740954d429145fba9157338ca P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.589749 1174 raft_consensus.cc:385] T 79f694b740954d429145fba9157338ca P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:26.589886 1174 raft_consensus.cc:740] T 79f694b740954d429145fba9157338ca P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:26.590184 1174 consensus_queue.cc:260] T 79f694b740954d429145fba9157338ca P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.590812 1174 ts_tablet_manager.cc:1434] T 79f694b740954d429145fba9157338ca P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:26.591302 525 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_e984fe23a3ab488c9344620087da422d" } new_table_name: "default.Table_25f0023dd22a4bb0a7c2863da66d0002" modify_external_catalogs: true
I20260202 02:52:26.591928 1173 ts_tablet_manager.cc:1434] T af5aa68e47bb420eac300754d274fa73 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.019s user 0.001s sys 0.001s
I20260202 02:52:26.593101 1174 tablet_bootstrap.cc:492] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
W20260202 02:52:26.598666 527 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c8a273ac8cd344719a48fd81945bed0a consensus state has no leader
I20260202 02:52:26.603564 1174 tablet_bootstrap.cc:654] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:26.613273 859 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 166448us
I20260202 02:52:26.613341 526 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_c9a1147d25bd49e48b70b58b373acfc7" } new_table_name: "default.Table_c7654f22d0044539a2eb8a736163a22d" modify_external_catalogs: true
I20260202 02:52:26.620453 527 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_caf008ae2a414bdbbfdec276ff3725ea" } modify_external_catalogs: true
I20260202 02:52:26.620703 527 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_caf008ae2a414bdbbfdec276ff3725ea" } modify_external_catalogs: true
I20260202 02:52:26.609359 745 tablet_service.cc:1505] Processing CreateTablet for tablet 8ba28ab7d7ae44059006179067dff3c0 (DEFAULT_TABLE table=default.Table_5ecb81038c9f48438e74fe874fc83e74 [id=c28e30416db3407b8c20c68091f392b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.625766 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ba28ab7d7ae44059006179067dff3c0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:26.626566 998 consensus_peers.cc:597] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:26.628221 1182 ts_tablet_manager.cc:1929] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:26.615082 1180 ts_tablet_manager.cc:1929] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:26.630506 1180 log.cc:1199] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/6154460838eb4e508f3646d3e3e4cf21
I20260202 02:52:26.630864 523 master_service.cc:954] ReplaceTablet: received request to replace tablet 1116872cc25a4ab5b1a3a6bbba1f0da7 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:26.632824 520 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 20ea39bc5d3d46f5a4ba427277d6c6a3 (table default.Table_3b335a0806154f98a907f551596ba353 [id=0da4db61b3fe4cf4846edd5530b9327f]). Expected version 1 got 0
I20260202 02:52:26.637249 1180 ts_tablet_manager.cc:1950] T 6154460838eb4e508f3646d3e3e4cf21 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:26.637810 523 master_service.cc:954] ReplaceTablet: received request to replace tablet f1c8f7e1dc9644a6b6dd2bc264cce8ef from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:26.638200 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 355) took 4579 ms. Trace:
I20260202 02:52:26.638293 1180 rpcz_store.cc:276] 0202 02:52:22.058898 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:22.106917 (+ 48019us) service_pool.cc:225] Handling call
0202 02:52:26.474728 (+4367811us) ts_tablet_manager.cc:1151] Deleting tablet 6154460838eb4e508f3646d3e3e4cf21
0202 02:52:26.474740 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:26.560267 (+ 85527us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:26.614942 (+ 54675us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:26.638142 (+ 23200us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4367225}
I20260202 02:52:26.641732 519 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 20ea39bc5d3d46f5a4ba427277d6c6a3 (table default.Table_3b335a0806154f98a907f551596ba353 [id=0da4db61b3fe4cf4846edd5530b9327f]). Expected version 1 got 0
I20260202 02:52:26.644773 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 6154460838eb4e508f3646d3e3e4cf21 (table default.Table_6e6ab095d9a6497d8a89bcc865b6f42f [id=7d8f7ac4c2ee4b72b2ff25aff060010f]) successfully deleted
I20260202 02:52:26.648818 520 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_7c01ea0d668a490099dbd42db17162ec" } modify_external_catalogs: true
I20260202 02:52:26.649107 520 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_7c01ea0d668a490099dbd42db17162ec" } modify_external_catalogs: true
I20260202 02:52:26.649269 2042 raft_consensus.cc:493] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:26.651665 2019 consensus_queue.cc:1048] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:26.652611 2042 raft_consensus.cc:515] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.653339 2042 leader_election.cc:290] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:26.654847 715 consensus_peers.cc:597] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d64f42d187bc4c9f9381c931449ec0cc. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:26.654835 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f0e8c5e2fd44370ae150a206f4e6cbc" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:26.655048 1060 raft_consensus.cc:2468] T 2f0e8c5e2fd44370ae150a206f4e6cbc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:26.655870 715 leader_election.cc:304] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:26.656473 2042 raft_consensus.cc:2804] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:26.656576 2042 raft_consensus.cc:493] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:26.656876 2042 raft_consensus.cc:3060] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:26.660928 885 tablet_service.cc:1505] Processing CreateTablet for tablet 8ba28ab7d7ae44059006179067dff3c0 (DEFAULT_TABLE table=default.Table_5ecb81038c9f48438e74fe874fc83e74 [id=c28e30416db3407b8c20c68091f392b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.661389 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ba28ab7d7ae44059006179067dff3c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.664474 2042 raft_consensus.cc:515] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.666385 2042 leader_election.cc:290] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:26.666672 1039 tablet_service.cc:1505] Processing CreateTablet for tablet 8ba28ab7d7ae44059006179067dff3c0 (DEFAULT_TABLE table=default.Table_5ecb81038c9f48438e74fe874fc83e74 [id=c28e30416db3407b8c20c68091f392b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.667167 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ba28ab7d7ae44059006179067dff3c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.671578 1180 tablet_replica.cc:333] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:26.672797 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f0e8c5e2fd44370ae150a206f4e6cbc" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:26.673055 1060 raft_consensus.cc:3060] T 2f0e8c5e2fd44370ae150a206f4e6cbc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:26.673777 1180 raft_consensus.cc:2243] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:26.674198 1180 pending_rounds.cc:70] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d: Trying to abort 1 pending ops.
I20260202 02:52:26.674270 1180 pending_rounds.cc:77] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923036706209792 op_type: NO_OP noop_request { }
I20260202 02:52:26.674399 1180 raft_consensus.cc:2889] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:26.674506 1180 raft_consensus.cc:2272] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:26.674997 1174 tablet_bootstrap.cc:492] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:26.675256 1174 ts_tablet_manager.cc:1403] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.082s user 0.006s sys 0.001s
I20260202 02:52:26.675812 1060 raft_consensus.cc:2468] T 2f0e8c5e2fd44370ae150a206f4e6cbc P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:26.676517 715 leader_election.cc:304] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:26.677891 2047 raft_consensus.cc:493] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:26.678030 2047 raft_consensus.cc:515] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.678689 2047 leader_election.cc:290] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:26.678846 2047 raft_consensus.cc:2804] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:26.678982 2047 raft_consensus.cc:697] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:26.679193 1174 raft_consensus.cc:359] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.679248 2047 consensus_queue.cc:237] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.679406 1174 raft_consensus.cc:385] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:26.679483 1174 raft_consensus.cc:740] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:26.682232 1180 ts_tablet_manager.cc:1916] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:26.683439 714 leader_election.cc:343] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 2f0e8c5e2fd44370ae150a206f4e6cbc
W20260202 02:52:26.683856 714 leader_election.cc:343] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 59990791123c4d4eb620090a1e8e31e1
W20260202 02:52:26.688591 526 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 550895b4010f4088b2cc34acff50e03f consensus state has no leader
W20260202 02:52:26.691817 714 leader_election.cc:343] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 2f0e8c5e2fd44370ae150a206f4e6cbc
I20260202 02:52:26.692845 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "59990791123c4d4eb620090a1e8e31e1" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:26.694258 1062 raft_consensus.cc:2468] T 59990791123c4d4eb620090a1e8e31e1 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:26.695005 715 leader_election.cc:304] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:26.696699 2042 raft_consensus.cc:2804] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:26.696903 2042 raft_consensus.cc:493] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:26.697044 2042 raft_consensus.cc:3060] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:26.700583 747 tablet_service.cc:1505] Processing CreateTablet for tablet 0a5bbb994cfa40f0add3163816b65269 (DEFAULT_TABLE table=default.Table_f034976ca8c24ddd84eba68e745b6821 [id=81e199e091b149a0a103eb3b8a232983]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.707479 1175 tablet_bootstrap.cc:492] T 33150e1456144b8e9c64cc2a2ac6dcca P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:26.708081 1175 ts_tablet_manager.cc:1403] T 33150e1456144b8e9c64cc2a2ac6dcca P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.285s user 0.022s sys 0.031s
I20260202 02:52:26.708621 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a5bbb994cfa40f0add3163816b65269. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.708907 1175 raft_consensus.cc:359] T 33150e1456144b8e9c64cc2a2ac6dcca P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:26.709075 1175 raft_consensus.cc:385] T 33150e1456144b8e9c64cc2a2ac6dcca P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:26.709125 1175 raft_consensus.cc:740] T 33150e1456144b8e9c64cc2a2ac6dcca P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:26.710654 1175 consensus_queue.cc:260] T 33150e1456144b8e9c64cc2a2ac6dcca P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:26.711170 1175 ts_tablet_manager.cc:1434] T 33150e1456144b8e9c64cc2a2ac6dcca P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20260202 02:52:26.714203 889 tablet_service.cc:1505] Processing CreateTablet for tablet 0a5bbb994cfa40f0add3163816b65269 (DEFAULT_TABLE table=default.Table_f034976ca8c24ddd84eba68e745b6821 [id=81e199e091b149a0a103eb3b8a232983]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.714588 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a5bbb994cfa40f0add3163816b65269. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.714648 2042 raft_consensus.cc:515] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.715341 1175 tablet_bootstrap.cc:492] T f94f25f4059a43b8babedb6b45a08a19 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:26.716504 1174 consensus_queue.cc:260] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.718128 1972 raft_consensus.cc:493] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:26.718364 1972 raft_consensus.cc:515] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.719122 1972 leader_election.cc:290] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:26.719228 1174 ts_tablet_manager.cc:1434] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.044s user 0.002s sys 0.000s
I20260202 02:52:26.719558 1174 tablet_bootstrap.cc:492] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:26.721887 1175 tablet_bootstrap.cc:654] T f94f25f4059a43b8babedb6b45a08a19 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:26.723462 885 tablet_service.cc:1505] Processing CreateTablet for tablet 83f30c7f03504cdbb379fc0469324306 (DEFAULT_TABLE table=default.Table_b0e344dc1a824cd9b41aa8991b7911dd [id=8e640d1261014706a067f522bda7ea61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.723840 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83f30c7f03504cdbb379fc0469324306. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.724098 1180 ts_tablet_manager.cc:1929] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:26.724875 891 tablet_service.cc:1505] Processing CreateTablet for tablet 2c2a535de2294cc697d75561d0463642 (DEFAULT_TABLE table=default.Table_950f6f5ae1ef43399de785df25c896e9 [id=cab83305ba6648e5b307441e71671a82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.725221 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c2a535de2294cc697d75561d0463642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.729597 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "91ec01d25f8f441cbe3783cacbee51db" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:26.730391 1860 raft_consensus.cc:493] T 9c1271ce0f194021b588b82ac9d3db79 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:26.730626 1860 raft_consensus.cc:515] T 9c1271ce0f194021b588b82ac9d3db79 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.730705 921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "91ec01d25f8f441cbe3783cacbee51db" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:26.731240 522 master_service.cc:954] ReplaceTablet: received request to replace tablet 0138ba7326674741b0729324735f1de7 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:26.731321 714 leader_election.cc:304] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:26.731751 1972 raft_consensus.cc:2749] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:26.731757 528 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_1fd1e6708a1a4421bd5d679a3b6539e9" } new_table_name: "default.Table_4d0600d5e4824d48ae2a8aabcb4b486b" modify_external_catalogs: true
I20260202 02:52:26.732352 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_843cd26c176e43c588824e3225124494"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:26.732810 521 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_e45bc97d737440b5b6c3ad7bb03d2583" } modify_external_catalogs: true
I20260202 02:52:26.732990 1174 tablet_bootstrap.cc:654] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:26.733021 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_843cd26c176e43c588824e3225124494 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:26.732317 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_58d48519aa304bf8a91507f697a92b04"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:26.733734 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_58d48519aa304bf8a91507f697a92b04 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:26.725281 1180 log.cc:1199] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/0d3633f5f9f54115bc78b9c1b78aba67
I20260202 02:52:26.735248 1180 ts_tablet_manager.cc:1950] T 0d3633f5f9f54115bc78b9c1b78aba67 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:26.736011 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 356) took 4672 ms. Trace:
I20260202 02:52:26.736097 1180 rpcz_store.cc:276] 0202 02:52:22.063456 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:22.107460 (+ 44004us) service_pool.cc:225] Handling call
0202 02:52:26.671456 (+4563996us) ts_tablet_manager.cc:1151] Deleting tablet 0d3633f5f9f54115bc78b9c1b78aba67
0202 02:52:26.671468 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:26.685401 (+ 13933us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:26.688267 (+ 2866us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:26.735950 (+ 47683us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1916,"tablet-delete.queue_time_us":4563596}
I20260202 02:52:26.736544 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 0d3633f5f9f54115bc78b9c1b78aba67 (table default.Table_63bb1b19c34a430e94ad0d7bd9bbe795 [id=af3058593c2747bf99db3b8da1aaedc7]) successfully deleted
I20260202 02:52:26.737010 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d4efc9e96f54e03bc1a990de90ac0db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.738121 1173 tablet_bootstrap.cc:492] T 13cd1383348b458894fdecc22bf24b74 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:26.745654 1972 raft_consensus.cc:493] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:26.745863 1039 tablet_service.cc:1505] Processing CreateTablet for tablet 0a5bbb994cfa40f0add3163816b65269 (DEFAULT_TABLE table=default.Table_f034976ca8c24ddd84eba68e745b6821 [id=81e199e091b149a0a103eb3b8a232983]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.745875 1972 raft_consensus.cc:515] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.746527 1972 leader_election.cc:290] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:26.746532 1033 tablet_service.cc:1505] Processing CreateTablet for tablet 83f30c7f03504cdbb379fc0469324306 (DEFAULT_TABLE table=default.Table_b0e344dc1a824cd9b41aa8991b7911dd [id=8e640d1261014706a067f522bda7ea61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.747294 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a5bbb994cfa40f0add3163816b65269. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.749732 1175 tablet_bootstrap.cc:492] T f94f25f4059a43b8babedb6b45a08a19 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:26.749877 1175 ts_tablet_manager.cc:1403] T f94f25f4059a43b8babedb6b45a08a19 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.035s user 0.002s sys 0.004s
W20260202 02:52:26.750528 714 leader_election.cc:343] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: d0671586e45e45ffb7fd9e1be6e2bbf8
I20260202 02:52:26.751097 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 366) took 4572 ms. Trace:
I20260202 02:52:26.750713 1175 raft_consensus.cc:359] T f94f25f4059a43b8babedb6b45a08a19 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.754317 1175 raft_consensus.cc:385] T f94f25f4059a43b8babedb6b45a08a19 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:26.754406 1175 raft_consensus.cc:740] T f94f25f4059a43b8babedb6b45a08a19 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:26.754451 1182 log.cc:1199] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/874ca4ba4ebc4721bb4b60cba8e01841
I20260202 02:52:26.752534 1180 rpcz_store.cc:276] 0202 02:52:22.178681 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:22.254336 (+ 75655us) service_pool.cc:225] Handling call
0202 02:52:26.750914 (+4496578us) ts_tablet_manager.cc:1151] Deleting tablet 79badde1001b432b95f58b5161ace15c
0202 02:52:26.750926 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:26.751043 (+ 117us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4496302}
I20260202 02:52:26.755393 1175 consensus_queue.cc:260] T f94f25f4059a43b8babedb6b45a08a19 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.757234 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d0671586e45e45ffb7fd9e1be6e2bbf8" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:26.757470 1060 raft_consensus.cc:2468] T d0671586e45e45ffb7fd9e1be6e2bbf8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:26.757929 715 leader_election.cc:304] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:26.758589 1182 ts_tablet_manager.cc:1950] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:26.758899 1972 raft_consensus.cc:2804] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:26.759006 1972 raft_consensus.cc:493] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:26.759024 1182 tablet_replica.cc:333] T fe9ea4c0502f4976b440465992f2ea2c P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:26.759111 1972 raft_consensus.cc:3060] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:26.759202 1182 raft_consensus.cc:2243] T fe9ea4c0502f4976b440465992f2ea2c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:26.762538 1860 leader_election.cc:290] T 9c1271ce0f194021b588b82ac9d3db79 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:26.763157 1882 tablet_replica.cc:333] T d1e0e74f32a24729b250c5751677cfb0 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:26.763527 1882 raft_consensus.cc:2243] T d1e0e74f32a24729b250c5751677cfb0 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:26.763779 1882 raft_consensus.cc:2272] T d1e0e74f32a24729b250c5751677cfb0 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:26.764441 1882 ts_tablet_manager.cc:1916] T d1e0e74f32a24729b250c5751677cfb0 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:26.766526 521 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_e45bc97d737440b5b6c3ad7bb03d2583" } modify_external_catalogs: true
W20260202 02:52:26.766695 999 leader_election.cc:343] T 9c1271ce0f194021b588b82ac9d3db79 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 9c1271ce0f194021b588b82ac9d3db79
W20260202 02:52:26.767217 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 79badde1001b432b95f58b5161ace15c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 79badde1001b432b95f58b5161ace15c already in progress: creating tablet
I20260202 02:52:26.767421 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 874ca4ba4ebc4721bb4b60cba8e01841 (table default.Table_9913ba1a0b0c4f30bc3dd302ee61b92d [id=6777a441613b40cbb21d431744fca23f]) successfully deleted
I20260202 02:52:26.760916 740 tablet_service.cc:1505] Processing CreateTablet for tablet 2c2a535de2294cc697d75561d0463642 (DEFAULT_TABLE table=default.Table_950f6f5ae1ef43399de785df25c896e9 [id=cab83305ba6648e5b307441e71671a82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.761667 1972 raft_consensus.cc:515] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:26.773653 525 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6a3dac5cd1ce47488a50b4c32eca67c7 consensus state has no leader
I20260202 02:52:26.774426 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d0671586e45e45ffb7fd9e1be6e2bbf8" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:26.774750 1060 raft_consensus.cc:3060] T d0671586e45e45ffb7fd9e1be6e2bbf8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:26.776711 1972 leader_election.cc:290] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:26.776994 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c2a535de2294cc697d75561d0463642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.777567 1060 raft_consensus.cc:2468] T d0671586e45e45ffb7fd9e1be6e2bbf8 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:26.777976 1182 raft_consensus.cc:2272] T fe9ea4c0502f4976b440465992f2ea2c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:26.778611 1182 ts_tablet_manager.cc:1916] T fe9ea4c0502f4976b440465992f2ea2c P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:26.781219 1175 ts_tablet_manager.cc:1434] T f94f25f4059a43b8babedb6b45a08a19 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.031s user 0.001s sys 0.001s
I20260202 02:52:26.781760 524 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet af5aa68e47bb420eac300754d274fa73 (table default.Table_12c352712c374a45b7de7a0e634a1327 [id=feb8b9f926434cec8b119843413f8c9b]). Expected version 1 got 0
I20260202 02:52:26.785769 2019 raft_consensus.cc:493] T fe53a8bdbd13417f9ce195c54a52a93c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:26.785925 2019 raft_consensus.cc:515] T fe53a8bdbd13417f9ce195c54a52a93c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:26.786530 2019 leader_election.cc:290] T fe53a8bdbd13417f9ce195c54a52a93c P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:26.787187 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fe53a8bdbd13417f9ce195c54a52a93c" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:26.787475 524 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet 91ec01d25f8f441cbe3783cacbee51db (table default.Table_882b22c2c2b0446c83239be15da80b95 [id=255f4d94812f45d0b4423150098460ff]). Expected version 1 got 0
W20260202 02:52:26.787998 857 leader_election.cc:343] T fe53a8bdbd13417f9ce195c54a52a93c P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:26.788372 1175 tablet_bootstrap.cc:492] T dada209d982d48f3b74a49905f19c5b6 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:26.791123 1175 tablet_bootstrap.cc:654] T dada209d982d48f3b74a49905f19c5b6 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:26.793608 1173 tablet_bootstrap.cc:654] T 13cd1383348b458894fdecc22bf24b74 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:26.794950 2042 leader_election.cc:290] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:26.795523 745 tablet_service.cc:1505] Processing CreateTablet for tablet 83f30c7f03504cdbb379fc0469324306 (DEFAULT_TABLE table=default.Table_b0e344dc1a824cd9b41aa8991b7911dd [id=8e640d1261014706a067f522bda7ea61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.795976 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83f30c7f03504cdbb379fc0469324306. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.797605 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 33150e1456144b8e9c64cc2a2ac6dcca with delete_type TABLET_DATA_DELETED (replaced by tablet 1d98c7a46139413b8f0d3697b74b1e52) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:26.801532 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fe53a8bdbd13417f9ce195c54a52a93c" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:26.801805 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "59990791123c4d4eb620090a1e8e31e1" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:26.802158 1062 raft_consensus.cc:3060] T 59990791123c4d4eb620090a1e8e31e1 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:26.804932 1062 raft_consensus.cc:2468] T 59990791123c4d4eb620090a1e8e31e1 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:26.805498 1182 ts_tablet_manager.cc:1929] T fe9ea4c0502f4976b440465992f2ea2c P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:26.805632 1182 log.cc:1199] T fe9ea4c0502f4976b440465992f2ea2c P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/fe9ea4c0502f4976b440465992f2ea2c
I20260202 02:52:26.805933 1182 ts_tablet_manager.cc:1950] T fe9ea4c0502f4976b440465992f2ea2c P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:26.806849 1182 tablet_replica.cc:333] T ef073880898049e2be93d2cb6dc0c0bb P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
W20260202 02:52:26.806815 859 leader_election.cc:343] T fe53a8bdbd13417f9ce195c54a52a93c P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:26.806965 859 leader_election.cc:304] T fe53a8bdbd13417f9ce195c54a52a93c P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:26.807057 1182 raft_consensus.cc:2243] T ef073880898049e2be93d2cb6dc0c0bb P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:26.807350 2019 raft_consensus.cc:2749] T fe53a8bdbd13417f9ce195c54a52a93c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:26.807917 1182 raft_consensus.cc:2272] T ef073880898049e2be93d2cb6dc0c0bb P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:26.808744 1182 ts_tablet_manager.cc:1916] T ef073880898049e2be93d2cb6dc0c0bb P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:26.809613 715 leader_election.cc:304] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:26.810578 1972 raft_consensus.cc:2804] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:26.810774 1972 raft_consensus.cc:697] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:26.810060 715 leader_election.cc:304] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:26.812490 1972 consensus_queue.cc:237] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.814327 2047 raft_consensus.cc:2804] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:26.814471 2047 raft_consensus.cc:697] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:26.814713 2047 consensus_queue.cc:237] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.815183 1182 ts_tablet_manager.cc:1929] T ef073880898049e2be93d2cb6dc0c0bb P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:26.815287 1182 log.cc:1199] T ef073880898049e2be93d2cb6dc0c0bb P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/ef073880898049e2be93d2cb6dc0c0bb
I20260202 02:52:26.806550 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet fe9ea4c0502f4976b440465992f2ea2c (table default.Table_9181c8b857b540ff846c397f9f1a5484 [id=d2fb3868962547c48e29f6353e5747a9]) successfully deleted
W20260202 02:52:26.821965 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 8ab5c6fbec394b9bab322a197181a618 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8ab5c6fbec394b9bab322a197181a618 already in progress: creating tablet
W20260202 02:52:26.822214 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 3191a80374724effa8389adc842e3448 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3191a80374724effa8389adc842e3448 already in progress: creating tablet
W20260202 02:52:26.823037 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 2616e1bbce4049dbb584550ccc150288 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2616e1bbce4049dbb584550ccc150288 already in progress: creating tablet
I20260202 02:52:26.763828 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 365) took 4606 ms. Trace:
I20260202 02:52:26.823405 1180 rpcz_store.cc:276] 0202 02:52:22.157430 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:22.253102 (+ 95672us) service_pool.cc:225] Handling call
0202 02:52:26.763650 (+4510548us) ts_tablet_manager.cc:1151] Deleting tablet 6df3d11cf578404ca41807f0777b6c53
0202 02:52:26.763662 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:26.763777 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4493475}
I20260202 02:52:26.823597 774 raft_consensus.cc:3060] T 25765fb687294e9db1289f76530dd877 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:26.823833 1180 tablet_replica.cc:333] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:26.824040 1180 raft_consensus.cc:2243] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d [term 2 LEADER]: Raft consensus shutting down.
I20260202 02:52:26.824420 1180 raft_consensus.cc:2272] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:26.825086 1180 ts_tablet_manager.cc:1916] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:26.767149 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9c1271ce0f194021b588b82ac9d3db79" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:26.767822 714 consensus_peers.cc:597] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d64f42d187bc4c9f9381c931449ec0cc. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:26.827878 714 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 60114.4us
W20260202 02:52:26.828655 998 leader_election.cc:343] T 9c1271ce0f194021b588b82ac9d3db79 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:26.828827 998 leader_election.cc:304] T 9c1271ce0f194021b588b82ac9d3db79 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:26.828998 524 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet f94f25f4059a43b8babedb6b45a08a19 (table default.Table_22445ccfd7b64298b92a0f70733f3625 [id=300ea019d2724bee8211d7462710dc26]). Expected version 1 got 0
W20260202 02:52:26.829358 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 6df3d11cf578404ca41807f0777b6c53 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6df3d11cf578404ca41807f0777b6c53 already in progress: creating tablet
I20260202 02:52:26.829836 524 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_9d2dd70237e04edbbc5c56011ed511ce" } new_table_name: "default.Table_731531d5a4d548d8b43461d6b87310dc" modify_external_catalogs: true
I20260202 02:52:26.830329 1860 raft_consensus.cc:2749] T 9c1271ce0f194021b588b82ac9d3db79 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:26.830912 714 leader_election.cc:343] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: d0671586e45e45ffb7fd9e1be6e2bbf8
W20260202 02:52:26.831312 714 leader_election.cc:343] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 59990791123c4d4eb620090a1e8e31e1
I20260202 02:52:26.747310 1036 tablet_service.cc:1505] Processing CreateTablet for tablet 2c2a535de2294cc697d75561d0463642 (DEFAULT_TABLE table=default.Table_950f6f5ae1ef43399de785df25c896e9 [id=cab83305ba6648e5b307441e71671a82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:26.836971 1882 ts_tablet_manager.cc:1929] T d1e0e74f32a24729b250c5751677cfb0 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:26.837224 1882 log.cc:1199] T d1e0e74f32a24729b250c5751677cfb0 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/d1e0e74f32a24729b250c5751677cfb0
I20260202 02:52:26.838009 525 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet af5aa68e47bb420eac300754d274fa73 (table default.Table_12c352712c374a45b7de7a0e634a1327 [id=feb8b9f926434cec8b119843413f8c9b]). Expected version 1 got 0
I20260202 02:52:26.838322 525 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 20ea39bc5d3d46f5a4ba427277d6c6a3 (table default.Table_3b335a0806154f98a907f551596ba353 [id=0da4db61b3fe4cf4846edd5530b9327f]). Expected version 1 got 0
I20260202 02:52:26.838595 525 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 91ec01d25f8f441cbe3783cacbee51db (table default.Table_882b22c2c2b0446c83239be15da80b95 [id=255f4d94812f45d0b4423150098460ff]). Expected version 1 got 0
I20260202 02:52:26.842908 527 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 36db4ee7f5404cffaa3c14ad58e71701
I20260202 02:52:26.843495 523 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet f1c8f7e1dc9644a6b6dd2bc264cce8ef
I20260202 02:52:26.843765 527 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_35e37309ffba469c975e911bffcff4ce" } modify_external_catalogs: true
I20260202 02:52:26.843892 523 catalog_manager.cc:6724] ReplaceTablet: tablet f1c8f7e1dc9644a6b6dd2bc264cce8ef deleted and replaced by tablet 21fe651f45bf43609cf972bef5eca149
I20260202 02:52:26.843941 527 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_35e37309ffba469c975e911bffcff4ce" } modify_external_catalogs: true
I20260202 02:52:26.845057 520 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 757492288d7c44fb91b25b7e6e21e01b
I20260202 02:52:26.844221 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_005b6c9953134fc3a42de2403722bdfe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:26.845645 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_005b6c9953134fc3a42de2403722bdfe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:26.845760 520 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_bed5f0c218b24f529e48138901d1afa0" } modify_external_catalogs: true
I20260202 02:52:26.845947 520 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_bed5f0c218b24f529e48138901d1afa0" } modify_external_catalogs: true
I20260202 02:52:26.847426 750 tablet_service.cc:1552] Processing DeleteTablet for tablet 757492288d7c44fb91b25b7e6e21e01b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:26.847600 747 tablet_service.cc:1552] Processing DeleteTablet for tablet f1c8f7e1dc9644a6b6dd2bc264cce8ef with delete_type TABLET_DATA_DELETED (replaced by tablet 21fe651f45bf43609cf972bef5eca149) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:26.854219 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_a2d18aa48b1e43f5a58ff4bd751992ff"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:26.837522 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83f30c7f03504cdbb379fc0469324306. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.857447 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet f1c8f7e1dc9644a6b6dd2bc264cce8ef with delete_type TABLET_DATA_DELETED (replaced by tablet 21fe651f45bf43609cf972bef5eca149) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:26.860426 1860 raft_consensus.cc:493] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:26.860680 1860 raft_consensus.cc:515] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.861351 1860 leader_election.cc:290] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:26.861737 1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c2a535de2294cc697d75561d0463642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:26.862555 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28a900bcaa8b41718a285e595886d805" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:26.860445 2038 raft_consensus.cc:493] T 38301e13e17c41fcae912c4b38c26a45 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:26.863351 2038 raft_consensus.cc:515] T 38301e13e17c41fcae912c4b38c26a45 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.864041 2038 leader_election.cc:290] T 38301e13e17c41fcae912c4b38c26a45 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:26.865653 998 leader_election.cc:343] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:26.866055 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "38301e13e17c41fcae912c4b38c26a45" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:26.866756 999 leader_election.cc:343] T 38301e13e17c41fcae912c4b38c26a45 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 38301e13e17c41fcae912c4b38c26a45
W20260202 02:52:26.867110 998 leader_election.cc:343] T 38301e13e17c41fcae912c4b38c26a45 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:26.867223 998 leader_election.cc:304] T 38301e13e17c41fcae912c4b38c26a45 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:26.868161 1860 raft_consensus.cc:2749] T 38301e13e17c41fcae912c4b38c26a45 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:26.874677 2047 consensus_queue.cc:1048] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:26.876785 1173 tablet_bootstrap.cc:492] T 13cd1383348b458894fdecc22bf24b74 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:26.876983 1173 ts_tablet_manager.cc:1403] T 13cd1383348b458894fdecc22bf24b74 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.139s user 0.003s sys 0.002s
I20260202 02:52:26.877467 1882 ts_tablet_manager.cc:1950] T d1e0e74f32a24729b250c5751677cfb0 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:26.877844 1173 raft_consensus.cc:359] T 13cd1383348b458894fdecc22bf24b74 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.878057 1173 raft_consensus.cc:385] T 13cd1383348b458894fdecc22bf24b74 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:26.878154 1173 raft_consensus.cc:740] T 13cd1383348b458894fdecc22bf24b74 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:26.878263 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 380) took 3541 ms. Trace:
I20260202 02:52:26.878414 1173 consensus_queue.cc:260] T 13cd1383348b458894fdecc22bf24b74 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.879139 1173 ts_tablet_manager.cc:1434] T 13cd1383348b458894fdecc22bf24b74 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:26.879426 1173 tablet_bootstrap.cc:492] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
W20260202 02:52:26.879437 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a2d18aa48b1e43f5a58ff4bd751992ff in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:26.854403 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 36db4ee7f5404cffaa3c14ad58e71701 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:26.855019 1034 tablet_service.cc:1552] Processing DeleteTablet for tablet 757492288d7c44fb91b25b7e6e21e01b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:26.882050 513 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 28f6161d0d0a4b41af4560f0f3bee0f7 (table default.Table_b5dcfcce852a48a0b1fbd3deed34f4d2 [id=a77fa0c30d27405294adaef676bc76f7], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 28f6161d0d0a4b41af4560f0f3bee0f7 consensus state has no leader
I20260202 02:52:26.886710 740 tablet_service.cc:1552] Processing DeleteTablet for tablet 36db4ee7f5404cffaa3c14ad58e71701 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:26.815652 1182 ts_tablet_manager.cc:1950] T ef073880898049e2be93d2cb6dc0c0bb P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:26.847993 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 36db4ee7f5404cffaa3c14ad58e71701 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:26.888469 1182 tablet_replica.cc:333] T fe53a8bdbd13417f9ce195c54a52a93c P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:26.888650 1182 raft_consensus.cc:2243] T fe53a8bdbd13417f9ce195c54a52a93c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:26.888823 1182 raft_consensus.cc:2272] T fe53a8bdbd13417f9ce195c54a52a93c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:26.889362 1182 ts_tablet_manager.cc:1916] T fe53a8bdbd13417f9ce195c54a52a93c P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:26.848135 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 757492288d7c44fb91b25b7e6e21e01b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:26.865000 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28a900bcaa8b41718a285e595886d805" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:26.889752 923 raft_consensus.cc:2468] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:26.848052 891 tablet_service.cc:1552] Processing DeleteTablet for tablet f1c8f7e1dc9644a6b6dd2bc264cce8ef with delete_type TABLET_DATA_DELETED (replaced by tablet 21fe651f45bf43609cf972bef5eca149) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:26.890002 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet ef073880898049e2be93d2cb6dc0c0bb (table default.Table_e8e9b4cdaf434b6cbbfed1a65bff1fc0 [id=78a7fb3a2eed4565b55d5dacb8a3d7df]) successfully deleted
I20260202 02:52:26.890532 999 leader_election.cc:304] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:26.891017 2038 raft_consensus.cc:2804] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:26.891129 2038 raft_consensus.cc:493] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:26.891240 2038 raft_consensus.cc:3060] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:26.894114 2038 raft_consensus.cc:515] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.895268 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28a900bcaa8b41718a285e595886d805" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
W20260202 02:52:26.895881 998 leader_election.cc:343] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:26.896948 714 consensus_peers.cc:597] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d67e91a022f1401ea724dff898c3b81b. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:26.897929 521 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 1c8317adc47a4c399f2db650ed3d2e53
I20260202 02:52:26.901237 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_8ffb80597a3c44f3ad9abbfffddda115"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:26.901625 528 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 604b3b863c9540d3b55381dbfe1afdc3 consensus state has no leader
I20260202 02:52:26.901623 1180 ts_tablet_manager.cc:1929] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260202 02:52:26.901757 1180 log.cc:1199] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/8095e0c6ff0041fc8b8c8b0778a20b00
I20260202 02:52:26.902005 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_f6c3d9d5bdc94a499428668182f0a8a9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:26.902196 1180 ts_tablet_manager.cc:1950] T 8095e0c6ff0041fc8b8c8b0778a20b00 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
W20260202 02:52:26.902539 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f6c3d9d5bdc94a499428668182f0a8a9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:26.904259 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_9465647c983f4c85b7b2b5454fcc80e7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:26.904845 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9465647c983f4c85b7b2b5454fcc80e7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:26.904814 1721 raft_consensus.cc:493] T 13cd1383348b458894fdecc22bf24b74 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:26.904970 1721 raft_consensus.cc:515] T 13cd1383348b458894fdecc22bf24b74 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.905751 1721 leader_election.cc:290] T 13cd1383348b458894fdecc22bf24b74 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:26.906069 520 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 4e79cd48fefb4ce29ef48beacfc4d26f
I20260202 02:52:26.906723 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 1c8317adc47a4c399f2db650ed3d2e53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:26.907169 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8ffb80597a3c44f3ad9abbfffddda115 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:26.907832 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "13cd1383348b458894fdecc22bf24b74" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:26.908818 998 leader_election.cc:343] T 13cd1383348b458894fdecc22bf24b74 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:26.909067 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 367) took 4729 ms. Trace:
W20260202 02:52:26.909123 999 leader_election.cc:343] T 13cd1383348b458894fdecc22bf24b74 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 13cd1383348b458894fdecc22bf24b74
I20260202 02:52:26.909253 999 leader_election.cc:304] T 13cd1383348b458894fdecc22bf24b74 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:26.909183 1180 rpcz_store.cc:276] 0202 02:52:22.180026 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:22.254615 (+ 74589us) service_pool.cc:225] Handling call
0202 02:52:26.823735 (+4569120us) ts_tablet_manager.cc:1151] Deleting tablet 8095e0c6ff0041fc8b8c8b0778a20b00
0202 02:52:26.823748 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:26.849976 (+ 26228us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:26.901467 (+ 51491us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:26.909028 (+ 7561us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4502608}
I20260202 02:52:26.909531 1180 tablet_replica.cc:333] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:26.909670 1721 raft_consensus.cc:2749] T 13cd1383348b458894fdecc22bf24b74 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:26.909773 1180 raft_consensus.cc:2243] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:26.909922 747 tablet_service.cc:1552] Processing DeleteTablet for tablet 4e79cd48fefb4ce29ef48beacfc4d26f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:26.910107 1180 raft_consensus.cc:2272] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:26.910619 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 4e79cd48fefb4ce29ef48beacfc4d26f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:26.910727 1180 ts_tablet_manager.cc:1916] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:26.920375 1182 ts_tablet_manager.cc:1929] T fe53a8bdbd13417f9ce195c54a52a93c P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:26.920421 2038 leader_election.cc:290] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:26.921984 1182 log.cc:1199] T fe53a8bdbd13417f9ce195c54a52a93c P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/fe53a8bdbd13417f9ce195c54a52a93c
I20260202 02:52:26.922379 1182 ts_tablet_manager.cc:1950] T fe53a8bdbd13417f9ce195c54a52a93c P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:26.923385 1182 tablet_replica.cc:333] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:26.923627 1182 raft_consensus.cc:2243] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:26.924049 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28a900bcaa8b41718a285e595886d805" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:26.924230 923 raft_consensus.cc:3060] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:26.925715 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet fe53a8bdbd13417f9ce195c54a52a93c (table default.Table_935434504bf249f8842274aa67486d8c [id=4f19275f416a425c9b9d100b2a83870a]) successfully deleted
I20260202 02:52:26.927158 923 raft_consensus.cc:2468] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:26.927618 999 leader_election.cc:304] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:26.927951 1721 raft_consensus.cc:2804] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:26.928081 1721 raft_consensus.cc:697] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:26.923864 1182 raft_consensus.cc:2272] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:26.928365 1721 consensus_queue.cc:237] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.928895 1182 ts_tablet_manager.cc:1916] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:26.932726 1180 ts_tablet_manager.cc:1929] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:26.932914 1180 log.cc:1199] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/929eb48881b141718873d0e359e768a8
I20260202 02:52:26.935086 2104 raft_consensus.cc:493] T 26eeef6eebbf4f2cb44f69255fb37594 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:26.935297 2104 raft_consensus.cc:515] T 26eeef6eebbf4f2cb44f69255fb37594 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:26.936524 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26eeef6eebbf4f2cb44f69255fb37594" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:26.937062 859 leader_election.cc:343] T 26eeef6eebbf4f2cb44f69255fb37594 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:26.937654 2104 leader_election.cc:290] T 26eeef6eebbf4f2cb44f69255fb37594 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:26.933318 1180 ts_tablet_manager.cc:1950] T 929eb48881b141718873d0e359e768a8 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:26.938057 527 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet 887ba680d75d4f4fa10ccca9c48ebe2f
I20260202 02:52:26.938063 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26eeef6eebbf4f2cb44f69255fb37594" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:26.938632 857 leader_election.cc:343] T 26eeef6eebbf4f2cb44f69255fb37594 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:26.938769 857 leader_election.cc:304] T 26eeef6eebbf4f2cb44f69255fb37594 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:26.934129 740 tablet_service.cc:1552] Processing DeleteTablet for tablet 1c8317adc47a4c399f2db650ed3d2e53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:26.878351 1882 rpcz_store.cc:276] 0202 02:52:23.336494 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.336557 (+ 63us) service_pool.cc:225] Handling call
0202 02:52:26.634128 (+3297571us) ts_tablet_manager.cc:1151] Deleting tablet d1e0e74f32a24729b250c5751677cfb0
0202 02:52:26.763089 (+128961us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:26.833615 (+ 70526us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:26.836829 (+ 3214us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:26.878209 (+ 41380us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3289570}
I20260202 02:52:26.939674 1882 tablet_replica.cc:333] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:26.939961 1882 raft_consensus.cc:2243] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:26.940405 1882 raft_consensus.cc:2272] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:26.941179 1882 ts_tablet_manager.cc:1916] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:26.942114 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet 1c8317adc47a4c399f2db650ed3d2e53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:26.943778 2104 raft_consensus.cc:2749] T 26eeef6eebbf4f2cb44f69255fb37594 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:26.944242 1174 tablet_bootstrap.cc:492] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:26.944432 1174 ts_tablet_manager.cc:1403] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.225s user 0.013s sys 0.023s
I20260202 02:52:26.945242 1174 raft_consensus.cc:359] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.945428 1174 raft_consensus.cc:385] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:26.945516 1174 raft_consensus.cc:740] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:26.945751 1174 consensus_queue.cc:260] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:26.948813 1039 tablet_service.cc:1552] Processing DeleteTablet for tablet 4e79cd48fefb4ce29ef48beacfc4d26f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:26.948942 1182 ts_tablet_manager.cc:1929] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:26.949080 1182 log.cc:1199] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/026e6ee0b3ae4d938fdf1550044ed1fa
I20260202 02:52:26.949373 1182 ts_tablet_manager.cc:1950] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:26.952428 522 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 0138ba7326674741b0729324735f1de7
I20260202 02:52:26.952482 1182 tablet_replica.cc:333] T 26eeef6eebbf4f2cb44f69255fb37594 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:26.952895 522 catalog_manager.cc:6724] ReplaceTablet: tablet 0138ba7326674741b0729324735f1de7 deleted and replaced by tablet 6671d3b545eb4371bfa71fe703c214f2
I20260202 02:52:26.954016 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_1f6c33e9af15400180eb052e679585e7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:26.956480 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 026e6ee0b3ae4d938fdf1550044ed1fa (table default.Table_f93d759396c04d25ac602a609ecc4962 [id=7482d135c01241b4b3752b88fb1a0b2c]) successfully deleted
I20260202 02:52:26.952678 1182 raft_consensus.cc:2243] T 26eeef6eebbf4f2cb44f69255fb37594 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:26.956794 1182 raft_consensus.cc:2272] T 26eeef6eebbf4f2cb44f69255fb37594 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:26.957046 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 0138ba7326674741b0729324735f1de7 with delete_type TABLET_DATA_DELETED (replaced by tablet 6671d3b545eb4371bfa71fe703c214f2) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:26.957172 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1f6c33e9af15400180eb052e679585e7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:26.957481 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 368) took 4769 ms. Trace:
I20260202 02:52:26.957568 1180 rpcz_store.cc:276] 0202 02:52:22.188029 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:22.270205 (+ 82176us) service_pool.cc:225] Handling call
0202 02:52:26.909440 (+4639235us) ts_tablet_manager.cc:1151] Deleting tablet 929eb48881b141718873d0e359e768a8
0202 02:52:26.909453 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:26.913900 (+ 4447us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:26.932530 (+ 18630us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:26.957458 (+ 24928us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4544034}
I20260202 02:52:26.957937 1182 ts_tablet_manager.cc:1916] T 26eeef6eebbf4f2cb44f69255fb37594 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:26.958670 1175 tablet_bootstrap.cc:492] T dada209d982d48f3b74a49905f19c5b6 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:26.958803 1175 ts_tablet_manager.cc:1403] T dada209d982d48f3b74a49905f19c5b6 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.171s user 0.009s sys 0.013s
I20260202 02:52:26.958815 1173 tablet_bootstrap.cc:654] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:26.925587 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 8095e0c6ff0041fc8b8c8b0778a20b00 (table default.Table_4e6c3886f58b49698a00d45b957f4c9a [id=b5cc03b77f414304b29c5fb4bb7dc3ee]) successfully deleted
W20260202 02:52:26.959542 513 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 49059.7us
I20260202 02:52:26.959483 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_e87d06c7523440f7a739370378832d12"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:26.960126 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e87d06c7523440f7a739370378832d12 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:26.960666 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 8ae6390a856d4c838e0689cd0c6c5c92 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8ae6390a856d4c838e0689cd0c6c5c92 already in progress: creating tablet
I20260202 02:52:26.961437 523 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_6f5a23937e4f4572b861c8889b453872" } new_table_name: "default.Table_3cc05aefd1034220bc3d74f6428e11f7" modify_external_catalogs: true
I20260202 02:52:26.964350 520 master_service.cc:954] ReplaceTablet: received request to replace tablet 1c8317adc47a4c399f2db650ed3d2e53 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:26.966176 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet 13cd1383348b458894fdecc22bf24b74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:26.966246 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 7ea7a5d8a92b4ac0903483627258ae66 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7ea7a5d8a92b4ac0903483627258ae66 already in progress: creating tablet
I20260202 02:52:26.961366 745 tablet_service.cc:1552] Processing DeleteTablet for tablet 0138ba7326674741b0729324735f1de7 with delete_type TABLET_DATA_DELETED (replaced by tablet 6671d3b545eb4371bfa71fe703c214f2) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:26.967633 522 master_service.cc:954] ReplaceTablet: received request to replace tablet 550895b4010f4088b2cc34acff50e03f from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:26.970885 1039 tablet_service.cc:1552] Processing DeleteTablet for tablet 0138ba7326674741b0729324735f1de7 with delete_type TABLET_DATA_DELETED (replaced by tablet 6671d3b545eb4371bfa71fe703c214f2) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:26.971045 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 929eb48881b141718873d0e359e768a8 (table default.Table_b0e344dc1a824cd9b41aa8991b7911dd [id=8e640d1261014706a067f522bda7ea61]) successfully deleted
W20260202 02:52:26.971359 524 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cafa1d1231c54a608422695409720112 has no consensus state
I20260202 02:52:26.971865 1175 raft_consensus.cc:359] T dada209d982d48f3b74a49905f19c5b6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:26.972066 1175 raft_consensus.cc:385] T dada209d982d48f3b74a49905f19c5b6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:26.972116 1175 raft_consensus.cc:740] T dada209d982d48f3b74a49905f19c5b6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:26.972807 1972 raft_consensus.cc:493] T a5622ef925bc46cca76f481ddfd12afd P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:26.972957 1972 raft_consensus.cc:515] T a5622ef925bc46cca76f481ddfd12afd P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:26.973531 1972 leader_election.cc:290] T a5622ef925bc46cca76f481ddfd12afd P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:26.975036 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a5622ef925bc46cca76f481ddfd12afd" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:26.975703 1182 ts_tablet_manager.cc:1929] T 26eeef6eebbf4f2cb44f69255fb37594 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:26.975816 1182 log.cc:1199] T 26eeef6eebbf4f2cb44f69255fb37594 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/26eeef6eebbf4f2cb44f69255fb37594
W20260202 02:52:26.975785 714 leader_election.cc:343] T a5622ef925bc46cca76f481ddfd12afd P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: a5622ef925bc46cca76f481ddfd12afd
I20260202 02:52:26.976223 1182 ts_tablet_manager.cc:1950] T 26eeef6eebbf4f2cb44f69255fb37594 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:26.976433 715 leader_election.cc:304] T a5622ef925bc46cca76f481ddfd12afd P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:26.976941 1972 raft_consensus.cc:2749] T a5622ef925bc46cca76f481ddfd12afd P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:26.977187 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 26eeef6eebbf4f2cb44f69255fb37594 (table default.Table_82c14c20a7bf41b487efaff52ebc15a9 [id=7fbb33c105f149ef98505dcf622930c9]) successfully deleted
I20260202 02:52:26.977421 1175 consensus_queue.cc:260] T dada209d982d48f3b74a49905f19c5b6 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:26.977988 1175 ts_tablet_manager.cc:1434] T dada209d982d48f3b74a49905f19c5b6 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.019s user 0.001s sys 0.001s
I20260202 02:52:26.983655 1175 tablet_bootstrap.cc:492] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:26.984238 1182 tablet_replica.cc:333] T 761f5a51335a4de5977f5f7fef0d4260 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:26.984449 1182 raft_consensus.cc:2243] T 761f5a51335a4de5977f5f7fef0d4260 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260202 02:52:26.984258 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 7e94cacaa0714f92ad6a2b2ab6171f44 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7e94cacaa0714f92ad6a2b2ab6171f44 already in progress: creating tablet
I20260202 02:52:26.984618 1182 raft_consensus.cc:2272] T 761f5a51335a4de5977f5f7fef0d4260 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:26.984890 2117 tablet.cc:1722] T 25765fb687294e9db1289f76530dd877 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:26.976532 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet d1e0e74f32a24729b250c5751677cfb0 (table default.Table_5db99844c1bb484a863a9e289a19da3e [id=f1f6c97fc88340da8bab0347ad6430ab]) successfully deleted
I20260202 02:52:26.962000 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 369) took 4764 ms. Trace:
I20260202 02:52:26.985898 1180 rpcz_store.cc:276] 0202 02:52:22.197309 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:22.282347 (+ 85038us) service_pool.cc:225] Handling call
0202 02:52:26.961833 (+4679486us) ts_tablet_manager.cc:1151] Deleting tablet fe53a8bdbd13417f9ce195c54a52a93c
0202 02:52:26.961846 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:26.961980 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4591684}
I20260202 02:52:26.986490 525 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 91ec01d25f8f441cbe3783cacbee51db (table default.Table_882b22c2c2b0446c83239be15da80b95 [id=255f4d94812f45d0b4423150098460ff]). Expected version 1 got 0
I20260202 02:52:26.987864 525 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_6959068a5e634039a454c64744f31522" } new_table_name: "default.Table_58d1c16bc0404476afd27abf2f3ce9da" modify_external_catalogs: true
W20260202 02:52:26.988200 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet fe53a8bdbd13417f9ce195c54a52a93c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fe53a8bdbd13417f9ce195c54a52a93c already in progress: creating tablet
I20260202 02:52:26.989274 1182 ts_tablet_manager.cc:1916] T 761f5a51335a4de5977f5f7fef0d4260 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:26.992096 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 385) took 3846 ms. Trace:
I20260202 02:52:26.992257 1180 rpcz_store.cc:276] 0202 02:52:23.145899 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.146024 (+ 125us) service_pool.cc:225] Handling call
0202 02:52:26.991957 (+3845933us) ts_tablet_manager.cc:1151] Deleting tablet 68823eff1bc04108ba83487605aec0ea
0202 02:52:26.991969 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:26.992073 (+ 104us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3845616}
W20260202 02:52:26.992790 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 68823eff1bc04108ba83487605aec0ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 68823eff1bc04108ba83487605aec0ea already in progress: creating tablet
I20260202 02:52:26.992883 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 387) took 3846 ms. Trace:
I20260202 02:52:26.992944 1180 rpcz_store.cc:276] 0202 02:52:23.146210 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.146250 (+ 40us) service_pool.cc:225] Handling call
0202 02:52:26.992770 (+3846520us) ts_tablet_manager.cc:1151] Deleting tablet 59ad016e15ec4a488921eb564029b689
0202 02:52:26.992781 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:26.992867 (+ 86us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3846349}
W20260202 02:52:26.993332 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 59ad016e15ec4a488921eb564029b689 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 59ad016e15ec4a488921eb564029b689 already in progress: creating tablet
I20260202 02:52:26.997025 1175 tablet_bootstrap.cc:654] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:26.997313 1180 tablet_replica.cc:333] T 79f694b740954d429145fba9157338ca P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:26.997551 1180 raft_consensus.cc:2243] T 79f694b740954d429145fba9157338ca P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:26.997792 1180 raft_consensus.cc:2272] T 79f694b740954d429145fba9157338ca P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:26.998893 1180 ts_tablet_manager.cc:1916] T 79f694b740954d429145fba9157338ca P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:26.999939 2047 raft_consensus.cc:493] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:27.000074 2047 raft_consensus.cc:515] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.000223 1174 ts_tablet_manager.cc:1434] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.056s user 0.015s sys 0.000s
I20260202 02:52:27.000540 1174 tablet_bootstrap.cc:492] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:27.000908 2047 leader_election.cc:290] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:27.001207 1882 ts_tablet_manager.cc:1929] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:27.001308 1882 log.cc:1199] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/9884c90cf1d245b2a7eaaf00d71b1c3e
I20260202 02:52:27.001616 1882 ts_tablet_manager.cc:1950] T 9884c90cf1d245b2a7eaaf00d71b1c3e P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:27.001973 1174 tablet_bootstrap.cc:654] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:27.003968 714 leader_election.cc:343] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 2b8ba9dd450948ada395207ae1c5b325
I20260202 02:52:27.006438 1182 ts_tablet_manager.cc:1929] T 761f5a51335a4de5977f5f7fef0d4260 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:27.006655 1182 log.cc:1199] T 761f5a51335a4de5977f5f7fef0d4260 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/761f5a51335a4de5977f5f7fef0d4260
I20260202 02:52:27.007048 1182 ts_tablet_manager.cc:1950] T 761f5a51335a4de5977f5f7fef0d4260 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:27.008034 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 761f5a51335a4de5977f5f7fef0d4260 (table default.Table_7913f8fb6eac487a971545a11cf257a9 [id=b40664207c364307ba6f5ade490a21d6]) successfully deleted
W20260202 02:52:27.008381 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 2f664f6119ba4d28b5ef1c212e611866 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2f664f6119ba4d28b5ef1c212e611866 already in progress: creating tablet
W20260202 02:52:27.008666 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 23671408fcbc454cbd918328015cf6e6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 23671408fcbc454cbd918328015cf6e6 already in progress: creating tablet
W20260202 02:52:27.008915 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet dc90eeb79ded4739883ad98928476680 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dc90eeb79ded4739883ad98928476680 already in progress: creating tablet
W20260202 02:52:27.009109 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 0a5bbb994cfa40f0add3163816b65269 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0a5bbb994cfa40f0add3163816b65269 already in progress: creating tablet
W20260202 02:52:27.009320 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 8ba28ab7d7ae44059006179067dff3c0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8ba28ab7d7ae44059006179067dff3c0 already in progress: creating tablet
I20260202 02:52:27.009338 1182 tablet_replica.cc:333] T 33150e1456144b8e9c64cc2a2ac6dcca P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:27.009511 1182 raft_consensus.cc:2243] T 33150e1456144b8e9c64cc2a2ac6dcca P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:27.009660 1182 raft_consensus.cc:2272] T 33150e1456144b8e9c64cc2a2ac6dcca P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:27.010696 1182 ts_tablet_manager.cc:1916] T 33150e1456144b8e9c64cc2a2ac6dcca P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:27.015007 1174 tablet_bootstrap.cc:492] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:27.015157 1174 ts_tablet_manager.cc:1403] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20260202 02:52:27.015797 528 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet 25765fb687294e9db1289f76530dd877 (table default.Table_03cb720f804b44e2b29f8cc47336b2db [id=ea0c52a72da8422ba14345fbfc242d27]). Expected version 2 got 0
I20260202 02:52:27.015826 1174 raft_consensus.cc:359] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:27.015977 1174 raft_consensus.cc:385] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:27.016021 1174 raft_consensus.cc:740] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:27.016225 1174 consensus_queue.cc:260] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:27.018627 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_d525711c9e014b459f6cd3732bae2eb6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:27.019114 1174 ts_tablet_manager.cc:1434] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
W20260202 02:52:27.019220 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d525711c9e014b459f6cd3732bae2eb6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:27.019412 1174 tablet_bootstrap.cc:492] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
W20260202 02:52:27.020239 523 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0d4efc9e96f54e03bc1a990de90ac0db consensus state has no leader
I20260202 02:52:27.020659 523 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_5a9beb18046647e182f792cc1ce30192" } new_table_name: "default.Table_5a703dc84eb34674bdc5793ac7295098" modify_external_catalogs: true
I20260202 02:52:27.021390 1180 ts_tablet_manager.cc:1929] T 79f694b740954d429145fba9157338ca P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:27.021492 1180 log.cc:1199] T 79f694b740954d429145fba9157338ca P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/79f694b740954d429145fba9157338ca
I20260202 02:52:27.021862 1174 tablet_bootstrap.cc:654] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:27.022140 527 master_service.cc:954] ReplaceTablet: received request to replace tablet b9fc33e8efc04734a951f7fcf9048142 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:27.024041 891 tablet_service.cc:1552] Processing DeleteTablet for tablet dada209d982d48f3b74a49905f19c5b6 with delete_type TABLET_DATA_DELETED (replaced by tablet 67a8c9a1998b47b3b214a06ce62381e9) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:27.026787 1175 tablet_bootstrap.cc:492] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:27.026930 1175 ts_tablet_manager.cc:1403] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.043s user 0.004s sys 0.000s
I20260202 02:52:27.027721 1175 raft_consensus.cc:359] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.027901 1175 raft_consensus.cc:385] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:27.027961 1175 raft_consensus.cc:740] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:27.028183 1175 consensus_queue.cc:260] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.028591 1180 ts_tablet_manager.cc:1950] T 79f694b740954d429145fba9157338ca P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:27.030099 1182 ts_tablet_manager.cc:1929] T 33150e1456144b8e9c64cc2a2ac6dcca P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:27.030191 1182 log.cc:1199] T 33150e1456144b8e9c64cc2a2ac6dcca P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/33150e1456144b8e9c64cc2a2ac6dcca
I20260202 02:52:27.030481 1182 ts_tablet_manager.cc:1950] T 33150e1456144b8e9c64cc2a2ac6dcca P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:27.031797 1175 ts_tablet_manager.cc:1434] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260202 02:52:27.031934 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 386) took 3885 ms. Trace:
I20260202 02:52:27.032038 1180 rpcz_store.cc:276] 0202 02:52:23.146021 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.146067 (+ 46us) service_pool.cc:225] Handling call
0202 02:52:26.997254 (+3851187us) ts_tablet_manager.cc:1151] Deleting tablet 79f694b740954d429145fba9157338ca
0202 02:52:26.997265 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:27.007876 (+ 10611us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:27.021279 (+ 13403us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:27.031909 (+ 10630us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3850615}
I20260202 02:52:27.032505 1182 tablet_replica.cc:333] T 757492288d7c44fb91b25b7e6e21e01b P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:27.032507 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 79f694b740954d429145fba9157338ca (table default.Table_662111850b514df5ad7c69aa21783397 [id=d595db36d702401ab81d5401ad3a76e2]) successfully deleted
I20260202 02:52:27.032668 1175 tablet_bootstrap.cc:492] T b7880cb051c24bc998003f0effed3cf8 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:27.032673 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 33150e1456144b8e9c64cc2a2ac6dcca (table default.Table_38e8a46dd2dd498baa45935fa4670b09 [id=97791994f92a47519b28e44ec02df5ba]) successfully deleted
W20260202 02:52:27.032948 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 36db4ee7f5404cffaa3c14ad58e71701 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 36db4ee7f5404cffaa3c14ad58e71701 already in progress: creating tablet
I20260202 02:52:27.032667 1182 raft_consensus.cc:2243] T 757492288d7c44fb91b25b7e6e21e01b P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:27.033939 1182 raft_consensus.cc:2272] T 757492288d7c44fb91b25b7e6e21e01b P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:27.034097 1180 tablet_replica.cc:333] T a5622ef925bc46cca76f481ddfd12afd P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:27.034333 1175 tablet_bootstrap.cc:654] T b7880cb051c24bc998003f0effed3cf8 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:27.034334 1180 raft_consensus.cc:2243] T a5622ef925bc46cca76f481ddfd12afd P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:27.034451 1182 ts_tablet_manager.cc:1916] T 757492288d7c44fb91b25b7e6e21e01b P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:27.034603 1180 raft_consensus.cc:2272] T a5622ef925bc46cca76f481ddfd12afd P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:27.035152 1180 ts_tablet_manager.cc:1916] T a5622ef925bc46cca76f481ddfd12afd P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:27.038451 528 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet 20ea39bc5d3d46f5a4ba427277d6c6a3 (table default.Table_3b335a0806154f98a907f551596ba353 [id=0da4db61b3fe4cf4846edd5530b9327f]). Expected version 1 got 0
W20260202 02:52:27.047844 714 consensus_peers.cc:597] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 761f5a51335a4de5977f5f7fef0d4260. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:27.050309 1182 ts_tablet_manager.cc:1929] T 757492288d7c44fb91b25b7e6e21e01b P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:27.050478 1182 log.cc:1199] T 757492288d7c44fb91b25b7e6e21e01b P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/757492288d7c44fb91b25b7e6e21e01b
I20260202 02:52:27.050860 1182 ts_tablet_manager.cc:1950] T 757492288d7c44fb91b25b7e6e21e01b P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:27.052067 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 757492288d7c44fb91b25b7e6e21e01b (table default.Table_7c01ea0d668a490099dbd42db17162ec [id=02e9e243f710428091de2f9497532c37]) successfully deleted
W20260202 02:52:27.054404 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet f1c8f7e1dc9644a6b6dd2bc264cce8ef because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f1c8f7e1dc9644a6b6dd2bc264cce8ef already in progress: creating tablet
W20260202 02:52:27.054921 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 1c8317adc47a4c399f2db650ed3d2e53 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1c8317adc47a4c399f2db650ed3d2e53 already in progress: creating tablet
I20260202 02:52:27.055169 1182 tablet_replica.cc:333] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:27.055408 1182 raft_consensus.cc:2243] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:27.055382 528 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_e6661a10889e498eaf10149c7998f713" } modify_external_catalogs: true
I20260202 02:52:27.055614 1182 raft_consensus.cc:2272] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:27.055773 528 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_e6661a10889e498eaf10149c7998f713" } modify_external_catalogs: true
I20260202 02:52:27.056391 521 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_569ab9ec7c9c4870b31eba3616ee73d3" } modify_external_catalogs: true
I20260202 02:52:27.056537 521 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_569ab9ec7c9c4870b31eba3616ee73d3" } modify_external_catalogs: true
I20260202 02:52:27.057055 1182 ts_tablet_manager.cc:1916] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:27.057317 1180 ts_tablet_manager.cc:1929] T a5622ef925bc46cca76f481ddfd12afd P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:27.057435 1180 log.cc:1199] T a5622ef925bc46cca76f481ddfd12afd P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/a5622ef925bc46cca76f481ddfd12afd
I20260202 02:52:27.057780 1180 ts_tablet_manager.cc:1950] T a5622ef925bc46cca76f481ddfd12afd P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:27.058568 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 388) took 3912 ms. Trace:
I20260202 02:52:27.058667 1180 rpcz_store.cc:276] 0202 02:52:23.146529 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.146563 (+ 34us) service_pool.cc:225] Handling call
0202 02:52:27.033992 (+3887429us) ts_tablet_manager.cc:1151] Deleting tablet a5622ef925bc46cca76f481ddfd12afd
0202 02:52:27.034055 (+ 63us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:27.048892 (+ 14837us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:27.057189 (+ 8297us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:27.058544 (+ 1355us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3887244}
I20260202 02:52:27.059180 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet a5622ef925bc46cca76f481ddfd12afd (table default.Table_3d26f7a34d0c48379e3cebda66e1ef13 [id=45e6d83589fa493195b06f633e295b00]) successfully deleted
I20260202 02:52:27.062862 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 389) took 3915 ms. Trace:
I20260202 02:52:27.063421 2093 raft_consensus.cc:493] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:27.063562 2093 raft_consensus.cc:515] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.064210 2093 leader_election.cc:290] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
W20260202 02:52:27.065063 525 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a44aa53c6b98468a8e36478f2fc9cdda has no consensus state
I20260202 02:52:27.064947 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79c73f4a92044bafb5b8384229cb3a4d" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:27.063196 1180 rpcz_store.cc:276] 0202 02:52:23.146914 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.146984 (+ 70us) service_pool.cc:225] Handling call
0202 02:52:27.062735 (+3915751us) ts_tablet_manager.cc:1151] Deleting tablet 723921dfe54c4e66af5cab76a25fee0a
0202 02:52:27.062749 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:27.062838 (+ 89us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3915566}
W20260202 02:52:27.065653 857 leader_election.cc:343] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:27.065973 523 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c8a273ac8cd344719a48fd81945bed0a consensus state has no leader
W20260202 02:52:27.066802 513 catalog_manager.cc:4952] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 723921dfe54c4e66af5cab76a25fee0a because the tablet was not found. No further retry: Not found: Tablet not found: 723921dfe54c4e66af5cab76a25fee0a
I20260202 02:52:27.067206 1180 tablet_replica.cc:333] T 0186572e7bb9499a932a636ab280f544 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:27.067440 1180 raft_consensus.cc:2243] T 0186572e7bb9499a932a636ab280f544 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:27.067672 1180 raft_consensus.cc:2272] T 0186572e7bb9499a932a636ab280f544 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:27.069706 1180 ts_tablet_manager.cc:1916] T 0186572e7bb9499a932a636ab280f544 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:27.081048 524 catalog_manager.cc:5654] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d reported cstate change: term changed from 0 to 1, leader changed from <none> to b1f05820b6034228bdad884cc348716d (127.0.104.131). New cstate: current_term: 1 leader_uuid: "b1f05820b6034228bdad884cc348716d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } }
I20260202 02:52:27.081471 891 tablet_service.cc:1505] Processing CreateTablet for tablet 85f754f006d6432bb1cba87043903ac2 (DEFAULT_TABLE table=default.Table_e6661a10889e498eaf10149c7998f713 [id=75ef69f7059b4db48ca454bd21afc21c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.081547 524 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 28a900bcaa8b41718a285e595886d805 (table default.Table_911bb209b4554ee8b9bc09edaf488d63 [id=cd07f9746ec94d9ebe956a7ca0d19b1c]). Expected version 1 got 0
I20260202 02:52:27.081863 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85f754f006d6432bb1cba87043903ac2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.082547 740 tablet_service.cc:1505] Processing CreateTablet for tablet 9d0212e64d934968bdbdc627c69874b1 (DEFAULT_TABLE table=default.Table_cccb5565674d4157a97fa0a565bfeea2 [id=afc25faf5ca44e2e8f50ea12003cdd9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.082952 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d0212e64d934968bdbdc627c69874b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.084960 887 tablet_service.cc:1505] Processing CreateTablet for tablet 9d0212e64d934968bdbdc627c69874b1 (DEFAULT_TABLE table=default.Table_cccb5565674d4157a97fa0a565bfeea2 [id=afc25faf5ca44e2e8f50ea12003cdd9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.085337 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d0212e64d934968bdbdc627c69874b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.087587 526 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 79c73f4a92044bafb5b8384229cb3a4d (table default.Table_a15c97bb32284dcfb36cdc47921b2346 [id=dc77705cb5704fd58af1eed3edd45675]). Expected version 1 got 0
I20260202 02:52:27.088680 885 tablet_service.cc:1505] Processing CreateTablet for tablet 957812ed9c424a62a27205179aab7f75 (DEFAULT_TABLE table=default.Table_64dd9d5476434c7baef533ca4bc498e2 [id=9e57e9c5a56648569bc62440b96e4935]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.089057 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 957812ed9c424a62a27205179aab7f75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.089478 747 tablet_service.cc:1505] Processing CreateTablet for tablet 957812ed9c424a62a27205179aab7f75 (DEFAULT_TABLE table=default.Table_64dd9d5476434c7baef533ca4bc498e2 [id=9e57e9c5a56648569bc62440b96e4935]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.089818 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 957812ed9c424a62a27205179aab7f75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.091689 519 master_service.cc:954] ReplaceTablet: received request to replace tablet 940e656cd34e4e53896fda6d38132b6e from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:27.093202 745 tablet_service.cc:1505] Processing CreateTablet for tablet 85f754f006d6432bb1cba87043903ac2 (DEFAULT_TABLE table=default.Table_e6661a10889e498eaf10149c7998f713 [id=75ef69f7059b4db48ca454bd21afc21c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.093639 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85f754f006d6432bb1cba87043903ac2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.094149 526 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 79c73f4a92044bafb5b8384229cb3a4d (table default.Table_a15c97bb32284dcfb36cdc47921b2346 [id=dc77705cb5704fd58af1eed3edd45675]). Expected version 1 got 0
I20260202 02:52:27.103231 750 tablet_service.cc:1505] Processing CreateTablet for tablet f1fadf2cb5be4ec8980871fc96ee9730 (DEFAULT_TABLE table=default.Table_a7d94a8b6efd406c845b24e72a7d4d25 [id=16752003a5914ecda6cb7a7264f1c285]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.103703 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1fadf2cb5be4ec8980871fc96ee9730. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.107191 889 tablet_service.cc:1505] Processing CreateTablet for tablet a44aa53c6b98468a8e36478f2fc9cdda (DEFAULT_TABLE table=default.Table_58d1c16bc0404476afd27abf2f3ce9da [id=c2dc79921cd6498f9470ed06f7712f0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.107633 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a44aa53c6b98468a8e36478f2fc9cdda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.111817 891 tablet_service.cc:1505] Processing CreateTablet for tablet ce757ddc5c144d1fbaec84c9723cdbf2 (DEFAULT_TABLE table=default.Table_a9f32b021bdd40d58ea29090ef488d6b [id=a7c84b08fa1f494cba4c110c53ec91ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.114022 757 tablet_service.cc:1505] Processing CreateTablet for tablet 767c004aa03d493c8d2301e1a49787e7 (DEFAULT_TABLE table=default.Table_d9237a0573e5464a995da7c7bafc08a6 [id=1ad7ef33b0d34dde8d4abbff6de61e71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.114735 901 tablet_service.cc:1505] Processing CreateTablet for tablet ec3511e5c58b4dba9e48e2b73df833a4 (DEFAULT_TABLE table=default.Table_190bbbb8a6a6454cb95f513c1c72b8fd [id=a22166fa338d40edbd0ca62ff0ce4778]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.112123 885 tablet_service.cc:1505] Processing CreateTablet for tablet f1fadf2cb5be4ec8980871fc96ee9730 (DEFAULT_TABLE table=default.Table_a7d94a8b6efd406c845b24e72a7d4d25 [id=16752003a5914ecda6cb7a7264f1c285]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.115684 884 tablet_service.cc:1505] Processing CreateTablet for tablet 41a4143247ed4eed98defbdfe2895372 (DEFAULT_TABLE table=default.Table_d24b7b40906a4dfba59c9907d7b3cd1a [id=9ae42d61d911493caaa622f688e6c9c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.117228 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce757ddc5c144d1fbaec84c9723cdbf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.117970 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec3511e5c58b4dba9e48e2b73df833a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.119254 522 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 550895b4010f4088b2cc34acff50e03f
I20260202 02:52:27.119457 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1fadf2cb5be4ec8980871fc96ee9730. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.119685 522 catalog_manager.cc:6724] ReplaceTablet: tablet 550895b4010f4088b2cc34acff50e03f deleted and replaced by tablet 5416eab7a0914ecb855fbccde4894285
I20260202 02:52:27.119926 892 tablet_service.cc:1505] Processing CreateTablet for tablet 9e67b8b719c1435b995c22ee0abc56a0 (DEFAULT_TABLE table=default.Table_9121c17ae28a45f2a1ce941e70260558 [id=7d630f11f4d74ef4b76057b4893b2a23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.120268 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e67b8b719c1435b995c22ee0abc56a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.120391 895 tablet_service.cc:1505] Processing CreateTablet for tablet 767c004aa03d493c8d2301e1a49787e7 (DEFAULT_TABLE table=default.Table_d9237a0573e5464a995da7c7bafc08a6 [id=1ad7ef33b0d34dde8d4abbff6de61e71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.116945 750 tablet_service.cc:1505] Processing CreateTablet for tablet a44aa53c6b98468a8e36478f2fc9cdda (DEFAULT_TABLE table=default.Table_58d1c16bc0404476afd27abf2f3ce9da [id=c2dc79921cd6498f9470ed06f7712f0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.121037 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 767c004aa03d493c8d2301e1a49787e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.121354 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a44aa53c6b98468a8e36478f2fc9cdda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.124455 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 767c004aa03d493c8d2301e1a49787e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.128984 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41a4143247ed4eed98defbdfe2895372. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.132710 521 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet c977cd5aa66a4f6c8eeb37bf67f18fa7
I20260202 02:52:27.133132 740 tablet_service.cc:1505] Processing CreateTablet for tablet ce757ddc5c144d1fbaec84c9723cdbf2 (DEFAULT_TABLE table=default.Table_a9f32b021bdd40d58ea29090ef488d6b [id=a7c84b08fa1f494cba4c110c53ec91ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.133538 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce757ddc5c144d1fbaec84c9723cdbf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.134291 527 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet b9fc33e8efc04734a951f7fcf9048142
I20260202 02:52:27.134662 527 catalog_manager.cc:6724] ReplaceTablet: tablet b9fc33e8efc04734a951f7fcf9048142 deleted and replaced by tablet 8a3ebe597da64da48db881cef46cfe4e
I20260202 02:52:27.135608 901 tablet_service.cc:1552] Processing DeleteTablet for tablet c977cd5aa66a4f6c8eeb37bf67f18fa7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:27 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:27.135629 887 tablet_service.cc:1552] Processing DeleteTablet for tablet b9fc33e8efc04734a951f7fcf9048142 with delete_type TABLET_DATA_DELETED (replaced by tablet 8a3ebe597da64da48db881cef46cfe4e) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:27.135847 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 550895b4010f4088b2cc34acff50e03f with delete_type TABLET_DATA_DELETED (replaced by tablet 5416eab7a0914ecb855fbccde4894285) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:27.141494 1180 ts_tablet_manager.cc:1929] T 0186572e7bb9499a932a636ab280f544 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:27.141619 1180 log.cc:1199] T 0186572e7bb9499a932a636ab280f544 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/0186572e7bb9499a932a636ab280f544
I20260202 02:52:27.141952 1180 ts_tablet_manager.cc:1950] T 0186572e7bb9499a932a636ab280f544 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:27.142155 1182 ts_tablet_manager.cc:1929] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:27.142269 1182 log.cc:1199] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/4e79cd48fefb4ce29ef48beacfc4d26f
I20260202 02:52:27.142602 1182 ts_tablet_manager.cc:1950] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:27.142737 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 390) took 3995 ms. Trace:
I20260202 02:52:27.142823 1180 rpcz_store.cc:276] 0202 02:52:23.147013 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.147232 (+ 219us) service_pool.cc:225] Handling call
0202 02:52:27.067149 (+3919917us) ts_tablet_manager.cc:1151] Deleting tablet 0186572e7bb9499a932a636ab280f544
0202 02:52:27.067161 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:27.072714 (+ 5553us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:27.141372 (+ 68658us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:27.142704 (+ 1332us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3919771}
I20260202 02:52:27.143791 522 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_378dba7a0b4f47c586bcdd3f53712ceb" } modify_external_catalogs: true
I20260202 02:52:27.144011 522 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_378dba7a0b4f47c586bcdd3f53712ceb" } modify_external_catalogs: true
I20260202 02:52:27.144773 524 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_5a24aceace304393ae51aa2018eab99f" } new_table_name: "default.Table_3ee8fd9451364de0b9fbb7eb0aaa7990" modify_external_catalogs: true
I20260202 02:52:27.145170 750 tablet_service.cc:1505] Processing CreateTablet for tablet 9e67b8b719c1435b995c22ee0abc56a0 (DEFAULT_TABLE table=default.Table_9121c17ae28a45f2a1ce941e70260558 [id=7d630f11f4d74ef4b76057b4893b2a23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.146231 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_910b1d34ecd54cd38bff82420fe1a154"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:27.146574 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e67b8b719c1435b995c22ee0abc56a0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:27.146777 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_910b1d34ecd54cd38bff82420fe1a154 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:27.148187 527 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_f7a491432d7b42cf82732e39dcb574ce" } modify_external_catalogs: true
I20260202 02:52:27.148371 527 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_f7a491432d7b42cf82732e39dcb574ce" } modify_external_catalogs: true
I20260202 02:52:27.152153 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 0186572e7bb9499a932a636ab280f544 (table default.Table_a15c97bb32284dcfb36cdc47921b2346 [id=dc77705cb5704fd58af1eed3edd45675]) successfully deleted
I20260202 02:52:27.153070 752 tablet_service.cc:1552] Processing DeleteTablet for tablet c977cd5aa66a4f6c8eeb37bf67f18fa7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:27 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:27.153169 756 tablet_service.cc:1552] Processing DeleteTablet for tablet b9fc33e8efc04734a951f7fcf9048142 with delete_type TABLET_DATA_DELETED (replaced by tablet 8a3ebe597da64da48db881cef46cfe4e) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:27.155982 1182 tablet_replica.cc:333] T dada209d982d48f3b74a49905f19c5b6 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:27.156406 1182 raft_consensus.cc:2243] T dada209d982d48f3b74a49905f19c5b6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:27.156879 1182 raft_consensus.cc:2272] T dada209d982d48f3b74a49905f19c5b6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:27.157799 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 4e79cd48fefb4ce29ef48beacfc4d26f (table default.Table_bed5f0c218b24f529e48138901d1afa0 [id=2a625eeef55b43f19cd4bbeb61e85bca]) successfully deleted
I20260202 02:52:27.157847 1182 ts_tablet_manager.cc:1916] T dada209d982d48f3b74a49905f19c5b6 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:27.158097 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 0138ba7326674741b0729324735f1de7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0138ba7326674741b0729324735f1de7 already in progress: creating tablet
I20260202 02:52:27.160262 519 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 940e656cd34e4e53896fda6d38132b6e
I20260202 02:52:27.160840 519 catalog_manager.cc:6724] ReplaceTablet: tablet 940e656cd34e4e53896fda6d38132b6e deleted and replaced by tablet 305648e83e314b7fb7c74fe528c04adf
I20260202 02:52:27.162068 745 tablet_service.cc:1552] Processing DeleteTablet for tablet 550895b4010f4088b2cc34acff50e03f with delete_type TABLET_DATA_DELETED (replaced by tablet 5416eab7a0914ecb855fbccde4894285) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:27.162356 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 940e656cd34e4e53896fda6d38132b6e with delete_type TABLET_DATA_DELETED (replaced by tablet 305648e83e314b7fb7c74fe528c04adf) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:27.164985 745 tablet_service.cc:1552] Processing DeleteTablet for tablet 940e656cd34e4e53896fda6d38132b6e with delete_type TABLET_DATA_DELETED (replaced by tablet 305648e83e314b7fb7c74fe528c04adf) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:27.167723 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_cb2845f3c47d4f96bb58a4ed5fe5b92b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:27.168131 1180 tablet_replica.cc:333] T eaa47f3388144e8b852804c167f232ce P 1f442ef85029461091de847118c34c0d: stopping tablet replica
W20260202 02:52:27.168294 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cb2845f3c47d4f96bb58a4ed5fe5b92b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:27.168318 1180 raft_consensus.cc:2243] T eaa47f3388144e8b852804c167f232ce P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:27.168553 1180 raft_consensus.cc:2272] T eaa47f3388144e8b852804c167f232ce P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:27.169100 1180 ts_tablet_manager.cc:1916] T eaa47f3388144e8b852804c167f232ce P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:27.180675 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_15cd35fa6b4341fab4701f7afb43bd74"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:27.181479 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_15cd35fa6b4341fab4701f7afb43bd74 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:27.183239 1182 ts_tablet_manager.cc:1929] T dada209d982d48f3b74a49905f19c5b6 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:27.183374 1182 log.cc:1199] T dada209d982d48f3b74a49905f19c5b6 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/dada209d982d48f3b74a49905f19c5b6
I20260202 02:52:27.184157 747 tablet_service.cc:1505] Processing CreateTablet for tablet ec3511e5c58b4dba9e48e2b73df833a4 (DEFAULT_TABLE table=default.Table_190bbbb8a6a6454cb95f513c1c72b8fd [id=a22166fa338d40edbd0ca62ff0ce4778]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.184535 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec3511e5c58b4dba9e48e2b73df833a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.186482 1182 ts_tablet_manager.cc:1950] T dada209d982d48f3b74a49905f19c5b6 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:27.188545 523 master_service.cc:954] ReplaceTablet: received request to replace tablet dada209d982d48f3b74a49905f19c5b6 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:27.189402 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet dada209d982d48f3b74a49905f19c5b6 (table default.Table_17a421804b4e4a088917d6fcca9a0d27 [id=116247212fca4c0a95ad1bdbbac4787f]) successfully deleted
W20260202 02:52:27.189728 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet c977cd5aa66a4f6c8eeb37bf67f18fa7 because the tablet was not found. No further retry: Not found: Tablet not found: c977cd5aa66a4f6c8eeb37bf67f18fa7
W20260202 02:52:27.190053 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet b9fc33e8efc04734a951f7fcf9048142 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b9fc33e8efc04734a951f7fcf9048142 already in progress: creating tablet
I20260202 02:52:27.190531 523 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ffaecf51f55d45a684b870bda106663a" } new_table_name: "default.Table_9591ac507e6d42babe22728c2ef5dcf2" modify_external_catalogs: true
I20260202 02:52:27.192179 528 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 85f754f006d6432bb1cba87043903ac2
I20260202 02:52:27.192521 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_6bec08bdda0a4c3ea7a50826fdd6e28a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:27.193265 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6bec08bdda0a4c3ea7a50826fdd6e28a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:27.194689 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 550895b4010f4088b2cc34acff50e03f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 550895b4010f4088b2cc34acff50e03f already in progress: creating tablet
W20260202 02:52:27.195072 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 940e656cd34e4e53896fda6d38132b6e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 940e656cd34e4e53896fda6d38132b6e already in progress: creating tablet
I20260202 02:52:27.201807 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 85f754f006d6432bb1cba87043903ac2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:27 UTC) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:27.202708 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 85f754f006d6432bb1cba87043903ac2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 85f754f006d6432bb1cba87043903ac2 already in progress: creating tablet
I20260202 02:52:27.212733 740 tablet_service.cc:1552] Processing DeleteTablet for tablet 85f754f006d6432bb1cba87043903ac2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:27 UTC) from {username='slave'} at 127.0.0.1:42950
W20260202 02:52:27.215983 524 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 258e0afdba894f3cb2a3c7238e5634b7 consensus state has no leader
W20260202 02:52:27.216146 523 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4126bf5218644ca583c82190ce952267 consensus state has no leader
I20260202 02:52:27.216500 526 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_58d48519aa304bf8a91507f697a92b04" } modify_external_catalogs: true
I20260202 02:52:27.216670 526 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_58d48519aa304bf8a91507f697a92b04" } modify_external_catalogs: true
I20260202 02:52:27.217429 522 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 7b45a2e6b4b14d3fa96a7131afb16208
I20260202 02:52:27.218464 740 tablet_service.cc:1552] Processing DeleteTablet for tablet 7b45a2e6b4b14d3fa96a7131afb16208 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:27 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:27.219668 527 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 25f1d698c4ee4be7a33380184b5e2b4d
I20260202 02:52:27.220602 740 tablet_service.cc:1552] Processing DeleteTablet for tablet 25f1d698c4ee4be7a33380184b5e2b4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:27 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:27.221936 740 tablet_service.cc:1505] Processing CreateTablet for tablet 41a4143247ed4eed98defbdfe2895372 (DEFAULT_TABLE table=default.Table_d24b7b40906a4dfba59c9907d7b3cd1a [id=9ae42d61d911493caaa622f688e6c9c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.222334 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41a4143247ed4eed98defbdfe2895372. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.223815 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_ee90cc0e866a44b28f684fc4236c297a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:27.224822 895 tablet_service.cc:1552] Processing DeleteTablet for tablet 25f1d698c4ee4be7a33380184b5e2b4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:27 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:27.225559 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_52ea2b9c8a854253941ccd8d0d68a409"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:27.226068 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_52ea2b9c8a854253941ccd8d0d68a409 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:27.228437 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_a369e324e7614aeb805af657e03b040a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:27.228977 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a369e324e7614aeb805af657e03b040a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:27.230866 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_ae21476eb08546c99d1f05bafcdd5eca"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:27.231357 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ae21476eb08546c99d1f05bafcdd5eca in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:27.233114 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 25f1d698c4ee4be7a33380184b5e2b4d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 25f1d698c4ee4be7a33380184b5e2b4d already in progress: creating tablet
I20260202 02:52:27.234292 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_9bc03f5de9284dd18457bca6459ca366"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:27.234838 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9bc03f5de9284dd18457bca6459ca366 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:27.236867 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ee90cc0e866a44b28f684fc4236c297a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:27.238701 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_3d5171d76fb147efb648b0e99b6c3d9d" } new_table_name: "default.Table_8d940a39b4194caca019b3cb5ee39830" modify_external_catalogs: true
I20260202 02:52:27.242532 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 7b45a2e6b4b14d3fa96a7131afb16208 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:27 UTC) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:27.244019 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 7b45a2e6b4b14d3fa96a7131afb16208 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7b45a2e6b4b14d3fa96a7131afb16208 already in progress: creating tablet
I20260202 02:52:27.244769 526 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet 9494a3b5bbee43209cfcf360dd7e589f
I20260202 02:52:27.252914 884 tablet_service.cc:1505] Processing CreateTablet for tablet c977cd5aa66a4f6c8eeb37bf67f18fa7 (DEFAULT_TABLE table=default.Table_569ab9ec7c9c4870b31eba3616ee73d3 [id=c1d203c00f2f424a9a12f3285c72976d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.253430 740 tablet_service.cc:1505] Processing CreateTablet for tablet c977cd5aa66a4f6c8eeb37bf67f18fa7 (DEFAULT_TABLE table=default.Table_569ab9ec7c9c4870b31eba3616ee73d3 [id=c1d203c00f2f424a9a12f3285c72976d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.254299 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c977cd5aa66a4f6c8eeb37bf67f18fa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.254643 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c977cd5aa66a4f6c8eeb37bf67f18fa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.267473 520 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_882b22c2c2b0446c83239be15da80b95" } new_table_name: "default.Table_026b5574b3af4ff08f92f7ef3b3025ff" modify_external_catalogs: true
I20260202 02:52:27.269141 526 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_c9f4a79d03a04c2e8135ba9950104fc8" } new_table_name: "default.Table_9ecd3c559b81444ba6573e92172911ee" modify_external_catalogs: true
I20260202 02:52:27.271646 895 tablet_service.cc:1505] Processing CreateTablet for tablet 0b873c5661074b6b9fbdfcb97fdd6a33 (DEFAULT_TABLE table=default.Table_d46939f310bc46c3a2e99a85374c4b5e [id=f6c59566f830450381bd9a45e237ebe7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.272099 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b873c5661074b6b9fbdfcb97fdd6a33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.275578 747 tablet_service.cc:1505] Processing CreateTablet for tablet 0b873c5661074b6b9fbdfcb97fdd6a33 (DEFAULT_TABLE table=default.Table_d46939f310bc46c3a2e99a85374c4b5e [id=f6c59566f830450381bd9a45e237ebe7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.276042 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b873c5661074b6b9fbdfcb97fdd6a33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.276420 745 tablet_service.cc:1505] Processing CreateTablet for tablet dd8f36d7a82f47adae1bd3c1f8628b6e (DEFAULT_TABLE table=default.Table_cbbe5e139f7e4e058d9e05816e811c98 [id=f2ebed1d29244040b3c98e5407112bfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.276531 757 tablet_service.cc:1505] Processing CreateTablet for tablet 561bf9263f0d44678b5f221ee5f36711 (DEFAULT_TABLE table=default.Table_642b9e6132634a6cb5876a88fb02e9a1 [id=f6b9251fc0714435a869daf3127400f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.276746 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd8f36d7a82f47adae1bd3c1f8628b6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.281705 892 tablet_service.cc:1505] Processing CreateTablet for tablet b05008bde58d47f7adaaf35ac5949bdc (DEFAULT_TABLE table=default.Table_8a25e41d22cf497aad652a6171bd142c [id=6e3a23f154f24bed8d632e28aaf2cba8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.281903 885 tablet_service.cc:1505] Processing CreateTablet for tablet dd8f36d7a82f47adae1bd3c1f8628b6e (DEFAULT_TABLE table=default.Table_cbbe5e139f7e4e058d9e05816e811c98 [id=f2ebed1d29244040b3c98e5407112bfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.282135 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b05008bde58d47f7adaaf35ac5949bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.282327 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd8f36d7a82f47adae1bd3c1f8628b6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.282670 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 561bf9263f0d44678b5f221ee5f36711. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.275590 750 tablet_service.cc:1505] Processing CreateTablet for tablet b05008bde58d47f7adaaf35ac5949bdc (DEFAULT_TABLE table=default.Table_8a25e41d22cf497aad652a6171bd142c [id=6e3a23f154f24bed8d632e28aaf2cba8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.285280 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b05008bde58d47f7adaaf35ac5949bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.003270 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2b8ba9dd450948ada395207ae1c5b325" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:27.287004 1060 raft_consensus.cc:2468] T 2b8ba9dd450948ada395207ae1c5b325 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:27.065411 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79c73f4a92044bafb5b8384229cb3a4d" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:27.086591 1039 tablet_service.cc:1505] Processing CreateTablet for tablet 85f754f006d6432bb1cba87043903ac2 (DEFAULT_TABLE table=default.Table_e6661a10889e498eaf10149c7998f713 [id=75ef69f7059b4db48ca454bd21afc21c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.287766 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85f754f006d6432bb1cba87043903ac2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.289395 715 leader_election.cc:304] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
W20260202 02:52:27.289889 859 leader_election.cc:343] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:27.290002 859 leader_election.cc:304] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:27.290498 2093 raft_consensus.cc:2749] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:27.086947 1036 tablet_service.cc:1505] Processing CreateTablet for tablet 9d0212e64d934968bdbdc627c69874b1 (DEFAULT_TABLE table=default.Table_cccb5565674d4157a97fa0a565bfeea2 [id=afc25faf5ca44e2e8f50ea12003cdd9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.292085 1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d0212e64d934968bdbdc627c69874b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.281705 891 tablet_service.cc:1505] Processing CreateTablet for tablet 561bf9263f0d44678b5f221ee5f36711 (DEFAULT_TABLE table=default.Table_642b9e6132634a6cb5876a88fb02e9a1 [id=f6b9251fc0714435a869daf3127400f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.293613 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 561bf9263f0d44678b5f221ee5f36711. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.296476 1972 raft_consensus.cc:2804] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:27.296603 1972 raft_consensus.cc:493] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:27.296677 1972 raft_consensus.cc:3060] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:27.299422 1972 raft_consensus.cc:515] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.098160 1721 raft_consensus.cc:493] T 75f7824681344d8d88df9402694fff1b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:27.100747 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 957812ed9c424a62a27205179aab7f75 (DEFAULT_TABLE table=default.Table_64dd9d5476434c7baef533ca4bc498e2 [id=9e57e9c5a56648569bc62440b96e4935]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.300916 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 957812ed9c424a62a27205179aab7f75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.300887 1972 leader_election.cc:290] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:27.101814 1034 tablet_service.cc:1505] Processing CreateTablet for tablet ce757ddc5c144d1fbaec84c9723cdbf2 (DEFAULT_TABLE table=default.Table_a9f32b021bdd40d58ea29090ef488d6b [id=a7c84b08fa1f494cba4c110c53ec91ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.304643 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce757ddc5c144d1fbaec84c9723cdbf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.304937 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 381) took 3967 ms. Trace:
I20260202 02:52:27.305027 1882 rpcz_store.cc:276] 0202 02:52:23.337523 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.337606 (+ 83us) service_pool.cc:225] Handling call
0202 02:52:26.939612 (+3602006us) ts_tablet_manager.cc:1151] Deleting tablet 9884c90cf1d245b2a7eaaf00d71b1c3e
0202 02:52:26.939625 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:26.998181 (+ 58556us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:27.001102 (+ 2921us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:27.304919 (+303817us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3555175}
I20260202 02:52:27.306238 1036 tablet_service.cc:1505] Processing CreateTablet for tablet 561bf9263f0d44678b5f221ee5f36711 (DEFAULT_TABLE table=default.Table_642b9e6132634a6cb5876a88fb02e9a1 [id=f6b9251fc0714435a869daf3127400f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.306636 1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 561bf9263f0d44678b5f221ee5f36711. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.306694 1882 tablet_replica.cc:333] T 8095e0c6ff0041fc8b8c8b0778a20b00 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:27.306860 1882 raft_consensus.cc:2243] T 8095e0c6ff0041fc8b8c8b0778a20b00 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:27.307011 1882 raft_consensus.cc:2272] T 8095e0c6ff0041fc8b8c8b0778a20b00 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:27.307503 1882 ts_tablet_manager.cc:1916] T 8095e0c6ff0041fc8b8c8b0778a20b00 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:27.105516 1033 tablet_service.cc:1505] Processing CreateTablet for tablet f1fadf2cb5be4ec8980871fc96ee9730 (DEFAULT_TABLE table=default.Table_a7d94a8b6efd406c845b24e72a7d4d25 [id=16752003a5914ecda6cb7a7264f1c285]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.308593 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1fadf2cb5be4ec8980871fc96ee9730. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.313035 1174 tablet_bootstrap.cc:492] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:27.313169 1174 ts_tablet_manager.cc:1403] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.294s user 0.050s sys 0.085s
I20260202 02:52:27.107379 1027 tablet_service.cc:1505] Processing CreateTablet for tablet a44aa53c6b98468a8e36478f2fc9cdda (DEFAULT_TABLE table=default.Table_58d1c16bc0404476afd27abf2f3ce9da [id=c2dc79921cd6498f9470ed06f7712f0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.109174 1028 tablet_service.cc:1505] Processing CreateTablet for tablet ec3511e5c58b4dba9e48e2b73df833a4 (DEFAULT_TABLE table=default.Table_190bbbb8a6a6454cb95f513c1c72b8fd [id=a22166fa338d40edbd0ca62ff0ce4778]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.110337 1032 tablet_service.cc:1505] Processing CreateTablet for tablet 767c004aa03d493c8d2301e1a49787e7 (DEFAULT_TABLE table=default.Table_d9237a0573e5464a995da7c7bafc08a6 [id=1ad7ef33b0d34dde8d4abbff6de61e71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.115938 1042 tablet_service.cc:1505] Processing CreateTablet for tablet 9e67b8b719c1435b995c22ee0abc56a0 (DEFAULT_TABLE table=default.Table_9121c17ae28a45f2a1ce941e70260558 [id=7d630f11f4d74ef4b76057b4893b2a23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.143132 1860 raft_consensus.cc:493] T b428356fba0c4fb5a6033c5d39740cc5 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f442ef85029461091de847118c34c0d)
I20260202 02:52:27.151271 1031 tablet_service.cc:1552] Processing DeleteTablet for tablet 550895b4010f4088b2cc34acff50e03f with delete_type TABLET_DATA_DELETED (replaced by tablet 5416eab7a0914ecb855fbccde4894285) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:27.313731 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a44aa53c6b98468a8e36478f2fc9cdda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.151314 1040 tablet_service.cc:1552] Processing DeleteTablet for tablet b9fc33e8efc04734a951f7fcf9048142 with delete_type TABLET_DATA_DELETED (replaced by tablet 8a3ebe597da64da48db881cef46cfe4e) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:27.151445 1035 tablet_service.cc:1552] Processing DeleteTablet for tablet c977cd5aa66a4f6c8eeb37bf67f18fa7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:27 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:27.161967 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet 940e656cd34e4e53896fda6d38132b6e with delete_type TABLET_DATA_DELETED (replaced by tablet 305648e83e314b7fb7c74fe528c04adf) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:27.200037 2038 raft_consensus.cc:493] T 08d8d781bb4e4acf915f11afb4fb2e22 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:27.314007 2038 raft_consensus.cc:515] T 08d8d781bb4e4acf915f11afb4fb2e22 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.314214 1174 raft_consensus.cc:359] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.314366 1174 raft_consensus.cc:385] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:27.314410 1174 raft_consensus.cc:740] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:27.314597 1174 consensus_queue.cc:260] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.314651 2038 leader_election.cc:290] T 08d8d781bb4e4acf915f11afb4fb2e22 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:27.314802 2038 raft_consensus.cc:493] T af5aa68e47bb420eac300754d274fa73 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:27.314896 2038 raft_consensus.cc:515] T af5aa68e47bb420eac300754d274fa73 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.315330 2038 leader_election.cc:290] T af5aa68e47bb420eac300754d274fa73 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:27.317008 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2b8ba9dd450948ada395207ae1c5b325" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:27.317157 1062 raft_consensus.cc:3060] T 2b8ba9dd450948ada395207ae1c5b325 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:27.317687 1180 ts_tablet_manager.cc:1929] T eaa47f3388144e8b852804c167f232ce P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:27.317778 1180 log.cc:1199] T eaa47f3388144e8b852804c167f232ce P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/eaa47f3388144e8b852804c167f232ce
I20260202 02:52:27.318099 1180 ts_tablet_manager.cc:1950] T eaa47f3388144e8b852804c167f232ce P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:27.318784 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 391) took 4171 ms. Trace:
I20260202 02:52:27.318863 1180 rpcz_store.cc:276] 0202 02:52:23.147101 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.147431 (+ 330us) service_pool.cc:225] Handling call
0202 02:52:27.168076 (+4020645us) ts_tablet_manager.cc:1151] Deleting tablet eaa47f3388144e8b852804c167f232ce
0202 02:52:27.168087 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:27.248207 (+ 80120us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:27.317583 (+ 69376us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:27.318761 (+ 1178us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4020503}
I20260202 02:52:27.319268 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 398) took 4087 ms. Trace:
I20260202 02:52:27.319326 1180 rpcz_store.cc:276] 0202 02:52:23.231799 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.231849 (+ 50us) service_pool.cc:225] Handling call
0202 02:52:27.319146 (+4087297us) ts_tablet_manager.cc:1151] Deleting tablet 2b352d9a815e4bccafb7e2251500c5bc
0202 02:52:27.319157 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:27.319255 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4087018}
I20260202 02:52:27.319593 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 399) took 4087 ms. Trace:
I20260202 02:52:27.319648 1062 raft_consensus.cc:2468] T 2b8ba9dd450948ada395207ae1c5b325 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:27.319651 1180 rpcz_store.cc:276] 0202 02:52:23.232288 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.232353 (+ 65us) service_pool.cc:225] Handling call
0202 02:52:27.319501 (+4087148us) ts_tablet_manager.cc:1151] Deleting tablet 08d8d781bb4e4acf915f11afb4fb2e22
0202 02:52:27.319509 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:27.319578 (+ 69us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4086937}
I20260202 02:52:27.320430 1174 ts_tablet_manager.cc:1434] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260202 02:52:27.313747 1860 raft_consensus.cc:515] T b428356fba0c4fb5a6033c5d39740cc5 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.321622 1860 leader_election.cc:290] T b428356fba0c4fb5a6033c5d39740cc5 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:27.321902 715 leader_election.cc:304] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:27.322111 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 767c004aa03d493c8d2301e1a49787e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.322423 1972 raft_consensus.cc:2804] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:27.322525 1972 raft_consensus.cc:697] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:27.322744 1972 consensus_queue.cc:237] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.324719 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e67b8b719c1435b995c22ee0abc56a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.325245 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec3511e5c58b4dba9e48e2b73df833a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.202816 1173 tablet_bootstrap.cc:492] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:27.213176 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet 85f754f006d6432bb1cba87043903ac2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:27 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:27.218430 1043 tablet_service.cc:1552] Processing DeleteTablet for tablet 7b45a2e6b4b14d3fa96a7131afb16208 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:27 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:27.220635 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 25f1d698c4ee4be7a33380184b5e2b4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:27 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:27.252197 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 41a4143247ed4eed98defbdfe2895372 (DEFAULT_TABLE table=default.Table_d24b7b40906a4dfba59c9907d7b3cd1a [id=9ae42d61d911493caaa622f688e6c9c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.330227 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41a4143247ed4eed98defbdfe2895372. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.271279 1025 tablet_service.cc:1505] Processing CreateTablet for tablet c977cd5aa66a4f6c8eeb37bf67f18fa7 (DEFAULT_TABLE table=default.Table_569ab9ec7c9c4870b31eba3616ee73d3 [id=c1d203c00f2f424a9a12f3285c72976d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.336673 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c977cd5aa66a4f6c8eeb37bf67f18fa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.337455 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_f0ad9b13e025411d844149bea0d37e2f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:27.338054 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f0ad9b13e025411d844149bea0d37e2f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:27.272390 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 0b873c5661074b6b9fbdfcb97fdd6a33 (DEFAULT_TABLE table=default.Table_d46939f310bc46c3a2e99a85374c4b5e [id=f6c59566f830450381bd9a45e237ebe7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.340736 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b873c5661074b6b9fbdfcb97fdd6a33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.274041 1030 tablet_service.cc:1505] Processing CreateTablet for tablet b05008bde58d47f7adaaf35ac5949bdc (DEFAULT_TABLE table=default.Table_8a25e41d22cf497aad652a6171bd142c [id=6e3a23f154f24bed8d632e28aaf2cba8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.341460 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b05008bde58d47f7adaaf35ac5949bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.300472 1721 raft_consensus.cc:515] T 75f7824681344d8d88df9402694fff1b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.342381 1721 leader_election.cc:290] T 75f7824681344d8d88df9402694fff1b P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:27.348276 1173 ts_tablet_manager.cc:1403] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.469s user 0.005s sys 0.000s
I20260202 02:52:27.349327 1173 raft_consensus.cc:359] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:27.350538 745 tablet_service.cc:1505] Processing CreateTablet for tablet 83d198d90a204c6396b569fc1b65abef (DEFAULT_TABLE table=default.Table_11c571ac785448ca9a174c50d15ec37f [id=a1681a194792400f8e5c295d166a0778]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.351513 756 tablet_service.cc:1505] Processing CreateTablet for tablet 32fb916ae82145f195cc8b951069e5da (DEFAULT_TABLE table=default.Table_2c56436bf39647f2b5af982d04f990dc [id=e9024f48361941d2bf97d0fb9ea93cb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.352388 752 tablet_service.cc:1505] Processing CreateTablet for tablet 53c6aa6b9a774b2997d2877e39bae181 (DEFAULT_TABLE table=default.Table_a2f4e72d62b34f0abf8290dd08bc279a [id=0501a3fa46ea40219de5c9c19c9dea33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.353348 741 tablet_service.cc:1505] Processing CreateTablet for tablet b6b2e78cc66941d8a4deca12a75ff2be (DEFAULT_TABLE table=default.Table_a115f2d803494367ad12eaafeb1cdf2a [id=2536981a34aa4e08a8b740319b9cbd1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.354136 744 tablet_service.cc:1505] Processing CreateTablet for tablet 8dd62de8e8234d6c9ce380b1aadc097d (DEFAULT_TABLE table=default.Table_4fb44d5cbe71439ba073bd75711e77e0 [id=8c2d0f27f5d04a98bfad01483c783ecb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.354713 1173 raft_consensus.cc:385] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:27.354976 1173 raft_consensus.cc:740] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:27.357282 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet eaa47f3388144e8b852804c167f232ce (table default.Table_87277c5d80184738b96b773f026b32a5 [id=52687dbd0e554cfb81d88e8fcdd4dd5d]) successfully deleted
I20260202 02:52:27.357739 1173 consensus_queue.cc:260] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
W20260202 02:52:27.358260 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 2b352d9a815e4bccafb7e2251500c5bc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2b352d9a815e4bccafb7e2251500c5bc already in progress: creating tablet
W20260202 02:52:27.373939 714 leader_election.cc:343] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 2b8ba9dd450948ada395207ae1c5b325
I20260202 02:52:27.379640 885 tablet_service.cc:1505] Processing CreateTablet for tablet 32fb916ae82145f195cc8b951069e5da (DEFAULT_TABLE table=default.Table_2c56436bf39647f2b5af982d04f990dc [id=e9024f48361941d2bf97d0fb9ea93cb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.384159 884 tablet_service.cc:1505] Processing CreateTablet for tablet 53c6aa6b9a774b2997d2877e39bae181 (DEFAULT_TABLE table=default.Table_a2f4e72d62b34f0abf8290dd08bc279a [id=0501a3fa46ea40219de5c9c19c9dea33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.387012 889 tablet_service.cc:1505] Processing CreateTablet for tablet b6b2e78cc66941d8a4deca12a75ff2be (DEFAULT_TABLE table=default.Table_a115f2d803494367ad12eaafeb1cdf2a [id=2536981a34aa4e08a8b740319b9cbd1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.387336 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32fb916ae82145f195cc8b951069e5da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.387557 895 tablet_service.cc:1505] Processing CreateTablet for tablet 83d198d90a204c6396b569fc1b65abef (DEFAULT_TABLE table=default.Table_11c571ac785448ca9a174c50d15ec37f [id=a1681a194792400f8e5c295d166a0778]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.388208 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6b2e78cc66941d8a4deca12a75ff2be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.388238 917 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "af5aa68e47bb420eac300754d274fa73" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:27.388447 917 raft_consensus.cc:2468] T af5aa68e47bb420eac300754d274fa73 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:27.388509 887 tablet_service.cc:1505] Processing CreateTablet for tablet 8dd62de8e8234d6c9ce380b1aadc097d (DEFAULT_TABLE table=default.Table_4fb44d5cbe71439ba073bd75711e77e0 [id=8c2d0f27f5d04a98bfad01483c783ecb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.392474 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dd62de8e8234d6c9ce380b1aadc097d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.392669 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83d198d90a204c6396b569fc1b65abef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.395164 1175 tablet_bootstrap.cc:492] T b7880cb051c24bc998003f0effed3cf8 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:27.395319 1175 ts_tablet_manager.cc:1403] T b7880cb051c24bc998003f0effed3cf8 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.363s user 0.080s sys 0.158s
I20260202 02:52:27.380152 523 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_9591ac507e6d42babe22728c2ef5dcf2" } modify_external_catalogs: true
I20260202 02:52:27.386440 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 9884c90cf1d245b2a7eaaf00d71b1c3e (table default.Table_22dd5a0be7944811bf358ec584461157 [id=7c8c316a6dd74622a257e70c3299ea1b]) successfully deleted
I20260202 02:52:27.395826 523 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_9591ac507e6d42babe22728c2ef5dcf2" } modify_external_catalogs: true
I20260202 02:52:27.399158 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53c6aa6b9a774b2997d2877e39bae181. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.403716 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "08d8d781bb4e4acf915f11afb4fb2e22" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:27.404129 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "af5aa68e47bb420eac300754d274fa73" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:27.404436 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 403) took 4149 ms. Trace:
I20260202 02:52:27.404532 1180 rpcz_store.cc:276] 0202 02:52:23.255250 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.255295 (+ 45us) service_pool.cc:225] Handling call
0202 02:52:27.319826 (+4064531us) ts_tablet_manager.cc:1151] Deleting tablet dada209d982d48f3b74a49905f19c5b6
0202 02:52:27.403464 (+ 83638us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:27.404419 (+ 955us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4019611}
I20260202 02:52:27.406591 1174 tablet_bootstrap.cc:492] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:27.407138 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dd62de8e8234d6c9ce380b1aadc097d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.408456 1174 tablet_bootstrap.cc:654] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:27.412520 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6b2e78cc66941d8a4deca12a75ff2be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.412750 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32fb916ae82145f195cc8b951069e5da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.359946 1173 ts_tablet_manager.cc:1434] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20260202 02:52:27.416493 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53c6aa6b9a774b2997d2877e39bae181. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.379215 1033 tablet_service.cc:1505] Processing CreateTablet for tablet dd8f36d7a82f47adae1bd3c1f8628b6e (DEFAULT_TABLE table=default.Table_cbbe5e139f7e4e058d9e05816e811c98 [id=f2ebed1d29244040b3c98e5407112bfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.417569 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd8f36d7a82f47adae1bd3c1f8628b6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.418635 1173 tablet_bootstrap.cc:492] T 06b80a1c9251420cbc16fc95ae98aca9 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:27.379431 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 8dd62de8e8234d6c9ce380b1aadc097d (DEFAULT_TABLE table=default.Table_4fb44d5cbe71439ba073bd75711e77e0 [id=8c2d0f27f5d04a98bfad01483c783ecb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.420789 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dd62de8e8234d6c9ce380b1aadc097d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.421144 524 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_dfebb268090044a0bc4b4b7a830a9535" } new_table_name: "default.Table_94b42fda242b4e46a0aadf9d900c1395" modify_external_catalogs: true
W20260202 02:52:27.421411 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet dada209d982d48f3b74a49905f19c5b6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dada209d982d48f3b74a49905f19c5b6 already in progress: creating tablet
I20260202 02:52:27.421841 1174 tablet_bootstrap.cc:492] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:27.421974 1174 ts_tablet_manager.cc:1403] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.002s
W20260202 02:52:27.425757 527 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 305648e83e314b7fb7c74fe528c04adf has no consensus state
W20260202 02:52:27.426837 526 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ee04563d3b8e4cb2b989bb8d6ee9b20e consensus state has no leader
I20260202 02:52:27.429193 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83d198d90a204c6396b569fc1b65abef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.430217 525 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_950f6f5ae1ef43399de785df25c896e9" } modify_external_catalogs: true
I20260202 02:52:27.430401 525 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_950f6f5ae1ef43399de785df25c896e9" } modify_external_catalogs: true
I20260202 02:52:27.430982 523 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 4126bf5218644ca583c82190ce952267
I20260202 02:52:27.432394 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 4126bf5218644ca583c82190ce952267 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:27 UTC) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:27.433466 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 4126bf5218644ca583c82190ce952267 because the tablet was not found. No further retry: Not found: Tablet not found: 4126bf5218644ca583c82190ce952267
I20260202 02:52:27.434266 1174 raft_consensus.cc:359] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.434445 1174 raft_consensus.cc:385] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:27.434494 1174 raft_consensus.cc:740] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:27.434682 1174 consensus_queue.cc:260] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.437342 1174 ts_tablet_manager.cc:1434] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.015s user 0.001s sys 0.001s
I20260202 02:52:27.437657 1174 tablet_bootstrap.cc:492] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:27.439369 1174 tablet_bootstrap.cc:654] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:27.441212 524 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b427c7e897c64977b924e7d586420e69 consensus state has no leader
I20260202 02:52:27.441636 525 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 2c2a535de2294cc697d75561d0463642
I20260202 02:52:27.442566 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 4126bf5218644ca583c82190ce952267 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:27 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:27.443116 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 2c2a535de2294cc697d75561d0463642 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:27 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:27.428426 1173 tablet_bootstrap.cc:654] T 06b80a1c9251420cbc16fc95ae98aca9 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:27.379601 1034 tablet_service.cc:1505] Processing CreateTablet for tablet 83d198d90a204c6396b569fc1b65abef (DEFAULT_TABLE table=default.Table_11c571ac785448ca9a174c50d15ec37f [id=a1681a194792400f8e5c295d166a0778]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.445272 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83d198d90a204c6396b569fc1b65abef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.380355 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 53c6aa6b9a774b2997d2877e39bae181 (DEFAULT_TABLE table=default.Table_a2f4e72d62b34f0abf8290dd08bc279a [id=0501a3fa46ea40219de5c9c19c9dea33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.447134 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53c6aa6b9a774b2997d2877e39bae181. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.379467 1029 tablet_service.cc:1505] Processing CreateTablet for tablet b6b2e78cc66941d8a4deca12a75ff2be (DEFAULT_TABLE table=default.Table_a115f2d803494367ad12eaafeb1cdf2a [id=2536981a34aa4e08a8b740319b9cbd1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.448132 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6b2e78cc66941d8a4deca12a75ff2be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.450438 528 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_5963d74e3b6143cdb2eaca2cb99a58df" } new_table_name: "default.Table_8fe88cb0c8f649b685a2c9f75d6ca759" modify_external_catalogs: true
I20260202 02:52:27.437124 1028 tablet_service.cc:1552] Processing DeleteTablet for tablet fe53a8bdbd13417f9ce195c54a52a93c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:21 UTC) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:27.406594 998 leader_election.cc:343] T 08d8d781bb4e4acf915f11afb4fb2e22 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:27.452571 998 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 45997.7us
I20260202 02:52:27.380573 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 32fb916ae82145f195cc8b951069e5da (DEFAULT_TABLE table=default.Table_2c56436bf39647f2b5af982d04f990dc [id=e9024f48361941d2bf97d0fb9ea93cb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.453223 526 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_072c7718ffd04cdfaafcce94c169bcd8" } modify_external_catalogs: true
I20260202 02:52:27.453312 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_58420fa37ead4e3c97dd3108c58b31a1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:27.398880 999 leader_election.cc:304] T af5aa68e47bb420eac300754d274fa73 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
W20260202 02:52:27.453819 998 leader_election.cc:343] T af5aa68e47bb420eac300754d274fa73 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:27.453888 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32fb916ae82145f195cc8b951069e5da. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:27.454233 998 leader_election.cc:343] T b428356fba0c4fb5a6033c5d39740cc5 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Not found: Tablet not found: b428356fba0c4fb5a6033c5d39740cc5
W20260202 02:52:27.454387 998 leader_election.cc:343] T 75f7824681344d8d88df9402694fff1b P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Not found: Tablet not found: 75f7824681344d8d88df9402694fff1b
I20260202 02:52:27.454632 1174 tablet_bootstrap.cc:492] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:27.454774 1174 ts_tablet_manager.cc:1403] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.002s
I20260202 02:52:27.455555 1174 raft_consensus.cc:359] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.455720 1174 raft_consensus.cc:385] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:27.455772 1174 raft_consensus.cc:740] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:27.455813 2117 tablet.cc:1722] T 1f54c1dfe0a942b3bafd85522953a032 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:27.455996 1174 consensus_queue.cc:260] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:27.456291 998 consensus_peers.cc:597] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:27.456583 1174 ts_tablet_manager.cc:1434] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:27.456856 1174 tablet_bootstrap.cc:492] T af5aa68e47bb420eac300754d274fa73 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
W20260202 02:52:27.456846 999 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 58003.9us
W20260202 02:52:27.457530 999 leader_election.cc:343] T 08d8d781bb4e4acf915f11afb4fb2e22 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 08d8d781bb4e4acf915f11afb4fb2e22
I20260202 02:52:27.457768 2038 raft_consensus.cc:2804] T af5aa68e47bb420eac300754d274fa73 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:27.457849 2038 raft_consensus.cc:493] T af5aa68e47bb420eac300754d274fa73 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:27.457902 2038 raft_consensus.cc:3060] T af5aa68e47bb420eac300754d274fa73 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:27.458001 999 leader_election.cc:304] T 08d8d781bb4e4acf915f11afb4fb2e22 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:27.458210 1860 raft_consensus.cc:2749] T 08d8d781bb4e4acf915f11afb4fb2e22 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:27.458521 999 leader_election.cc:343] T b428356fba0c4fb5a6033c5d39740cc5 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: b428356fba0c4fb5a6033c5d39740cc5
I20260202 02:52:27.458606 999 leader_election.cc:304] T b428356fba0c4fb5a6033c5d39740cc5 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:27.458789 1860 raft_consensus.cc:2749] T b428356fba0c4fb5a6033c5d39740cc5 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260202 02:52:27.459112 999 leader_election.cc:343] T 75f7824681344d8d88df9402694fff1b P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 75f7824681344d8d88df9402694fff1b
I20260202 02:52:27.459298 999 leader_election.cc:304] T 75f7824681344d8d88df9402694fff1b P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:27.459481 1860 raft_consensus.cc:2749] T 75f7824681344d8d88df9402694fff1b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:27.459813 999 consensus_peers.cc:597] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ef073880898049e2be93d2cb6dc0c0bb. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:27.460685 2038 raft_consensus.cc:515] T af5aa68e47bb420eac300754d274fa73 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.461272 2038 leader_election.cc:290] T af5aa68e47bb420eac300754d274fa73 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:27.461846 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 2c2a535de2294cc697d75561d0463642 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:27 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:27.463251 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "af5aa68e47bb420eac300754d274fa73" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
W20260202 02:52:27.464604 998 leader_election.cc:343] T af5aa68e47bb420eac300754d274fa73 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:27.467206 1882 ts_tablet_manager.cc:1929] T 8095e0c6ff0041fc8b8c8b0778a20b00 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260202 02:52:27.467366 1882 log.cc:1199] T 8095e0c6ff0041fc8b8c8b0778a20b00 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/8095e0c6ff0041fc8b8c8b0778a20b00
I20260202 02:52:27.467741 1180 tablet_replica.cc:333] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:27.469240 1882 ts_tablet_manager.cc:1950] T 8095e0c6ff0041fc8b8c8b0778a20b00 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:27.482725 918 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "af5aa68e47bb420eac300754d274fa73" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:27.482921 918 raft_consensus.cc:3060] T af5aa68e47bb420eac300754d274fa73 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:27.485046 714 consensus_peers.cc:597] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 757492288d7c44fb91b25b7e6e21e01b. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:27.485680 526 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_072c7718ffd04cdfaafcce94c169bcd8" } modify_external_catalogs: true
W20260202 02:52:27.486209 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 2c2a535de2294cc697d75561d0463642 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2c2a535de2294cc697d75561d0463642 already in progress: creating tablet
I20260202 02:52:27.487237 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_12bbcb575c1d40578514ce802c1ed547"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:27.489394 1174 tablet_bootstrap.cc:654] T af5aa68e47bb420eac300754d274fa73 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:27.491981 918 raft_consensus.cc:2468] T af5aa68e47bb420eac300754d274fa73 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:27.492487 1180 raft_consensus.cc:2243] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:27.492307 527 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_a9f32b021bdd40d58ea29090ef488d6b" } modify_external_catalogs: true
I20260202 02:52:27.467199 1175 raft_consensus.cc:359] T b7880cb051c24bc998003f0effed3cf8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:27.492420 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_12bbcb575c1d40578514ce802c1ed547 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:27.493112 1175 raft_consensus.cc:385] T b7880cb051c24bc998003f0effed3cf8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:27.493172 1175 raft_consensus.cc:740] T b7880cb051c24bc998003f0effed3cf8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:27.494390 1180 pending_rounds.cc:70] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d: Trying to abort 1 pending ops.
I20260202 02:52:27.494457 1180 pending_rounds.cc:77] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923054653411328 op_type: NO_OP noop_request { }
I20260202 02:52:27.494541 1180 raft_consensus.cc:2889] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:27.494614 1180 raft_consensus.cc:2272] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:27.495137 1180 ts_tablet_manager.cc:1916] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:27.478245 2135 tablet.cc:1722] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:27.480451 1173 tablet_bootstrap.cc:492] T 06b80a1c9251420cbc16fc95ae98aca9 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:27.497426 1175 consensus_queue.cc:260] T b7880cb051c24bc998003f0effed3cf8 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.497843 1173 ts_tablet_manager.cc:1403] T 06b80a1c9251420cbc16fc95ae98aca9 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.079s user 0.005s sys 0.007s
I20260202 02:52:27.497961 1175 ts_tablet_manager.cc:1434] T b7880cb051c24bc998003f0effed3cf8 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.103s user 0.001s sys 0.001s
I20260202 02:52:27.495453 999 leader_election.cc:304] T af5aa68e47bb420eac300754d274fa73 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:27.498281 1175 tablet_bootstrap.cc:492] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:27.498891 2127 raft_consensus.cc:2804] T af5aa68e47bb420eac300754d274fa73 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:27.499130 2127 raft_consensus.cc:697] T af5aa68e47bb420eac300754d274fa73 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:27.499611 2127 consensus_queue.cc:237] T af5aa68e47bb420eac300754d274fa73 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.502123 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_e0fefcc5497b47daa332f70f03a81639"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:27.454089 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_ea358e346aa9402ba015964e2a8a906d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:27.503204 524 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_d525711c9e014b459f6cd3732bae2eb6" } modify_external_catalogs: true
W20260202 02:52:27.506008 999 consensus_peers.cc:597] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4e79cd48fefb4ce29ef48beacfc4d26f. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:27.507576 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_58420fa37ead4e3c97dd3108c58b31a1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:27.507550 1173 raft_consensus.cc:359] T 06b80a1c9251420cbc16fc95ae98aca9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:27.507810 1173 raft_consensus.cc:385] T 06b80a1c9251420cbc16fc95ae98aca9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:27.507915 1173 raft_consensus.cc:740] T 06b80a1c9251420cbc16fc95ae98aca9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:27.508262 1173 consensus_queue.cc:260] T 06b80a1c9251420cbc16fc95ae98aca9 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:27.509305 1173 ts_tablet_manager.cc:1434] T 06b80a1c9251420cbc16fc95ae98aca9 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260202 02:52:27.509763 1173 tablet_bootstrap.cc:492] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:27.510150 527 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_a9f32b021bdd40d58ea29090ef488d6b" } modify_external_catalogs: true
I20260202 02:52:27.511616 524 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_d525711c9e014b459f6cd3732bae2eb6" } modify_external_catalogs: true
I20260202 02:52:27.511713 1173 tablet_bootstrap.cc:654] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:27.513314 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 382) took 4166 ms. Trace:
I20260202 02:52:27.517171 1972 raft_consensus.cc:493] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:27.517361 1972 raft_consensus.cc:515] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:27.519079 714 leader_election.cc:343] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 68823eff1bc04108ba83487605aec0ea
I20260202 02:52:27.519362 1972 leader_election.cc:290] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:27.521531 1174 tablet_bootstrap.cc:492] T af5aa68e47bb420eac300754d274fa73 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:27.521667 1174 ts_tablet_manager.cc:1403] T af5aa68e47bb420eac300754d274fa73 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.065s user 0.008s sys 0.004s
I20260202 02:52:27.521868 1882 rpcz_store.cc:276] 0202 02:52:23.347167 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.352039 (+ 4872us) service_pool.cc:225] Handling call
0202 02:52:27.306647 (+3954608us) ts_tablet_manager.cc:1151] Deleting tablet 8095e0c6ff0041fc8b8c8b0778a20b00
0202 02:52:27.306657 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:27.330555 (+ 23898us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:27.467002 (+136447us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:27.513290 (+ 46288us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3919525}
I20260202 02:52:27.522997 1174 raft_consensus.cc:359] T af5aa68e47bb420eac300754d274fa73 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.523161 1174 raft_consensus.cc:385] T af5aa68e47bb420eac300754d274fa73 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:27.523214 1174 raft_consensus.cc:740] T af5aa68e47bb420eac300754d274fa73 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:27.523411 1174 consensus_queue.cc:260] T af5aa68e47bb420eac300754d274fa73 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.524014 1174 ts_tablet_manager.cc:1434] T af5aa68e47bb420eac300754d274fa73 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:27.524099 776 raft_consensus.cc:3060] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:27.524288 1174 tablet_bootstrap.cc:492] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:27.524881 895 tablet_service.cc:1552] Processing DeleteTablet for tablet b7880cb051c24bc998003f0effed3cf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:27.526227 1174 tablet_bootstrap.cc:654] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:27.527410 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e0fefcc5497b47daa332f70f03a81639 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:27.529335 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ea358e346aa9402ba015964e2a8a906d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:27.529836 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 8095e0c6ff0041fc8b8c8b0778a20b00 (table default.Table_4e6c3886f58b49698a00d45b957f4c9a [id=b5cc03b77f414304b29c5fb4bb7dc3ee]) successfully deleted
W20260202 02:52:27.530567 528 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d1342810762e41588ea00ea02acad3e2 consensus state has no leader
I20260202 02:52:27.530449 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_8ce461ec04534b9cb8878004a0cc3bf5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:27.530608 2127 raft_consensus.cc:493] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:27.530757 2127 raft_consensus.cc:515] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:27.530872 528 master_service.cc:954] ReplaceTablet: received request to replace tablet 1c8317adc47a4c399f2db650ed3d2e53 from {username='slave'} at 127.0.0.1:47430
W20260202 02:52:27.531028 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8ce461ec04534b9cb8878004a0cc3bf5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:27.531512 526 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 8cf6a52eea054ab688410e2dd029abf3
I20260202 02:52:27.531762 2127 leader_election.cc:290] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:27.532299 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fe53a8bdbd13417f9ce195c54a52a93c" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:27.532436 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_5c9207e4741746849a9cadbbc86ee35f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:27.532897 998 leader_election.cc:343] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:27.532975 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5c9207e4741746849a9cadbbc86ee35f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:27.534299 999 leader_election.cc:343] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: fe53a8bdbd13417f9ce195c54a52a93c
I20260202 02:52:27.534409 999 leader_election.cc:304] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:27.534989 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 8cf6a52eea054ab688410e2dd029abf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:27 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:27.535151 740 tablet_service.cc:1552] Processing DeleteTablet for tablet 8cf6a52eea054ab688410e2dd029abf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:27 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:27.535280 895 tablet_service.cc:1552] Processing DeleteTablet for tablet 8cf6a52eea054ab688410e2dd029abf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:27 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:27.536422 2127 raft_consensus.cc:2749] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:27.536203 885 tablet_service.cc:1505] Processing CreateTablet for tablet 425dab1cf74c4f379f10e202613093dd (DEFAULT_TABLE table=default.Table_3c414f5adb634d8eb3c5341307bad9fa [id=2ebcccee63d64cada179f34874510448]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.536880 1972 raft_consensus.cc:493] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:27.537051 1972 raft_consensus.cc:515] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:27.537443 740 tablet_service.cc:1505] Processing CreateTablet for tablet 425dab1cf74c4f379f10e202613093dd (DEFAULT_TABLE table=default.Table_3c414f5adb634d8eb3c5341307bad9fa [id=2ebcccee63d64cada179f34874510448]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.537616 895 tablet_service.cc:1505] Processing CreateTablet for tablet 82680e577c504a729122cd4f6b4916a6 (DEFAULT_TABLE table=default.Table_a7f3566b065c4bfbafe1349368f05d99 [id=a5157a00dae74209b78220ecd1b7c794]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.537863 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 425dab1cf74c4f379f10e202613093dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.538422 1972 leader_election.cc:290] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:27.539088 918 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "20ea39bc5d3d46f5a4ba427277d6c6a3" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:27.544109 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet 2c2a535de2294cc697d75561d0463642 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:27 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:27.544391 1034 tablet_service.cc:1552] Processing DeleteTablet for tablet 4126bf5218644ca583c82190ce952267 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:27 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:27.547063 891 tablet_service.cc:1505] Processing CreateTablet for tablet c6223cb6ad4c496896c7cf63a1b52f17 (DEFAULT_TABLE table=default.Table_fca1d0422c4d49b080e0203a229298a7 [id=b5cce6a2987d4291aa46a96d055a7e79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.547601 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 82680e577c504a729122cd4f6b4916a6 (DEFAULT_TABLE table=default.Table_a7f3566b065c4bfbafe1349368f05d99 [id=a5157a00dae74209b78220ecd1b7c794]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.548108 1034 tablet_service.cc:1505] Processing CreateTablet for tablet 425dab1cf74c4f379f10e202613093dd (DEFAULT_TABLE table=default.Table_3c414f5adb634d8eb3c5341307bad9fa [id=2ebcccee63d64cada179f34874510448]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.550035 524 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet d8917fe0328642a5a50501f7e7730649
I20260202 02:52:27.550649 527 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet ce757ddc5c144d1fbaec84c9723cdbf2
I20260202 02:52:27.552205 901 tablet_service.cc:1552] Processing DeleteTablet for tablet ce757ddc5c144d1fbaec84c9723cdbf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:27 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:27.552456 1173 tablet_bootstrap.cc:492] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:27.552928 1173 ts_tablet_manager.cc:1403] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.043s user 0.003s sys 0.000s
I20260202 02:52:27.553442 892 tablet_service.cc:1505] Processing CreateTablet for tablet d1342810762e41588ea00ea02acad3e2 (DEFAULT_TABLE table=default.Table_8fe88cb0c8f649b685a2c9f75d6ca759 [id=c851ea6e1b22496c9881d7c14d63d7e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.553854 756 tablet_service.cc:1505] Processing CreateTablet for tablet c6223cb6ad4c496896c7cf63a1b52f17 (DEFAULT_TABLE table=default.Table_fca1d0422c4d49b080e0203a229298a7 [id=b5cce6a2987d4291aa46a96d055a7e79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.554245 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6223cb6ad4c496896c7cf63a1b52f17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.554481 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 29cde4226db6428987d4a39cde804a5c (DEFAULT_TABLE table=default.Table_44086d9f3e324314af6a63c86ade13c9 [id=cc2b6a4c360a48cebe72c981b47d708a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.554862 1028 tablet_service.cc:1505] Processing CreateTablet for tablet d1342810762e41588ea00ea02acad3e2 (DEFAULT_TABLE table=default.Table_8fe88cb0c8f649b685a2c9f75d6ca759 [id=c851ea6e1b22496c9881d7c14d63d7e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.555262 901 tablet_service.cc:1505] Processing CreateTablet for tablet 49dfbf0149874e1d904b78913daf2c84 (DEFAULT_TABLE table=default.Table_61d946ae91ff440c9960b0136900fa6e [id=4e6e4c15209543bbb24bd90ce2cc75dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.555328 745 tablet_service.cc:1505] Processing CreateTablet for tablet 49dfbf0149874e1d904b78913daf2c84 (DEFAULT_TABLE table=default.Table_61d946ae91ff440c9960b0136900fa6e [id=4e6e4c15209543bbb24bd90ce2cc75dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.555703 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49dfbf0149874e1d904b78913daf2c84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.556458 752 tablet_service.cc:1552] Processing DeleteTablet for tablet ce757ddc5c144d1fbaec84c9723cdbf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:27 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:27.556838 744 tablet_service.cc:1505] Processing CreateTablet for tablet d1342810762e41588ea00ea02acad3e2 (DEFAULT_TABLE table=default.Table_8fe88cb0c8f649b685a2c9f75d6ca759 [id=c851ea6e1b22496c9881d7c14d63d7e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.557144 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1342810762e41588ea00ea02acad3e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.557976 747 tablet_service.cc:1505] Processing CreateTablet for tablet d594295561ff4b01bf13451533a5743c (DEFAULT_TABLE table=default.Table_df0a1db6c8b048938a0bbd425a0a65dd [id=64eea8b3e2d34698ad2dbe65d87d29f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.558307 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d594295561ff4b01bf13451533a5743c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.560722 752 tablet_service.cc:1505] Processing CreateTablet for tablet 8aee6931b74547ad9e0bfe0671eaaf1f (DEFAULT_TABLE table=default.Table_0205a93a018940bbbd0bcf218e404fb6 [id=4fc69b4f5938402c87268b1f39290894]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.560725 1042 tablet_service.cc:1505] Processing CreateTablet for tablet 8aee6931b74547ad9e0bfe0671eaaf1f (DEFAULT_TABLE table=default.Table_0205a93a018940bbbd0bcf218e404fb6 [id=4fc69b4f5938402c87268b1f39290894]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.560938 1043 tablet_service.cc:1505] Processing CreateTablet for tablet 4126bf5218644ca583c82190ce952267 (DEFAULT_TABLE table=default.Table_9591ac507e6d42babe22728c2ef5dcf2 [id=6eed13054048413dad1596d33417c353]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.561084 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8aee6931b74547ad9e0bfe0671eaaf1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.561299 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 5b02daa510384c0dab6a9627bbf425a0 (DEFAULT_TABLE table=default.Table_d816f250a1204733bad1ba0967a17297 [id=191815ba67414a75956cc0b850fa75e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.561628 1038 tablet_service.cc:1505] Processing CreateTablet for tablet cafa1d1231c54a608422695409720112 (DEFAULT_TABLE table=default.Table_731531d5a4d548d8b43461d6b87310dc [id=1f8f98bfe8e5403c886d2f290a626dd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.562155 1041 tablet_service.cc:1505] Processing CreateTablet for tablet 258e0afdba894f3cb2a3c7238e5634b7 (DEFAULT_TABLE table=default.Table_3ee8fd9451364de0b9fbb7eb0aaa7990 [id=b054694513ee4465893da8ff2999ab96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.562479 1035 tablet_service.cc:1505] Processing CreateTablet for tablet 77f60925b6ca496d9b43502ed77bb55a (DEFAULT_TABLE table=default.Table_82c14c20a7bf41b487efaff52ebc15a9 [id=7fbb33c105f149ef98505dcf622930c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.563509 1030 tablet_service.cc:1505] Processing CreateTablet for tablet d594295561ff4b01bf13451533a5743c (DEFAULT_TABLE table=default.Table_df0a1db6c8b048938a0bbd425a0a65dd [id=64eea8b3e2d34698ad2dbe65d87d29f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.563890 1040 tablet_service.cc:1505] Processing CreateTablet for tablet 78a9a483b95c4bb5a5ab69abc899837b (DEFAULT_TABLE table=default.Table_a115529cbd054b83bc0b16a843f8017d [id=c0870f0e1a9d48d9a58b46b73f7f397a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.564250 1032 tablet_service.cc:1505] Processing CreateTablet for tablet 118753e8cb344b47a35a745aa8fed265 (DEFAULT_TABLE table=default.Table_7913f8fb6eac487a971545a11cf257a9 [id=b40664207c364307ba6f5ade490a21d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.564801 1031 tablet_service.cc:1505] Processing CreateTablet for tablet 34a01a4ce1be4ad08d911763de158073 (DEFAULT_TABLE table=default.Table_6e6ab095d9a6497d8a89bcc865b6f42f [id=7d8f7ac4c2ee4b72b2ff25aff060010f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.564990 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 49dfbf0149874e1d904b78913daf2c84 (DEFAULT_TABLE table=default.Table_61d946ae91ff440c9960b0136900fa6e [id=4e6e4c15209543bbb24bd90ce2cc75dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.565229 1029 tablet_service.cc:1505] Processing CreateTablet for tablet c6223cb6ad4c496896c7cf63a1b52f17 (DEFAULT_TABLE table=default.Table_fca1d0422c4d49b080e0203a229298a7 [id=b5cce6a2987d4291aa46a96d055a7e79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.565407 1033 tablet_service.cc:1552] Processing DeleteTablet for tablet ce757ddc5c144d1fbaec84c9723cdbf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:27 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:27.565450 741 tablet_service.cc:1505] Processing CreateTablet for tablet 29cde4226db6428987d4a39cde804a5c (DEFAULT_TABLE table=default.Table_44086d9f3e324314af6a63c86ade13c9 [id=cc2b6a4c360a48cebe72c981b47d708a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.565843 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29cde4226db6428987d4a39cde804a5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.565925 1033 tablet_service.cc:1505] Processing CreateTablet for tablet 9d02de5549ba4436b4eea6750a3fef09 (DEFAULT_TABLE table=default.Table_5991ee9682fa4e3396ece0dbf283e777 [id=6bb9e68b3baa46f882acd623187e5ea3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.568287 750 tablet_service.cc:1505] Processing CreateTablet for tablet 5b02daa510384c0dab6a9627bbf425a0 (DEFAULT_TABLE table=default.Table_d816f250a1204733bad1ba0967a17297 [id=191815ba67414a75956cc0b850fa75e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.569064 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b02daa510384c0dab6a9627bbf425a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.568990 740 tablet_service.cc:1505] Processing CreateTablet for tablet 82680e577c504a729122cd4f6b4916a6 (DEFAULT_TABLE table=default.Table_a7f3566b065c4bfbafe1349368f05d99 [id=a5157a00dae74209b78220ecd1b7c794]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.569361 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82680e577c504a729122cd4f6b4916a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.572600 757 tablet_service.cc:1505] Processing CreateTablet for tablet 4126bf5218644ca583c82190ce952267 (DEFAULT_TABLE table=default.Table_9591ac507e6d42babe22728c2ef5dcf2 [id=6eed13054048413dad1596d33417c353]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.572827 758 tablet_service.cc:1505] Processing CreateTablet for tablet cafa1d1231c54a608422695409720112 (DEFAULT_TABLE table=default.Table_731531d5a4d548d8b43461d6b87310dc [id=1f8f98bfe8e5403c886d2f290a626dd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.573050 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4126bf5218644ca583c82190ce952267. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.573235 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cafa1d1231c54a608422695409720112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.577750 746 tablet_service.cc:1505] Processing CreateTablet for tablet 118753e8cb344b47a35a745aa8fed265 (DEFAULT_TABLE table=default.Table_7913f8fb6eac487a971545a11cf257a9 [id=b40664207c364307ba6f5ade490a21d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.578222 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 118753e8cb344b47a35a745aa8fed265. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.582429 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_a4f1629e749a48559f702106aa3833b3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:27.584139 759 tablet_service.cc:1505] Processing CreateTablet for tablet 258e0afdba894f3cb2a3c7238e5634b7 (DEFAULT_TABLE table=default.Table_3ee8fd9451364de0b9fbb7eb0aaa7990 [id=b054694513ee4465893da8ff2999ab96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.584605 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 258e0afdba894f3cb2a3c7238e5634b7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:27.589267 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a4f1629e749a48559f702106aa3833b3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:27.592083 773 raft_consensus.cc:3060] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:27.592397 888 tablet_service.cc:1505] Processing CreateTablet for tablet 8aee6931b74547ad9e0bfe0671eaaf1f (DEFAULT_TABLE table=default.Table_0205a93a018940bbbd0bcf218e404fb6 [id=4fc69b4f5938402c87268b1f39290894]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.593304 889 tablet_service.cc:1505] Processing CreateTablet for tablet 29cde4226db6428987d4a39cde804a5c (DEFAULT_TABLE table=default.Table_44086d9f3e324314af6a63c86ade13c9 [id=cc2b6a4c360a48cebe72c981b47d708a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.595299 1182 tablet_replica.cc:333] T b7880cb051c24bc998003f0effed3cf8 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:27.595506 1182 raft_consensus.cc:2243] T b7880cb051c24bc998003f0effed3cf8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:27.595662 1182 raft_consensus.cc:2272] T b7880cb051c24bc998003f0effed3cf8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:27.596225 890 tablet_service.cc:1505] Processing CreateTablet for tablet 5b02daa510384c0dab6a9627bbf425a0 (DEFAULT_TABLE table=default.Table_d816f250a1204733bad1ba0967a17297 [id=191815ba67414a75956cc0b850fa75e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.597561 886 tablet_service.cc:1505] Processing CreateTablet for tablet cafa1d1231c54a608422695409720112 (DEFAULT_TABLE table=default.Table_731531d5a4d548d8b43461d6b87310dc [id=1f8f98bfe8e5403c886d2f290a626dd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.597894 747 tablet_service.cc:1505] Processing CreateTablet for tablet 77f60925b6ca496d9b43502ed77bb55a (DEFAULT_TABLE table=default.Table_82c14c20a7bf41b487efaff52ebc15a9 [id=7fbb33c105f149ef98505dcf622930c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.598230 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 425dab1cf74c4f379f10e202613093dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.598271 898 tablet_service.cc:1505] Processing CreateTablet for tablet 77f60925b6ca496d9b43502ed77bb55a (DEFAULT_TABLE table=default.Table_82c14c20a7bf41b487efaff52ebc15a9 [id=7fbb33c105f149ef98505dcf622930c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.598369 1182 ts_tablet_manager.cc:1916] T b7880cb051c24bc998003f0effed3cf8 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:27.598433 893 tablet_service.cc:1505] Processing CreateTablet for tablet 4126bf5218644ca583c82190ce952267 (DEFAULT_TABLE table=default.Table_9591ac507e6d42babe22728c2ef5dcf2 [id=6eed13054048413dad1596d33417c353]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.598631 896 tablet_service.cc:1505] Processing CreateTablet for tablet 9d02de5549ba4436b4eea6750a3fef09 (DEFAULT_TABLE table=default.Table_5991ee9682fa4e3396ece0dbf283e777 [id=6bb9e68b3baa46f882acd623187e5ea3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.598999 902 tablet_service.cc:1505] Processing CreateTablet for tablet 258e0afdba894f3cb2a3c7238e5634b7 (DEFAULT_TABLE table=default.Table_3ee8fd9451364de0b9fbb7eb0aaa7990 [id=b054694513ee4465893da8ff2999ab96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.599280 900 tablet_service.cc:1505] Processing CreateTablet for tablet 78a9a483b95c4bb5a5ab69abc899837b (DEFAULT_TABLE table=default.Table_a115529cbd054b83bc0b16a843f8017d [id=c0870f0e1a9d48d9a58b46b73f7f397a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.598290 899 tablet_service.cc:1505] Processing CreateTablet for tablet d594295561ff4b01bf13451533a5743c (DEFAULT_TABLE table=default.Table_df0a1db6c8b048938a0bbd425a0a65dd [id=64eea8b3e2d34698ad2dbe65d87d29f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.599731 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77f60925b6ca496d9b43502ed77bb55a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.601222 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1342810762e41588ea00ea02acad3e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.601459 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49dfbf0149874e1d904b78913daf2c84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.602567 894 tablet_service.cc:1505] Processing CreateTablet for tablet 118753e8cb344b47a35a745aa8fed265 (DEFAULT_TABLE table=default.Table_7913f8fb6eac487a971545a11cf257a9 [id=b40664207c364307ba6f5ade490a21d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.602869 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 118753e8cb344b47a35a745aa8fed265. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.606920 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8aee6931b74547ad9e0bfe0671eaaf1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.612452 1182 ts_tablet_manager.cc:1929] T b7880cb051c24bc998003f0effed3cf8 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:27.615542 1182 log.cc:1199] T b7880cb051c24bc998003f0effed3cf8 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/b7880cb051c24bc998003f0effed3cf8
I20260202 02:52:27.612710 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29cde4226db6428987d4a39cde804a5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.614554 1175 tablet_bootstrap.cc:654] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:27.617056 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d594295561ff4b01bf13451533a5743c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.617928 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4126bf5218644ca583c82190ce952267. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.618238 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6223cb6ad4c496896c7cf63a1b52f17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.618410 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cafa1d1231c54a608422695409720112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.619151 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78a9a483b95c4bb5a5ab69abc899837b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.621097 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82680e577c504a729122cd4f6b4916a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.621063 897 tablet_service.cc:1505] Processing CreateTablet for tablet 34a01a4ce1be4ad08d911763de158073 (DEFAULT_TABLE table=default.Table_6e6ab095d9a6497d8a89bcc865b6f42f [id=7d8f7ac4c2ee4b72b2ff25aff060010f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.621397 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34a01a4ce1be4ad08d911763de158073. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.622231 2093 raft_consensus.cc:493] T f94f25f4059a43b8babedb6b45a08a19 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:27.622342 2093 raft_consensus.cc:515] T f94f25f4059a43b8babedb6b45a08a19 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.622874 2093 leader_election.cc:290] T f94f25f4059a43b8babedb6b45a08a19 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:27.624539 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 258e0afdba894f3cb2a3c7238e5634b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.628451 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d02de5549ba4436b4eea6750a3fef09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.629297 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b02daa510384c0dab6a9627bbf425a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.631186 525 master_service.cc:954] ReplaceTablet: received request to replace tablet dc4c3f52f4f44a84b0d0e3c9a4a49f09 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:27.635520 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77f60925b6ca496d9b43502ed77bb55a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.620868 750 tablet_service.cc:1505] Processing CreateTablet for tablet 78a9a483b95c4bb5a5ab69abc899837b (DEFAULT_TABLE table=default.Table_a115529cbd054b83bc0b16a843f8017d [id=c0870f0e1a9d48d9a58b46b73f7f397a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.626291 746 tablet_service.cc:1505] Processing CreateTablet for tablet 34a01a4ce1be4ad08d911763de158073 (DEFAULT_TABLE table=default.Table_6e6ab095d9a6497d8a89bcc865b6f42f [id=7d8f7ac4c2ee4b72b2ff25aff060010f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.640604 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34a01a4ce1be4ad08d911763de158073. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.641377 756 tablet_service.cc:1505] Processing CreateTablet for tablet 9d02de5549ba4436b4eea6750a3fef09 (DEFAULT_TABLE table=default.Table_5991ee9682fa4e3396ece0dbf283e777 [id=6bb9e68b3baa46f882acd623187e5ea3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.641893 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d02de5549ba4436b4eea6750a3fef09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.644586 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78a9a483b95c4bb5a5ab69abc899837b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.645994 525 master_service.cc:954] ReplaceTablet: received request to replace tablet 2c31b45dc8004eae993e972e9faa3ff8 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:27.652416 1182 ts_tablet_manager.cc:1950] T b7880cb051c24bc998003f0effed3cf8 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:27.657797 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f94f25f4059a43b8babedb6b45a08a19" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:27.659291 857 leader_election.cc:343] T f94f25f4059a43b8babedb6b45a08a19 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:27.659336 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet b7880cb051c24bc998003f0effed3cf8 (table default.Table_a0f1f2d30d064244bec97181c11c3a06 [id=1787be853d4a40cdada2c2735cad1a01]) successfully deleted
I20260202 02:52:27.659749 2127 consensus_queue.cc:1048] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:27.660282 1174 tablet_bootstrap.cc:492] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:27.660434 1174 ts_tablet_manager.cc:1403] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.136s user 0.025s sys 0.030s
I20260202 02:52:27.661186 1174 raft_consensus.cc:359] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.661360 1174 raft_consensus.cc:385] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:27.661409 1174 raft_consensus.cc:740] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:27.661588 1174 consensus_queue.cc:260] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.662158 1174 ts_tablet_manager.cc:1434] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260202 02:52:27.662418 1174 tablet_bootstrap.cc:492] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:27.663723 1174 tablet_bootstrap.cc:654] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:27.664371 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "68823eff1bc04108ba83487605aec0ea" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:27.664568 1060 raft_consensus.cc:2468] T 68823eff1bc04108ba83487605aec0ea P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:27.667973 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f94f25f4059a43b8babedb6b45a08a19" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:27.669520 519 catalog_manager.cc:5654] T af5aa68e47bb420eac300754d274fa73 P b1f05820b6034228bdad884cc348716d reported cstate change: term changed from 0 to 1, leader changed from <none> to b1f05820b6034228bdad884cc348716d (127.0.104.131). New cstate: current_term: 1 leader_uuid: "b1f05820b6034228bdad884cc348716d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } }
I20260202 02:52:27.669973 519 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet af5aa68e47bb420eac300754d274fa73 (table default.Table_12c352712c374a45b7de7a0e634a1327 [id=feb8b9f926434cec8b119843413f8c9b]). Expected version 1 got 0
I20260202 02:52:27.673493 1036 tablet_service.cc:1505] Processing CreateTablet for tablet fc3bfd542b8e41d98586645358061261 (DEFAULT_TABLE table=default.Table_ea358e346aa9402ba015964e2a8a906d [id=0bbfe96f93634565980d72b1b058c722]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.673493 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 06016cee61b64c59b2f01e8f01e445cc (DEFAULT_TABLE table=default.Table_8ce461ec04534b9cb8878004a0cc3bf5 [id=2a20bf11ada9426cb8c580e2fa143d71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.675851 1174 tablet_bootstrap.cc:492] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:27.676028 1174 ts_tablet_manager.cc:1403] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20260202 02:52:27.676973 1174 raft_consensus.cc:359] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.677158 1174 raft_consensus.cc:385] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:27.677199 1174 raft_consensus.cc:740] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:27.677174 1039 tablet_service.cc:1505] Processing CreateTablet for tablet cb3d328d8c434f0e8bf0c639c67dc794 (DEFAULT_TABLE table=default.Table_12bbcb575c1d40578514ce802c1ed547 [id=ea20d3d1ff8845b3874eaecf094fd2ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.677371 1174 consensus_queue.cc:260] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.677582 2042 raft_consensus.cc:493] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:27.677677 2042 raft_consensus.cc:515] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.678234 2042 leader_election.cc:290] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:27.678757 1174 ts_tablet_manager.cc:1434] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260202 02:52:27.679124 1174 tablet_bootstrap.cc:492] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:27.680923 715 leader_election.cc:304] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:27.681123 1174 tablet_bootstrap.cc:654] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:27.681272 2042 raft_consensus.cc:2804] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:27.681360 2042 raft_consensus.cc:493] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:27.681427 2042 raft_consensus.cc:3060] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:27.687296 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8774c6bc0fe9414ea2f17ed859d9392a" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:27.687292 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_7d832c4a3a854a09909c552c8f6229a8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:27.687526 1059 raft_consensus.cc:2393] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f442ef85029461091de847118c34c0d in current term 1: Already voted for candidate b1f05820b6034228bdad884cc348716d in this term.
W20260202 02:52:27.687886 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7d832c4a3a854a09909c552c8f6229a8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:27.684278 2042 raft_consensus.cc:515] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.689188 2042 leader_election.cc:290] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:27.690057 520 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_b13428495dde400e83de7d1032559c35" } new_table_name: "default.Table_dd517ae94bf54267bc3a03dee4896708" modify_external_catalogs: true
I20260202 02:52:27.690649 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "68823eff1bc04108ba83487605aec0ea" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:27.690832 1059 raft_consensus.cc:3060] T 68823eff1bc04108ba83487605aec0ea P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:27.692239 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_c826a60439364f9eb3cbcd4a5cd99527"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:27.692970 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c826a60439364f9eb3cbcd4a5cd99527 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:27.693504 1059 raft_consensus.cc:2468] T 68823eff1bc04108ba83487605aec0ea P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:27.694012 2042 consensus_queue.cc:1048] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:27.694543 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_17a421804b4e4a088917d6fcca9a0d27" } new_table_name: "default.Table_5485f3195b4f4bd7b9220ff7b1459d5a" modify_external_catalogs: true
I20260202 02:52:27.694716 715 leader_election.cc:304] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:27.695914 2042 raft_consensus.cc:2804] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:27.696161 2042 raft_consensus.cc:697] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:27.696856 2042 consensus_queue.cc:237] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.698520 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "20ea39bc5d3d46f5a4ba427277d6c6a3" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:27.698832 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78a9a483b95c4bb5a5ab69abc899837b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.699100 715 leader_election.cc:304] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:27.699353 2047 raft_consensus.cc:2749] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:27.700917 2047 raft_consensus.cc:493] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:27.701159 2047 raft_consensus.cc:515] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.701826 2047 leader_election.cc:290] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:27.702510 740 tablet_service.cc:1505] Processing CreateTablet for tablet 06016cee61b64c59b2f01e8f01e445cc (DEFAULT_TABLE table=default.Table_8ce461ec04534b9cb8878004a0cc3bf5 [id=2a20bf11ada9426cb8c580e2fa143d71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.702975 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06016cee61b64c59b2f01e8f01e445cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.704636 524 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_5991ee9682fa4e3396ece0dbf283e777" } new_table_name: "default.Table_8c9a49f3db8f4576b5e1cb0b08df190f" modify_external_catalogs: true
I20260202 02:52:27.705018 526 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_82c14c20a7bf41b487efaff52ebc15a9" } new_table_name: "default.Table_684386396f2a4be68080349404089bf8" modify_external_catalogs: true
I20260202 02:52:27.706156 523 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_3c414f5adb634d8eb3c5341307bad9fa" } modify_external_catalogs: true
I20260202 02:52:27.706369 523 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_3c414f5adb634d8eb3c5341307bad9fa" } modify_external_catalogs: true
I20260202 02:52:27.709561 1040 tablet_service.cc:1505] Processing CreateTablet for tablet eb410274d21e4fe097b0c312e4aa4446 (DEFAULT_TABLE table=default.Table_ae21476eb08546c99d1f05bafcdd5eca [id=47e7be18cd974a98a59d55a31c846362]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.710129 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb410274d21e4fe097b0c312e4aa4446. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.710598 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_56abcf537bc74585bad14f2d06769ccf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:27.710892 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "042d1ebf357b4abe8814a407891ce265" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:27.711082 1062 raft_consensus.cc:2468] T 042d1ebf357b4abe8814a407891ce265 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
W20260202 02:52:27.711229 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_56abcf537bc74585bad14f2d06769ccf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:27.711675 715 leader_election.cc:304] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:27.711982 2047 raft_consensus.cc:2804] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:27.712055 2047 raft_consensus.cc:493] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:27.712112 2047 raft_consensus.cc:3060] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:27.714834 2047 raft_consensus.cc:515] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.715448 2047 leader_election.cc:290] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:27.716161 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "042d1ebf357b4abe8814a407891ce265" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:27.716424 1062 raft_consensus.cc:3060] T 042d1ebf357b4abe8814a407891ce265 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:27.718811 1062 raft_consensus.cc:2468] T 042d1ebf357b4abe8814a407891ce265 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:27.721215 745 tablet_service.cc:1505] Processing CreateTablet for tablet cb3d328d8c434f0e8bf0c639c67dc794 (DEFAULT_TABLE table=default.Table_12bbcb575c1d40578514ce802c1ed547 [id=ea20d3d1ff8845b3874eaecf094fd2ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.721550 744 tablet_service.cc:1505] Processing CreateTablet for tablet 6167abf7b93b48f090ff56a57a49fe41 (DEFAULT_TABLE table=default.Table_15cd35fa6b4341fab4701f7afb43bd74 [id=38c146d8a3954e6cb992c939513b98ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.721653 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb3d328d8c434f0e8bf0c639c67dc794. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.721292 756 tablet_service.cc:1505] Processing CreateTablet for tablet bbc789419cf54a00a1653c3801fe0bfd (DEFAULT_TABLE table=default.Table_6bec08bdda0a4c3ea7a50826fdd6e28a [id=eb75ceec451540ada1c0109dad3f1907]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.721565 759 tablet_service.cc:1505] Processing CreateTablet for tablet d3f6a30e08244a07a1f81b5e98bc6965 (DEFAULT_TABLE table=default.Table_cb2845f3c47d4f96bb58a4ed5fe5b92b [id=4d8b49dd050f4a28856103bb02f4ab42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.721966 715 leader_election.cc:304] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:27.722546 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3f6a30e08244a07a1f81b5e98bc6965. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.722824 2042 raft_consensus.cc:2804] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:27.723030 2042 raft_consensus.cc:697] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:27.723371 2042 consensus_queue.cc:237] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.724025 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbc789419cf54a00a1653c3801fe0bfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.726166 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6167abf7b93b48f090ff56a57a49fe41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.728856 758 tablet_service.cc:1505] Processing CreateTablet for tablet f381e9c3fe3d4dbba6895cec616cb6f6 (DEFAULT_TABLE table=default.Table_e87d06c7523440f7a739370378832d12 [id=e2c20a10802c4054862d6841a1865510]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.729331 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f381e9c3fe3d4dbba6895cec616cb6f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.728856 740 tablet_service.cc:1505] Processing CreateTablet for tablet fc3bfd542b8e41d98586645358061261 (DEFAULT_TABLE table=default.Table_ea358e346aa9402ba015964e2a8a906d [id=0bbfe96f93634565980d72b1b058c722]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.730997 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc3bfd542b8e41d98586645358061261. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.731105 757 tablet_service.cc:1505] Processing CreateTablet for tablet 00f2d3bafb2d472aaffc0a7ba926c825 (DEFAULT_TABLE table=default.Table_1f6c33e9af15400180eb052e679585e7 [id=28cc2ced6205455c98cebe9127e9257c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.731467 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00f2d3bafb2d472aaffc0a7ba926c825. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.732852 751 tablet_service.cc:1505] Processing CreateTablet for tablet c66f415bf66b48a9af192301b982f8dd (DEFAULT_TABLE table=default.Table_005b6c9953134fc3a42de2403722bdfe [id=dd31c07ae69b45dd969135aaccf2b999]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.733327 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c66f415bf66b48a9af192301b982f8dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.734763 1174 tablet_bootstrap.cc:492] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:27.734900 755 tablet_service.cc:1505] Processing CreateTablet for tablet 404125def15a4f1fb1529447785fc93e (DEFAULT_TABLE table=default.Table_843cd26c176e43c588824e3225124494 [id=3256d26f41564428b41ecf40e820ad04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.735700 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 404125def15a4f1fb1529447785fc93e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.737345 1040 tablet_service.cc:1505] Processing CreateTablet for tablet bbc789419cf54a00a1653c3801fe0bfd (DEFAULT_TABLE table=default.Table_6bec08bdda0a4c3ea7a50826fdd6e28a [id=eb75ceec451540ada1c0109dad3f1907]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.738417 888 tablet_service.cc:1505] Processing CreateTablet for tablet eb410274d21e4fe097b0c312e4aa4446 (DEFAULT_TABLE table=default.Table_ae21476eb08546c99d1f05bafcdd5eca [id=47e7be18cd974a98a59d55a31c846362]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.734894 1174 ts_tablet_manager.cc:1403] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.056s user 0.016s sys 0.014s
I20260202 02:52:27.739279 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb410274d21e4fe097b0c312e4aa4446. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.740952 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34a01a4ce1be4ad08d911763de158073. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.741108 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d02de5549ba4436b4eea6750a3fef09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.743019 899 tablet_service.cc:1505] Processing CreateTablet for tablet cb3d328d8c434f0e8bf0c639c67dc794 (DEFAULT_TABLE table=default.Table_12bbcb575c1d40578514ce802c1ed547 [id=ea20d3d1ff8845b3874eaecf094fd2ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.743453 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb3d328d8c434f0e8bf0c639c67dc794. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.744568 884 tablet_service.cc:1505] Processing CreateTablet for tablet d3f6a30e08244a07a1f81b5e98bc6965 (DEFAULT_TABLE table=default.Table_cb2845f3c47d4f96bb58a4ed5fe5b92b [id=4d8b49dd050f4a28856103bb02f4ab42]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:27.744895 859 leader_election.cc:343] T f94f25f4059a43b8babedb6b45a08a19 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:27.745338 887 tablet_service.cc:1505] Processing CreateTablet for tablet f381e9c3fe3d4dbba6895cec616cb6f6 (DEFAULT_TABLE table=default.Table_e87d06c7523440f7a739370378832d12 [id=e2c20a10802c4054862d6841a1865510]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.747599 1031 tablet_service.cc:1505] Processing CreateTablet for tablet 6167abf7b93b48f090ff56a57a49fe41 (DEFAULT_TABLE table=default.Table_15cd35fa6b4341fab4701f7afb43bd74 [id=38c146d8a3954e6cb992c939513b98ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.748032 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6167abf7b93b48f090ff56a57a49fe41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.748174 2127 raft_consensus.cc:493] T 06b80a1c9251420cbc16fc95ae98aca9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:27.748286 2127 raft_consensus.cc:515] T 06b80a1c9251420cbc16fc95ae98aca9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:27.748971 2127 leader_election.cc:290] T 06b80a1c9251420cbc16fc95ae98aca9 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:27.749552 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb3d328d8c434f0e8bf0c639c67dc794. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.752431 1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc3bfd542b8e41d98586645358061261. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.756773 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06016cee61b64c59b2f01e8f01e445cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.756654 900 tablet_service.cc:1505] Processing CreateTablet for tablet fc3bfd542b8e41d98586645358061261 (DEFAULT_TABLE table=default.Table_ea358e346aa9402ba015964e2a8a906d [id=0bbfe96f93634565980d72b1b058c722]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.757133 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc3bfd542b8e41d98586645358061261. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.757540 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f381e9c3fe3d4dbba6895cec616cb6f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.761049 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49dfbf0149874e1d904b78913daf2c84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.764459 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6223cb6ad4c496896c7cf63a1b52f17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.765139 1174 raft_consensus.cc:359] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.765313 1174 raft_consensus.cc:385] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:27.765383 1174 raft_consensus.cc:740] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:27.765245 741 tablet_service.cc:1505] Processing CreateTablet for tablet eb410274d21e4fe097b0c312e4aa4446 (DEFAULT_TABLE table=default.Table_ae21476eb08546c99d1f05bafcdd5eca [id=47e7be18cd974a98a59d55a31c846362]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.765851 1174 consensus_queue.cc:260] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.766103 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb410274d21e4fe097b0c312e4aa4446. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.767306 902 tablet_service.cc:1505] Processing CreateTablet for tablet e7df9a78735547cbbaa006c604b95fec (DEFAULT_TABLE table=default.Table_9465647c983f4c85b7b2b5454fcc80e7 [id=24b8f6a5c7b9466e9ff1dbb43988ac3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.767722 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7df9a78735547cbbaa006c604b95fec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.770071 885 tablet_service.cc:1505] Processing CreateTablet for tablet 6167abf7b93b48f090ff56a57a49fe41 (DEFAULT_TABLE table=default.Table_15cd35fa6b4341fab4701f7afb43bd74 [id=38c146d8a3954e6cb992c939513b98ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.770347 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbc789419cf54a00a1653c3801fe0bfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.770505 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6167abf7b93b48f090ff56a57a49fe41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.771294 895 tablet_service.cc:1505] Processing CreateTablet for tablet 06016cee61b64c59b2f01e8f01e445cc (DEFAULT_TABLE table=default.Table_8ce461ec04534b9cb8878004a0cc3bf5 [id=2a20bf11ada9426cb8c580e2fa143d71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.771770 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3f6a30e08244a07a1f81b5e98bc6965. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.745043 859 leader_election.cc:304] T f94f25f4059a43b8babedb6b45a08a19 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:27.773386 2093 raft_consensus.cc:2749] T f94f25f4059a43b8babedb6b45a08a19 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:27.775535 887 tablet_service.cc:1505] Processing CreateTablet for tablet c66f415bf66b48a9af192301b982f8dd (DEFAULT_TABLE table=default.Table_005b6c9953134fc3a42de2403722bdfe [id=dd31c07ae69b45dd969135aaccf2b999]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.775929 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 118753e8cb344b47a35a745aa8fed265. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.775970 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c66f415bf66b48a9af192301b982f8dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.776433 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cafa1d1231c54a608422695409720112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.776542 1174 ts_tablet_manager.cc:1434] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.037s user 0.002s sys 0.000s
I20260202 02:52:27.776705 1173 raft_consensus.cc:359] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:27.776865 1173 raft_consensus.cc:385] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:27.776867 1174 tablet_bootstrap.cc:492] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:27.776911 1173 raft_consensus.cc:740] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:27.777163 1173 consensus_queue.cc:260] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:27.780467 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8aee6931b74547ad9e0bfe0671eaaf1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.782601 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "06b80a1c9251420cbc16fc95ae98aca9" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:27.783671 998 leader_election.cc:343] T 06b80a1c9251420cbc16fc95ae98aca9 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:27.747670 894 tablet_service.cc:1505] Processing CreateTablet for tablet bbc789419cf54a00a1653c3801fe0bfd (DEFAULT_TABLE table=default.Table_6bec08bdda0a4c3ea7a50826fdd6e28a [id=eb75ceec451540ada1c0109dad3f1907]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.784883 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbc789419cf54a00a1653c3801fe0bfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.785190 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77f60925b6ca496d9b43502ed77bb55a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.788535 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1342810762e41588ea00ea02acad3e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.792732 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d594295561ff4b01bf13451533a5743c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.795462 1042 tablet_service.cc:1505] Processing CreateTablet for tablet d3f6a30e08244a07a1f81b5e98bc6965 (DEFAULT_TABLE table=default.Table_cb2845f3c47d4f96bb58a4ed5fe5b92b [id=4d8b49dd050f4a28856103bb02f4ab42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.796147 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 425dab1cf74c4f379f10e202613093dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.796623 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29cde4226db6428987d4a39cde804a5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.797008 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b02daa510384c0dab6a9627bbf425a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.797338 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 258e0afdba894f3cb2a3c7238e5634b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.797705 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4126bf5218644ca583c82190ce952267. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.798069 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82680e577c504a729122cd4f6b4916a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.798171 1029 tablet_service.cc:1505] Processing CreateTablet for tablet f381e9c3fe3d4dbba6895cec616cb6f6 (DEFAULT_TABLE table=default.Table_e87d06c7523440f7a739370378832d12 [id=e2c20a10802c4054862d6841a1865510]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.798494 1972 raft_consensus.cc:493] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:27.798671 1972 raft_consensus.cc:515] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.800777 1038 tablet_service.cc:1505] Processing CreateTablet for tablet 00f2d3bafb2d472aaffc0a7ba926c825 (DEFAULT_TABLE table=default.Table_1f6c33e9af15400180eb052e679585e7 [id=28cc2ced6205455c98cebe9127e9257c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.802314 1032 tablet_service.cc:1505] Processing CreateTablet for tablet e7df9a78735547cbbaa006c604b95fec (DEFAULT_TABLE table=default.Table_9465647c983f4c85b7b2b5454fcc80e7 [id=24b8f6a5c7b9466e9ff1dbb43988ac3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.802771 1026 tablet_service.cc:1505] Processing CreateTablet for tablet c66f415bf66b48a9af192301b982f8dd (DEFAULT_TABLE table=default.Table_005b6c9953134fc3a42de2403722bdfe [id=dd31c07ae69b45dd969135aaccf2b999]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.809281 1039 tablet_service.cc:1505] Processing CreateTablet for tablet 404125def15a4f1fb1529447785fc93e (DEFAULT_TABLE table=default.Table_843cd26c176e43c588824e3225124494 [id=3256d26f41564428b41ecf40e820ad04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.809665 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 404125def15a4f1fb1529447785fc93e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.814792 1033 tablet_service.cc:1505] Processing CreateTablet for tablet 2d18485f5a4a4b11916c64a487cd28b4 (DEFAULT_TABLE table=default.Table_58420fa37ead4e3c97dd3108c58b31a1 [id=66b46ed502ce4c7f920c18089e17d579]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.815243 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d18485f5a4a4b11916c64a487cd28b4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:27.815328 515 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.190:33981: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20260202 02:52:27.816372 515 consensus_peers.cc:597] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b -> Peer 7dab2abef14445d688ce1335b1f9b2f8 (127.0.104.190:33981): Couldn't send request to peer 7dab2abef14445d688ce1335b1f9b2f8. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.190:33981: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260202 02:52:27.819761 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7df9a78735547cbbaa006c604b95fec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.820016 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f381e9c3fe3d4dbba6895cec616cb6f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.820984 1030 tablet_service.cc:1505] Processing CreateTablet for tablet 305648e83e314b7fb7c74fe528c04adf (DEFAULT_TABLE table=default.Table_8d940a39b4194caca019b3cb5ee39830 [id=68d698b5760e455a8584cb0069f97f7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.824684 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 8a3ebe597da64da48db881cef46cfe4e (DEFAULT_TABLE table=default.Table_fbe3212129904307af7133ead4053a59 [id=0aeb16c903f642d8bfa236a96e203767]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.825947 893 tablet_service.cc:1505] Processing CreateTablet for tablet 404125def15a4f1fb1529447785fc93e (DEFAULT_TABLE table=default.Table_843cd26c176e43c588824e3225124494 [id=3256d26f41564428b41ecf40e820ad04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.830531 1036 tablet_service.cc:1505] Processing CreateTablet for tablet 21fe651f45bf43609cf972bef5eca149 (DEFAULT_TABLE table=default.Table_06273452ab3e4d2e8bdb1e3a41721dcc [id=4096475017214ffdba27d20aca4e8d64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.830682 1882 tablet_replica.cc:333] T 75f7824681344d8d88df9402694fff1b P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:27.831017 1882 raft_consensus.cc:2243] T 75f7824681344d8d88df9402694fff1b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:27.831256 1882 raft_consensus.cc:2272] T 75f7824681344d8d88df9402694fff1b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:27.831888 1882 ts_tablet_manager.cc:1916] T 75f7824681344d8d88df9402694fff1b P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:27.832369 903 tablet_service.cc:1505] Processing CreateTablet for tablet 00f2d3bafb2d472aaffc0a7ba926c825 (DEFAULT_TABLE table=default.Table_1f6c33e9af15400180eb052e679585e7 [id=28cc2ced6205455c98cebe9127e9257c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.836141 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a3ebe597da64da48db881cef46cfe4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.837221 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00f2d3bafb2d472aaffc0a7ba926c825. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:27.837255 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 8cf6a52eea054ab688410e2dd029abf3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8cf6a52eea054ab688410e2dd029abf3 already in progress: creating tablet
W20260202 02:52:27.837544 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet ce757ddc5c144d1fbaec84c9723cdbf2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ce757ddc5c144d1fbaec84c9723cdbf2 already in progress: creating tablet
I20260202 02:52:27.837894 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c66f415bf66b48a9af192301b982f8dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.838346 1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21fe651f45bf43609cf972bef5eca149. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.838474 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 305648e83e314b7fb7c74fe528c04adf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.841936 1031 tablet_service.cc:1505] Processing CreateTablet for tablet e481fcf5282c4f3caf276ee8b96a5d51 (DEFAULT_TABLE table=default.Table_52ea2b9c8a854253941ccd8d0d68a409 [id=8c5ae84413054e7a9c2d508f0dfbaf83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.842917 1173 ts_tablet_manager.cc:1434] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.290s user 0.001s sys 0.000s
I20260202 02:52:27.845333 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3f6a30e08244a07a1f81b5e98bc6965. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.851016 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 6671d3b545eb4371bfa71fe703c214f2 (DEFAULT_TABLE table=default.Table_516d610640654ea4b49f7de3b04cbb0c [id=5c8367fbb78840d18a26a026d75c3d6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.851508 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6671d3b545eb4371bfa71fe703c214f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.853202 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 4ec86e245f8c4d9e87c3ee3409a164b3 (DEFAULT_TABLE table=default.Table_f6c3d9d5bdc94a499428668182f0a8a9 [id=3cd20462832f4e1286530e6863fa9333]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.853879 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ec86e245f8c4d9e87c3ee3409a164b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.858809 1039 tablet_service.cc:1505] Processing CreateTablet for tablet 6b2a86a251da4196ba21848166b29de1 (DEFAULT_TABLE table=default.Table_8ffb80597a3c44f3ad9abbfffddda115 [id=91b1f0248e7f4b6eb23a83463a2190f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.859206 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b2a86a251da4196ba21848166b29de1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.861979 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06016cee61b64c59b2f01e8f01e445cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.862067 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e481fcf5282c4f3caf276ee8b96a5d51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.863941 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 404125def15a4f1fb1529447785fc93e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.864199 903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00f2d3bafb2d472aaffc0a7ba926c825. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.868479 1034 tablet_service.cc:1505] Processing CreateTablet for tablet f66be6c9ca98432cb0b18b6cbf1d8a0f (DEFAULT_TABLE table=default.Table_9bc03f5de9284dd18457bca6459ca366 [id=904ccf91a3544ad29a947faa0385c46f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.869046 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f66be6c9ca98432cb0b18b6cbf1d8a0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.869158 1041 tablet_service.cc:1505] Processing CreateTablet for tablet 5416eab7a0914ecb855fbccde4894285 (DEFAULT_TABLE table=default.Table_c7654f22d0044539a2eb8a736163a22d [id=427f58cc70ef47a6afac8de0353975f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.869505 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5416eab7a0914ecb855fbccde4894285. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.873746 1173 tablet_bootstrap.cc:492] T d30557fbe9084cd3860820f69dda8752 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:27.875376 886 tablet_service.cc:1505] Processing CreateTablet for tablet 2d18485f5a4a4b11916c64a487cd28b4 (DEFAULT_TABLE table=default.Table_58420fa37ead4e3c97dd3108c58b31a1 [id=66b46ed502ce4c7f920c18089e17d579]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.876092 1173 tablet_bootstrap.cc:654] T d30557fbe9084cd3860820f69dda8752 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:27.876736 1175 tablet_bootstrap.cc:492] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:27.876861 1175 ts_tablet_manager.cc:1403] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.379s user 0.065s sys 0.107s
I20260202 02:52:27.877584 1175 raft_consensus.cc:359] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.878108 1175 raft_consensus.cc:385] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:27.878165 1175 raft_consensus.cc:740] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:27.878342 1175 consensus_queue.cc:260] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:27.882058 1035 tablet_service.cc:1505] Processing CreateTablet for tablet be12284e01c34523b4a041fa0c9851c4 (DEFAULT_TABLE table=default.Table_a369e324e7614aeb805af657e03b040a [id=3a3cc0d3787d4aa59b8c3bf5fa77f272]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.882447 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be12284e01c34523b4a041fa0c9851c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.884958 1972 leader_election.cc:290] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:27.886087 1040 tablet_service.cc:1505] Processing CreateTablet for tablet 79e65a55f29449a0b13349bc152c651b (DEFAULT_TABLE table=default.Table_f0ad9b13e025411d844149bea0d37e2f [id=0baa8b57f9204d9ba68abd93c9a3f095]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.886462 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79e65a55f29449a0b13349bc152c651b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.889323 1043 tablet_service.cc:1505] Processing CreateTablet for tablet be7d0469760b48eaba87c7e762d40d13 (DEFAULT_TABLE table=default.Table_ee90cc0e866a44b28f684fc4236c297a [id=e9f1972a5e90480da13a72cacefa8295]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.889676 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be7d0469760b48eaba87c7e762d40d13. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:27.893630 999 leader_election.cc:343] T 06b80a1c9251420cbc16fc95ae98aca9 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 06b80a1c9251420cbc16fc95ae98aca9
I20260202 02:52:27.893790 999 leader_election.cc:304] T 06b80a1c9251420cbc16fc95ae98aca9 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:27.894260 2127 raft_consensus.cc:2749] T 06b80a1c9251420cbc16fc95ae98aca9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:27.851926 1028 tablet_service.cc:1505] Processing CreateTablet for tablet 6f81123900e648be8a117e1b925319e8 (DEFAULT_TABLE table=default.Table_e0fefcc5497b47daa332f70f03a81639 [id=1c4c1146e0b749e683efd14824758eff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.895304 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f81123900e648be8a117e1b925319e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.903992 901 tablet_service.cc:1505] Processing CreateTablet for tablet 21fe651f45bf43609cf972bef5eca149 (DEFAULT_TABLE table=default.Table_06273452ab3e4d2e8bdb1e3a41721dcc [id=4096475017214ffdba27d20aca4e8d64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.855257 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 7c07195a46184d24bc709e4967945472 (DEFAULT_TABLE table=default.Table_910b1d34ecd54cd38bff82420fe1a154 [id=2d8587df1c174dfd8a4b2a4d053c7821]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.905655 892 tablet_service.cc:1505] Processing CreateTablet for tablet 305648e83e314b7fb7c74fe528c04adf (DEFAULT_TABLE table=default.Table_8d940a39b4194caca019b3cb5ee39830 [id=68d698b5760e455a8584cb0069f97f7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.905848 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c07195a46184d24bc709e4967945472. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.906090 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 305648e83e314b7fb7c74fe528c04adf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.907999 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21fe651f45bf43609cf972bef5eca149. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.909636 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d18485f5a4a4b11916c64a487cd28b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.912853 897 tablet_service.cc:1505] Processing CreateTablet for tablet e481fcf5282c4f3caf276ee8b96a5d51 (DEFAULT_TABLE table=default.Table_52ea2b9c8a854253941ccd8d0d68a409 [id=8c5ae84413054e7a9c2d508f0dfbaf83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.913338 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e481fcf5282c4f3caf276ee8b96a5d51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.916777 898 tablet_service.cc:1505] Processing CreateTablet for tablet 6671d3b545eb4371bfa71fe703c214f2 (DEFAULT_TABLE table=default.Table_516d610640654ea4b49f7de3b04cbb0c [id=5c8367fbb78840d18a26a026d75c3d6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.917241 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6671d3b545eb4371bfa71fe703c214f2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:27.918771 515 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 79c73f4a92044bafb5b8384229cb3a4d (table default.Table_a15c97bb32284dcfb36cdc47921b2346 [id=dc77705cb5704fd58af1eed3edd45675], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 79c73f4a92044bafb5b8384229cb3a4d consensus state has no leader
I20260202 02:52:27.926103 885 tablet_service.cc:1505] Processing CreateTablet for tablet 6f81123900e648be8a117e1b925319e8 (DEFAULT_TABLE table=default.Table_e0fefcc5497b47daa332f70f03a81639 [id=1c4c1146e0b749e683efd14824758eff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.930745 889 tablet_service.cc:1505] Processing CreateTablet for tablet 4ec86e245f8c4d9e87c3ee3409a164b3 (DEFAULT_TABLE table=default.Table_f6c3d9d5bdc94a499428668182f0a8a9 [id=3cd20462832f4e1286530e6863fa9333]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.931177 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ec86e245f8c4d9e87c3ee3409a164b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.932766 899 tablet_service.cc:1505] Processing CreateTablet for tablet 7c07195a46184d24bc709e4967945472 (DEFAULT_TABLE table=default.Table_910b1d34ecd54cd38bff82420fe1a154 [id=2d8587df1c174dfd8a4b2a4d053c7821]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.933918 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c07195a46184d24bc709e4967945472. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.856920 752 tablet_service.cc:1505] Processing CreateTablet for tablet e7df9a78735547cbbaa006c604b95fec (DEFAULT_TABLE table=default.Table_9465647c983f4c85b7b2b5454fcc80e7 [id=24b8f6a5c7b9466e9ff1dbb43988ac3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.940560 746 tablet_service.cc:1505] Processing CreateTablet for tablet 305648e83e314b7fb7c74fe528c04adf (DEFAULT_TABLE table=default.Table_8d940a39b4194caca019b3cb5ee39830 [id=68d698b5760e455a8584cb0069f97f7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.933478 884 tablet_service.cc:1505] Processing CreateTablet for tablet 8a3ebe597da64da48db881cef46cfe4e (DEFAULT_TABLE table=default.Table_fbe3212129904307af7133ead4053a59 [id=0aeb16c903f642d8bfa236a96e203767]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.943430 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a3ebe597da64da48db881cef46cfe4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.952617 750 tablet_service.cc:1505] Processing CreateTablet for tablet 2d18485f5a4a4b11916c64a487cd28b4 (DEFAULT_TABLE table=default.Table_58420fa37ead4e3c97dd3108c58b31a1 [id=66b46ed502ce4c7f920c18089e17d579]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.959210 773 raft_consensus.cc:3060] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:27.964816 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7df9a78735547cbbaa006c604b95fec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.965427 754 tablet_service.cc:1505] Processing CreateTablet for tablet 8a3ebe597da64da48db881cef46cfe4e (DEFAULT_TABLE table=default.Table_fbe3212129904307af7133ead4053a59 [id=0aeb16c903f642d8bfa236a96e203767]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.966709 749 tablet_service.cc:1505] Processing CreateTablet for tablet e481fcf5282c4f3caf276ee8b96a5d51 (DEFAULT_TABLE table=default.Table_52ea2b9c8a854253941ccd8d0d68a409 [id=8c5ae84413054e7a9c2d508f0dfbaf83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.976545 742 tablet_service.cc:1505] Processing CreateTablet for tablet 21fe651f45bf43609cf972bef5eca149 (DEFAULT_TABLE table=default.Table_06273452ab3e4d2e8bdb1e3a41721dcc [id=4096475017214ffdba27d20aca4e8d64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.969159 890 tablet_service.cc:1505] Processing CreateTablet for tablet 6b2a86a251da4196ba21848166b29de1 (DEFAULT_TABLE table=default.Table_8ffb80597a3c44f3ad9abbfffddda115 [id=91b1f0248e7f4b6eb23a83463a2190f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.978341 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b2a86a251da4196ba21848166b29de1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:27.995204 743 tablet_service.cc:1505] Processing CreateTablet for tablet 6671d3b545eb4371bfa71fe703c214f2 (DEFAULT_TABLE table=default.Table_516d610640654ea4b49f7de3b04cbb0c [id=5c8367fbb78840d18a26a026d75c3d6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.995100 753 tablet_service.cc:1505] Processing CreateTablet for tablet 4ec86e245f8c4d9e87c3ee3409a164b3 (DEFAULT_TABLE table=default.Table_f6c3d9d5bdc94a499428668182f0a8a9 [id=3cd20462832f4e1286530e6863fa9333]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:27.997291 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d18485f5a4a4b11916c64a487cd28b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.000813 745 tablet_service.cc:1505] Processing CreateTablet for tablet 7c07195a46184d24bc709e4967945472 (DEFAULT_TABLE table=default.Table_910b1d34ecd54cd38bff82420fe1a154 [id=2d8587df1c174dfd8a4b2a4d053c7821]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:28.002892 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f81123900e648be8a117e1b925319e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.010506 747 tablet_service.cc:1505] Processing CreateTablet for tablet 6b2a86a251da4196ba21848166b29de1 (DEFAULT_TABLE table=default.Table_8ffb80597a3c44f3ad9abbfffddda115 [id=91b1f0248e7f4b6eb23a83463a2190f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:28.005363 1044 tablet_service.cc:1505] Processing CreateTablet for tablet b85f0d1222274ff09a1f2db1c1bfe036 (DEFAULT_TABLE table=default.Table_a2d18aa48b1e43f5a58ff4bd751992ff [id=e9950c1aa1604999825dbe4a817ee403]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:28.020876 891 tablet_service.cc:1505] Processing CreateTablet for tablet f66be6c9ca98432cb0b18b6cbf1d8a0f (DEFAULT_TABLE table=default.Table_9bc03f5de9284dd18457bca6459ca366 [id=904ccf91a3544ad29a947faa0385c46f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:28.021351 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f66be6c9ca98432cb0b18b6cbf1d8a0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.022269 2042 raft_consensus.cc:493] T af5aa68e47bb420eac300754d274fa73 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:28.025204 894 tablet_service.cc:1505] Processing CreateTablet for tablet 5416eab7a0914ecb855fbccde4894285 (DEFAULT_TABLE table=default.Table_c7654f22d0044539a2eb8a736163a22d [id=427f58cc70ef47a6afac8de0353975f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:28.025609 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5416eab7a0914ecb855fbccde4894285. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.027458 748 tablet_service.cc:1505] Processing CreateTablet for tablet 6f81123900e648be8a117e1b925319e8 (DEFAULT_TABLE table=default.Table_e0fefcc5497b47daa332f70f03a81639 [id=1c4c1146e0b749e683efd14824758eff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:28.030483 1174 tablet_bootstrap.cc:654] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:28.031744 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 305648e83e314b7fb7c74fe528c04adf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.032394 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ec86e245f8c4d9e87c3ee3409a164b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.032642 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f81123900e648be8a117e1b925319e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.034809 743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6671d3b545eb4371bfa71fe703c214f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.038892 2042 raft_consensus.cc:515] T af5aa68e47bb420eac300754d274fa73 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.045461 740 tablet_service.cc:1505] Processing CreateTablet for tablet 79e65a55f29449a0b13349bc152c651b (DEFAULT_TABLE table=default.Table_f0ad9b13e025411d844149bea0d37e2f [id=0baa8b57f9204d9ba68abd93c9a3f095]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:28.047479 1180 ts_tablet_manager.cc:1929] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:28.047891 755 tablet_service.cc:1505] Processing CreateTablet for tablet f66be6c9ca98432cb0b18b6cbf1d8a0f (DEFAULT_TABLE table=default.Table_9bc03f5de9284dd18457bca6459ca366 [id=904ccf91a3544ad29a947faa0385c46f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:28.047976 744 tablet_service.cc:1505] Processing CreateTablet for tablet 5416eab7a0914ecb855fbccde4894285 (DEFAULT_TABLE table=default.Table_c7654f22d0044539a2eb8a736163a22d [id=427f58cc70ef47a6afac8de0353975f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:28.048393 756 tablet_service.cc:1505] Processing CreateTablet for tablet be12284e01c34523b4a041fa0c9851c4 (DEFAULT_TABLE table=default.Table_a369e324e7614aeb805af657e03b040a [id=3a3cc0d3787d4aa59b8c3bf5fa77f272]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:28.049749 714 leader_election.cc:343] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 8774c6bc0fe9414ea2f17ed859d9392a
I20260202 02:52:28.051240 920 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "af5aa68e47bb420eac300754d274fa73" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:28.051903 920 raft_consensus.cc:2393] T af5aa68e47bb420eac300754d274fa73 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f442ef85029461091de847118c34c0d in current term 1: Already voted for candidate b1f05820b6034228bdad884cc348716d in this term.
I20260202 02:52:28.053076 714 leader_election.cc:304] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
W20260202 02:52:28.053715 714 leader_election.cc:343] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 68823eff1bc04108ba83487605aec0ea
I20260202 02:52:28.038961 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b85f0d1222274ff09a1f2db1c1bfe036. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.055609 1175 ts_tablet_manager.cc:1434] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.179s user 0.013s sys 0.054s
I20260202 02:52:28.051043 2042 leader_election.cc:290] T af5aa68e47bb420eac300754d274fa73 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:28.057276 1175 tablet_bootstrap.cc:492] T d8418a9dd0944456b4d4820dde0cde02 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
W20260202 02:52:28.070034 715 leader_election.cc:343] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Not found: Tablet not found: eb0ac55fbb534b29a86d6b3e8aa72ebc
W20260202 02:52:28.072423 714 leader_election.cc:343] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 042d1ebf357b4abe8814a407891ce265
I20260202 02:52:28.073777 2126 raft_consensus.cc:493] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:28.073927 2126 raft_consensus.cc:515] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:28.074541 2126 leader_election.cc:290] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:28.075780 919 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "16b47a2831f54d809ed0b16deaec7e3d" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:28.076128 919 raft_consensus.cc:2468] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:28.076977 999 leader_election.cc:304] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:28.077126 2150 raft_consensus.cc:3060] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:28.077476 2126 raft_consensus.cc:2804] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:28.077555 2126 raft_consensus.cc:493] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:28.077610 2126 raft_consensus.cc:3060] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:28.077610 759 tablet_service.cc:1505] Processing CreateTablet for tablet be7d0469760b48eaba87c7e762d40d13 (DEFAULT_TABLE table=default.Table_ee90cc0e866a44b28f684fc4236c297a [id=e9f1972a5e90480da13a72cacefa8295]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:28.078387 714 leader_election.cc:343] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 042d1ebf357b4abe8814a407891ce265
I20260202 02:52:28.080199 2126 raft_consensus.cc:515] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:28.081610 2126 leader_election.cc:290] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:28.077638 2042 consensus_queue.cc:1048] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260202 02:52:28.082185 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "af5aa68e47bb420eac300754d274fa73" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:28.082378 2151 raft_consensus.cc:493] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:28.082558 2151 raft_consensus.cc:515] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.084508 1175 tablet_bootstrap.cc:654] T d8418a9dd0944456b4d4820dde0cde02 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:28.084771 2151 leader_election.cc:290] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:28.085511 1180 log.cc:1199] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/d64f42d187bc4c9f9381c931449ec0cc
I20260202 02:52:28.085913 1180 ts_tablet_manager.cc:1950] T d64f42d187bc4c9f9381c931449ec0cc P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:28.088290 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 404) took 4831 ms. Trace:
I20260202 02:52:28.088428 1180 rpcz_store.cc:276] 0202 02:52:23.255573 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.255618 (+ 45us) service_pool.cc:225] Handling call
0202 02:52:27.467615 (+4211997us) ts_tablet_manager.cc:1151] Deleting tablet d64f42d187bc4c9f9381c931449ec0cc
0202 02:52:27.467693 (+ 78us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:27.550764 (+ 83071us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.047367 (+496603us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.086669 (+ 39302us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4167174}
I20260202 02:52:28.085755 919 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "16b47a2831f54d809ed0b16deaec7e3d" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:28.089207 919 raft_consensus.cc:3060] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:28.089316 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a3ebe597da64da48db881cef46cfe4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.089844 2042 consensus_queue.cc:1048] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260202 02:52:28.096046 2150 raft_consensus.cc:2749] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:28.096603 887 tablet_service.cc:1505] Processing CreateTablet for tablet 79e65a55f29449a0b13349bc152c651b (DEFAULT_TABLE table=default.Table_f0ad9b13e025411d844149bea0d37e2f [id=0baa8b57f9204d9ba68abd93c9a3f095]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:28.097086 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79e65a55f29449a0b13349bc152c651b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.098464 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79e65a55f29449a0b13349bc152c651b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.099407 2158 raft_consensus.cc:493] T ffd80badce1048f3ac95366cad05ae53 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:28.099588 2158 raft_consensus.cc:515] T ffd80badce1048f3ac95366cad05ae53 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.100524 2158 leader_election.cc:290] T ffd80badce1048f3ac95366cad05ae53 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:28.101056 714 consensus_peers.cc:597] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d0671586e45e45ffb7fd9e1be6e2bbf8. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:28.101444 714 leader_election.cc:343] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: eb0ac55fbb534b29a86d6b3e8aa72ebc
I20260202 02:52:28.101711 714 leader_election.cc:304] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:28.102862 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 405) took 4840 ms. Trace:
I20260202 02:52:28.103029 1180 rpcz_store.cc:276] 0202 02:52:23.255873 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.255918 (+ 45us) service_pool.cc:225] Handling call
0202 02:52:28.088701 (+4832783us) ts_tablet_manager.cc:1151] Deleting tablet 26eeef6eebbf4f2cb44f69255fb37594
0202 02:52:28.088711 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.096418 (+ 7707us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4787670}
I20260202 02:52:28.103488 1180 tablet_replica.cc:333] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:28.103729 1180 raft_consensus.cc:2243] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:28.104166 1180 raft_consensus.cc:2272] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:28.104887 1180 ts_tablet_manager.cc:1916] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:28.105396 2150 raft_consensus.cc:2749] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260202 02:52:28.105720 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c07195a46184d24bc709e4967945472. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.106220 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e481fcf5282c4f3caf276ee8b96a5d51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.106328 757 tablet_service.cc:1505] Processing CreateTablet for tablet b85f0d1222274ff09a1f2db1c1bfe036 (DEFAULT_TABLE table=default.Table_a2d18aa48b1e43f5a58ff4bd751992ff [id=e9950c1aa1604999825dbe4a817ee403]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:28.106560 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f66be6c9ca98432cb0b18b6cbf1d8a0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.106853 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b85f0d1222274ff09a1f2db1c1bfe036. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.107405 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be7d0469760b48eaba87c7e762d40d13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.107731 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5416eab7a0914ecb855fbccde4894285. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.110069 1173 tablet_bootstrap.cc:492] T d30557fbe9084cd3860820f69dda8752 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:28.110291 1173 ts_tablet_manager.cc:1403] T d30557fbe9084cd3860820f69dda8752 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.237s user 0.022s sys 0.000s
W20260202 02:52:28.110719 998 consensus_peers.cc:597] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0186572e7bb9499a932a636ab280f544. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:28.111436 1173 raft_consensus.cc:359] T d30557fbe9084cd3860820f69dda8752 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:28.111694 1173 raft_consensus.cc:385] T d30557fbe9084cd3860820f69dda8752 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:28.112779 902 tablet_service.cc:1505] Processing CreateTablet for tablet be12284e01c34523b4a041fa0c9851c4 (DEFAULT_TABLE table=default.Table_a369e324e7614aeb805af657e03b040a [id=3a3cc0d3787d4aa59b8c3bf5fa77f272]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:28.114725 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be12284e01c34523b4a041fa0c9851c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.133464 525 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 2c31b45dc8004eae993e972e9faa3ff8
I20260202 02:52:28.133965 525 catalog_manager.cc:6724] ReplaceTablet: tablet 2c31b45dc8004eae993e972e9faa3ff8 deleted and replaced by tablet aed1081136f7431b9b14784c21112d1c
W20260202 02:52:28.141534 520 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet be993c1512b04df4b92cab3195241d84 consensus state has no leader
W20260202 02:52:28.141813 527 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 67a8c9a1998b47b3b214a06ce62381e9 consensus state has no leader
I20260202 02:52:28.172022 900 tablet_service.cc:1505] Processing CreateTablet for tablet be7d0469760b48eaba87c7e762d40d13 (DEFAULT_TABLE table=default.Table_ee90cc0e866a44b28f684fc4236c297a [id=e9f1972a5e90480da13a72cacefa8295]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:28.173306 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be7d0469760b48eaba87c7e762d40d13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.205082 896 tablet_service.cc:1505] Processing CreateTablet for tablet b85f0d1222274ff09a1f2db1c1bfe036 (DEFAULT_TABLE table=default.Table_a2d18aa48b1e43f5a58ff4bd751992ff [id=e9950c1aa1604999825dbe4a817ee403]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:28.205546 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b85f0d1222274ff09a1f2db1c1bfe036. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:28.211081 714 consensus_peers.cc:597] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 59990791123c4d4eb620090a1e8e31e1. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:28.121250 2126 consensus_queue.cc:1048] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:28.214793 1173 raft_consensus.cc:740] T d30557fbe9084cd3860820f69dda8752 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:28.215199 1173 consensus_queue.cc:260] T d30557fbe9084cd3860820f69dda8752 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:28.215839 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be12284e01c34523b4a041fa0c9851c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.216034 1173 ts_tablet_manager.cc:1434] T d30557fbe9084cd3860820f69dda8752 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.106s user 0.002s sys 0.000s
I20260202 02:52:28.216313 1173 tablet_bootstrap.cc:492] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:28.218462 494 master_runner.cc:186] Error getting master registration for 127.0.104.190:33981: Network error: Client connection negotiation failed: client connection to 127.0.104.190:33981: connect: Connection refused (error 111),
I20260202 02:52:28.216787 2160 tablet.cc:1722] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:28.218634 494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:52:28.175666 1882 ts_tablet_manager.cc:1929] T 75f7824681344d8d88df9402694fff1b P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:28.220118 1882 log.cc:1199] T 75f7824681344d8d88df9402694fff1b P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/75f7824681344d8d88df9402694fff1b
I20260202 02:52:28.220607 1882 ts_tablet_manager.cc:1950] T 75f7824681344d8d88df9402694fff1b P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:28.221604 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 376) took 4931 ms. Trace:
I20260202 02:52:28.221756 1882 rpcz_store.cc:276] 0202 02:52:23.290317 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.290385 (+ 68us) service_pool.cc:225] Handling call
0202 02:52:27.522405 (+4232020us) ts_tablet_manager.cc:1151] Deleting tablet 75f7824681344d8d88df9402694fff1b
0202 02:52:27.830623 (+308218us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:27.942077 (+111454us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.175562 (+233485us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.221580 (+ 46018us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4097589}
I20260202 02:52:28.222609 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 75f7824681344d8d88df9402694fff1b (table default.Table_ae49654353a64141b6d7f0e6afe13d9d [id=431440e879c04f0f9ad63c64d8741b56]) successfully deleted
I20260202 02:52:28.223037 1882 tablet_replica.cc:333] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:28.223150 892 tablet_service.cc:1552] Processing DeleteTablet for tablet 2c31b45dc8004eae993e972e9faa3ff8 with delete_type TABLET_DATA_DELETED (replaced by tablet aed1081136f7431b9b14784c21112d1c) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:28.223289 1882 raft_consensus.cc:2243] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:28.223687 1882 raft_consensus.cc:2272] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:28.193006 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 06b80a1c9251420cbc16fc95ae98aca9 with delete_type TABLET_DATA_DELETED (replaced by tablet 2f5aae089fcc492f9ce55f84674a452c) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:28.209916 1042 tablet_service.cc:1552] Processing DeleteTablet for tablet 2c31b45dc8004eae993e972e9faa3ff8 with delete_type TABLET_DATA_DELETED (replaced by tablet aed1081136f7431b9b14784c21112d1c) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:28.231308 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21fe651f45bf43609cf972bef5eca149. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.231499 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "16b47a2831f54d809ed0b16deaec7e3d" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:28.232874 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "16b47a2831f54d809ed0b16deaec7e3d" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
W20260202 02:52:28.235697 715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 163814us
I20260202 02:52:28.236912 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28a900bcaa8b41718a285e595886d805" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:28.237931 919 raft_consensus.cc:2468] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
W20260202 02:52:28.250617 524 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9d02de5549ba4436b4eea6750a3fef09 consensus state has no leader
I20260202 02:52:28.251472 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b2a86a251da4196ba21848166b29de1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.251518 2163 raft_consensus.cc:493] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260202 02:52:28.251756 715 leader_election.cc:343] T ffd80badce1048f3ac95366cad05ae53 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Not found: Tablet not found: ffd80badce1048f3ac95366cad05ae53
W20260202 02:52:28.260213 714 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 108477us
I20260202 02:52:28.261631 523 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 425dab1cf74c4f379f10e202613093dd
W20260202 02:52:28.261792 526 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 77f60925b6ca496d9b43502ed77bb55a consensus state has no leader
I20260202 02:52:28.263837 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet 425dab1cf74c4f379f10e202613093dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:27 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:28.268493 521 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_843cd26c176e43c588824e3225124494" } new_table_name: "default.Table_defe1743f1314f478ac7a7715442942e" modify_external_catalogs: true
I20260202 02:52:28.270483 520 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_f6c3d9d5bdc94a499428668182f0a8a9" } modify_external_catalogs: true
I20260202 02:52:28.270718 520 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_f6c3d9d5bdc94a499428668182f0a8a9" } modify_external_catalogs: true
I20260202 02:52:28.271498 527 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_1e4db591b5664638a2cd494429582cdf" } modify_external_catalogs: true
I20260202 02:52:28.271654 527 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_1e4db591b5664638a2cd494429582cdf" } modify_external_catalogs: true
I20260202 02:52:28.272203 522 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_8a25e41d22cf497aad652a6171bd142c" } modify_external_catalogs: true
I20260202 02:52:28.272362 522 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_8a25e41d22cf497aad652a6171bd142c" } modify_external_catalogs: true
I20260202 02:52:28.272877 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_f4a5e710b7a444bbbddef402779ebc8d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:28.273749 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_b08cae12b7694236b6451641de70c882"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:28.277468 526 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet cb634d4bae554ef3b39eeaa9958adc6c (table default.Table_e5f4ac82d1ef4b85999695997eb9df2a [id=c97e407e519d4a538860f66931f6aefd]). Expected version 2 got 0
I20260202 02:52:28.278363 526 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_11c571ac785448ca9a174c50d15ec37f" } modify_external_catalogs: true
I20260202 02:52:28.278590 526 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_11c571ac785448ca9a174c50d15ec37f" } modify_external_catalogs: true
I20260202 02:52:28.270510 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_3295106ca36c404cb1acc75bd288dd40"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:28.280128 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3295106ca36c404cb1acc75bd288dd40 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:28.281116 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b08cae12b7694236b6451641de70c882 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:28.282117 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f4a5e710b7a444bbbddef402779ebc8d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:28.304394 1175 tablet_bootstrap.cc:492] T d8418a9dd0944456b4d4820dde0cde02 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:28.304656 1175 ts_tablet_manager.cc:1403] T d8418a9dd0944456b4d4820dde0cde02 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.247s user 0.004s sys 0.004s
W20260202 02:52:28.307850 998 leader_election.cc:343] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:28.308104 998 leader_election.cc:343] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:28.309964 1173 tablet_bootstrap.cc:654] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:28.311715 2173 tablet.cc:1722] T 91ec01d25f8f441cbe3783cacbee51db P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:28.317932 1174 tablet_bootstrap.cc:492] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:28.319904 1721 raft_consensus.cc:493] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:28.320051 1721 raft_consensus.cc:515] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.320704 1721 leader_election.cc:290] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:28.324519 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "94bada52db6a47179415d302b541d01f" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:28.330621 1882 ts_tablet_manager.cc:1916] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:28.341143 714 leader_election.cc:304] T af5aa68e47bb420eac300754d274fa73 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:28.350019 776 raft_consensus.cc:2468] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:28.350296 2161 tablet.cc:1722] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:28.317013 2163 raft_consensus.cc:515] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.352433 2163 leader_election.cc:290] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:28.352632 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c25e329023c04f609d8a743b4f7a95a4" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:28.352828 1062 raft_consensus.cc:2468] T c25e329023c04f609d8a743b4f7a95a4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:28.352869 2150 raft_consensus.cc:3060] T af5aa68e47bb420eac300754d274fa73 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:28.352806 1175 raft_consensus.cc:359] T d8418a9dd0944456b4d4820dde0cde02 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.352975 1175 raft_consensus.cc:385] T d8418a9dd0944456b4d4820dde0cde02 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:28.353024 1175 raft_consensus.cc:740] T d8418a9dd0944456b4d4820dde0cde02 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:28.353035 998 leader_election.cc:304] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:28.353266 1721 raft_consensus.cc:2804] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260202 02:52:28.353332 1721 raft_consensus.cc:493] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:28.353386 1721 raft_consensus.cc:3060] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:28.354451 715 leader_election.cc:304] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:28.354851 1174 ts_tablet_manager.cc:1403] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.541s user 0.003s sys 0.003s
I20260202 02:52:28.355990 1721 raft_consensus.cc:515] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.356796 1721 leader_election.cc:290] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:28.360967 2173 tablet.cc:1722] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260202 02:52:28.362434 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 2c31b45dc8004eae993e972e9faa3ff8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2c31b45dc8004eae993e972e9faa3ff8 already in progress: creating tablet
W20260202 02:52:28.362664 714 consensus_peers.cc:597] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2f0e8c5e2fd44370ae150a206f4e6cbc. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:28.362792 999 leader_election.cc:304] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:28.363260 1721 raft_consensus.cc:2804] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:28.363515 1721 raft_consensus.cc:697] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:28.363765 1721 consensus_queue.cc:237] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
W20260202 02:52:28.364317 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 26eeef6eebbf4f2cb44f69255fb37594 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 26eeef6eebbf4f2cb44f69255fb37594 already in progress: creating tablet
I20260202 02:52:28.364893 2042 raft_consensus.cc:493] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:28.353214 1175 consensus_queue.cc:260] T d8418a9dd0944456b4d4820dde0cde02 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.371471 1175 ts_tablet_manager.cc:1434] T d8418a9dd0944456b4d4820dde0cde02 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.067s user 0.001s sys 0.001s
I20260202 02:52:28.371860 1175 tablet_bootstrap.cc:492] T 072bb7f0e4ab4935be49461bc8a3c163 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:28.373421 1175 tablet_bootstrap.cc:654] T 072bb7f0e4ab4935be49461bc8a3c163 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:28.365021 2042 raft_consensus.cc:515] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.378268 748 tablet_service.cc:1552] Processing DeleteTablet for tablet 425dab1cf74c4f379f10e202613093dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:27 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:28.385977 2151 raft_consensus.cc:2804] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:28.386090 2151 raft_consensus.cc:493] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:28.386152 2151 raft_consensus.cc:3060] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:28.386845 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "08d8d781bb4e4acf915f11afb4fb2e22" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:28.387037 1062 raft_consensus.cc:2468] T 08d8d781bb4e4acf915f11afb4fb2e22 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:28.388093 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "94bada52db6a47179415d302b541d01f" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:28.388254 773 raft_consensus.cc:3060] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:28.388832 753 tablet_service.cc:1552] Processing DeleteTablet for tablet 2c31b45dc8004eae993e972e9faa3ff8 with delete_type TABLET_DATA_DELETED (replaced by tablet aed1081136f7431b9b14784c21112d1c) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:28.389647 2163 consensus_queue.cc:1048] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260202 02:52:28.390012 2158 raft_consensus.cc:493] T 75b97d7e848241cf8d894b0ae92a8f8f P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:28.387418 715 leader_election.cc:304] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:28.391460 2169 raft_consensus.cc:493] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:28.391623 2169 raft_consensus.cc:515] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.392356 2169 leader_election.cc:290] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:28.390127 2158 raft_consensus.cc:515] T 75b97d7e848241cf8d894b0ae92a8f8f P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.393452 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6b8f924aecf44e959e47d2f32e6b0bf5" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:28.393882 859 leader_election.cc:343] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:28.394646 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 425dab1cf74c4f379f10e202613093dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:27 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:28.394778 2042 leader_election.cc:290] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:28.394924 2042 raft_consensus.cc:2804] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:28.394987 2042 raft_consensus.cc:493] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:28.395040 2042 raft_consensus.cc:3060] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:28.407440 1173 tablet_bootstrap.cc:492] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:28.407794 1173 ts_tablet_manager.cc:1403] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.192s user 0.001s sys 0.003s
I20260202 02:52:28.408700 1173 raft_consensus.cc:359] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:28.409008 1173 raft_consensus.cc:385] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:28.409108 1173 raft_consensus.cc:740] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:28.409415 1173 consensus_queue.cc:260] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
W20260202 02:52:28.415746 521 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 404125def15a4f1fb1529447785fc93e consensus state has no leader
I20260202 02:52:28.419003 2161 tablet.cc:1722] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:28.423533 1175 tablet_bootstrap.cc:492] T 072bb7f0e4ab4935be49461bc8a3c163 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:28.423710 1175 ts_tablet_manager.cc:1403] T 072bb7f0e4ab4935be49461bc8a3c163 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.052s user 0.007s sys 0.000s
I20260202 02:52:28.433020 1175 raft_consensus.cc:359] T 072bb7f0e4ab4935be49461bc8a3c163 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.433254 1175 raft_consensus.cc:385] T 072bb7f0e4ab4935be49461bc8a3c163 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:28.433301 1175 raft_consensus.cc:740] T 072bb7f0e4ab4935be49461bc8a3c163 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:28.433564 1175 consensus_queue.cc:260] T 072bb7f0e4ab4935be49461bc8a3c163 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.434085 1175 ts_tablet_manager.cc:1434] T 072bb7f0e4ab4935be49461bc8a3c163 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.010s user 0.001s sys 0.000s
I20260202 02:52:28.434374 1175 tablet_bootstrap.cc:492] T 550895b4010f4088b2cc34acff50e03f P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:28.438398 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6b8f924aecf44e959e47d2f32e6b0bf5" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:28.439289 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 425dab1cf74c4f379f10e202613093dd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 425dab1cf74c4f379f10e202613093dd already in progress: creating tablet
I20260202 02:52:28.440426 2126 consensus_queue.cc:1048] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:28.441938 1173 ts_tablet_manager.cc:1434] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.034s user 0.002s sys 0.000s
I20260202 02:52:28.445231 2158 leader_election.cc:290] T 75b97d7e848241cf8d894b0ae92a8f8f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:28.448213 1174 raft_consensus.cc:359] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.448398 1174 raft_consensus.cc:385] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:28.448450 1174 raft_consensus.cc:740] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
W20260202 02:52:28.442281 999 leader_election.cc:343] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 94bada52db6a47179415d302b541d01f
I20260202 02:52:28.448603 1174 consensus_queue.cc:260] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:28.449000 999 leader_election.cc:343] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 94bada52db6a47179415d302b541d01f
I20260202 02:52:28.449056 1174 ts_tablet_manager.cc:1434] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.094s user 0.001s sys 0.000s
I20260202 02:52:28.450076 2173 tablet.cc:1722] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:28.451496 773 raft_consensus.cc:2468] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 2.
I20260202 02:52:28.452224 998 leader_election.cc:304] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:28.452610 2126 raft_consensus.cc:2804] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Leader election won for term 2
I20260202 02:52:28.452728 2126 raft_consensus.cc:697] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [term 2 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:28.452932 2126 consensus_queue.cc:237] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.454806 1180 ts_tablet_manager.cc:1929] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:28.454896 1180 log.cc:1199] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/526f2c7b95184a0987e7aa94efc8475a
I20260202 02:52:28.455511 1180 ts_tablet_manager.cc:1950] T 526f2c7b95184a0987e7aa94efc8475a P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:28.458976 1174 tablet_bootstrap.cc:492] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:28.460955 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 415) took 4957 ms. Trace:
I20260202 02:52:28.461153 1180 rpcz_store.cc:276] 0202 02:52:23.503285 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.503332 (+ 47us) service_pool.cc:225] Handling call
0202 02:52:28.103382 (+4600050us) ts_tablet_manager.cc:1151] Deleting tablet 526f2c7b95184a0987e7aa94efc8475a
0202 02:52:28.103398 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.335171 (+231773us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.454719 (+119548us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.460934 (+ 6215us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4582905}
I20260202 02:52:28.461793 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 526f2c7b95184a0987e7aa94efc8475a (table default.Table_0842fd6a82e145e7a98b3fed2c0c6993 [id=4e076b2023be4c888ac1c27babde1607]) successfully deleted
I20260202 02:52:28.462226 1180 tablet_replica.cc:333] T 75b97d7e848241cf8d894b0ae92a8f8f P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:28.462468 1180 raft_consensus.cc:2243] T 75b97d7e848241cf8d894b0ae92a8f8f P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:28.462688 1180 raft_consensus.cc:2272] T 75b97d7e848241cf8d894b0ae92a8f8f P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:28.465591 1173 tablet_bootstrap.cc:492] T fe9ea4c0502f4976b440465992f2ea2c P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:28.467427 922 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28a900bcaa8b41718a285e595886d805" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:28.467684 922 raft_consensus.cc:2468] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:28.475368 1175 tablet_bootstrap.cc:654] T 550895b4010f4088b2cc34acff50e03f P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:28.479839 525 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet 91ec01d25f8f441cbe3783cacbee51db (table default.Table_026b5574b3af4ff08f92f7ef3b3025ff [id=255f4d94812f45d0b4423150098460ff]). Expected version 2 got 0
I20260202 02:52:28.480129 525 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet af5aa68e47bb420eac300754d274fa73 (table default.Table_12c352712c374a45b7de7a0e634a1327 [id=feb8b9f926434cec8b119843413f8c9b]). Expected version 1 got 0
I20260202 02:52:28.482867 524 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 16b47a2831f54d809ed0b16deaec7e3d (table default.Table_e4018d1c57714a24be5d3a20c1ccaef4 [id=1e291b35a6764638acfb781bb4e6adb3]). Expected version 2 got 0
I20260202 02:52:28.484961 1039 tablet_service.cc:1505] Processing CreateTablet for tablet 35b3b4cdb6f646d5b28fae360ce82dad (DEFAULT_TABLE table=default.Table_a4f1629e749a48559f702106aa3833b3 [id=ce6c443a523a486aab3a1c1e0f7db946]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:28.485342 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35b3b4cdb6f646d5b28fae360ce82dad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.486872 1044 tablet_service.cc:1505] Processing CreateTablet for tablet b2149aff3eea4845b944f9fdc7303d2b (DEFAULT_TABLE table=default.Table_5c9207e4741746849a9cadbbc86ee35f [id=52f55002ce324313b27658ed262941d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:28.487362 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2149aff3eea4845b944f9fdc7303d2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.492579 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_e6e3448ca5cf4a07a1493a341487187a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:28.493165 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e6e3448ca5cf4a07a1493a341487187a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:28.492290 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_cba18d80d56e482f84815b720dc2a83f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:28.470723 2042 raft_consensus.cc:515] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:28.493988 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cba18d80d56e482f84815b720dc2a83f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:28.494133 2042 leader_election.cc:290] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:28.477380 2150 raft_consensus.cc:2749] T af5aa68e47bb420eac300754d274fa73 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:28.487674 2151 raft_consensus.cc:515] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.494726 1180 ts_tablet_manager.cc:1916] T 75b97d7e848241cf8d894b0ae92a8f8f P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:28.494802 2151 leader_election.cc:290] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:28.490821 752 tablet_service.cc:1552] Processing DeleteTablet for tablet 8774c6bc0fe9414ea2f17ed859d9392a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:28.491003 1174 tablet_bootstrap.cc:654] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:28.491183 742 tablet_service.cc:1505] Processing CreateTablet for tablet 35b3b4cdb6f646d5b28fae360ce82dad (DEFAULT_TABLE table=default.Table_a4f1629e749a48559f702106aa3833b3 [id=ce6c443a523a486aab3a1c1e0f7db946]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:28.495389 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35b3b4cdb6f646d5b28fae360ce82dad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.491339 744 tablet_service.cc:1552] Processing DeleteTablet for tablet 68823eff1bc04108ba83487605aec0ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:21 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:28.491907 759 tablet_service.cc:1505] Processing CreateTablet for tablet b2149aff3eea4845b944f9fdc7303d2b (DEFAULT_TABLE table=default.Table_5c9207e4741746849a9cadbbc86ee35f [id=52f55002ce324313b27658ed262941d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:28.499656 887 tablet_service.cc:1505] Processing CreateTablet for tablet 35b3b4cdb6f646d5b28fae360ce82dad (DEFAULT_TABLE table=default.Table_a4f1629e749a48559f702106aa3833b3 [id=ce6c443a523a486aab3a1c1e0f7db946]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:28.499883 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2149aff3eea4845b944f9fdc7303d2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.500314 2160 tablet.cc:1722] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:28.501230 1882 ts_tablet_manager.cc:1929] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:28.501575 1882 log.cc:1199] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/f93e8d3c4faf48eaa98c3cbf88d585e2
I20260202 02:52:28.501820 714 leader_election.cc:304] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:28.502066 2151 raft_consensus.cc:2804] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260202 02:52:28.502136 2151 raft_consensus.cc:493] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election (detected failure of leader b1f05820b6034228bdad884cc348716d)
I20260202 02:52:28.502197 2151 raft_consensus.cc:3060] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:28.502338 1882 ts_tablet_manager.cc:1950] T f93e8d3c4faf48eaa98c3cbf88d585e2 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:28.503319 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 384) took 5060 ms. Trace:
I20260202 02:52:28.503409 1882 rpcz_store.cc:276] 0202 02:52:23.444927 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.445009 (+ 82us) service_pool.cc:225] Handling call
0202 02:52:28.222970 (+4777961us) ts_tablet_manager.cc:1151] Deleting tablet f93e8d3c4faf48eaa98c3cbf88d585e2
0202 02:52:28.222994 (+ 24us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.410868 (+187874us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.501119 (+ 90251us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.503298 (+ 2179us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4776196}
I20260202 02:52:28.503726 1882 tablet_replica.cc:333] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:28.503901 1882 raft_consensus.cc:2243] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:28.503978 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet f93e8d3c4faf48eaa98c3cbf88d585e2 (table default.Table_2ebdd65f856c4fd1881a573bcd3c46d4 [id=84a13cdc74684fc1ade2eb2d0a7f3c86]) successfully deleted
I20260202 02:52:28.504231 1882 raft_consensus.cc:2272] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:28.504660 714 leader_election.cc:343] T ffd80badce1048f3ac95366cad05ae53 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: ffd80badce1048f3ac95366cad05ae53
I20260202 02:52:28.504789 714 leader_election.cc:304] T ffd80badce1048f3ac95366cad05ae53 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:28.504829 1882 ts_tablet_manager.cc:1916] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:28.505002 2150 raft_consensus.cc:2749] T ffd80badce1048f3ac95366cad05ae53 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260202 02:52:28.505229 714 leader_election.cc:343] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: c25e329023c04f609d8a743b4f7a95a4
W20260202 02:52:28.505461 714 leader_election.cc:343] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 08d8d781bb4e4acf915f11afb4fb2e22
W20260202 02:52:28.505587 714 leader_election.cc:343] T 75b97d7e848241cf8d894b0ae92a8f8f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 75b97d7e848241cf8d894b0ae92a8f8f
W20260202 02:52:28.505930 714 leader_election.cc:343] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 08d8d781bb4e4acf915f11afb4fb2e22
I20260202 02:52:28.506381 2151 raft_consensus.cc:515] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.507092 2151 leader_election.cc:290] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:28.507298 2173 tablet.cc:1722] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:28.507798 885 tablet_service.cc:1505] Processing CreateTablet for tablet b2149aff3eea4845b944f9fdc7303d2b (DEFAULT_TABLE table=default.Table_5c9207e4741746849a9cadbbc86ee35f [id=52f55002ce324313b27658ed262941d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:28.508298 773 raft_consensus.cc:1240] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Rejecting Update request from peer b1f05820b6034228bdad884cc348716d for earlier term 1. Current term is 2. Ops: []
I20260202 02:52:28.509320 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2149aff3eea4845b944f9fdc7303d2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.511858 2038 consensus_queue.cc:1059] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260202 02:52:28.512127 2038 raft_consensus.cc:3055] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Stepping down as leader of term 1
I20260202 02:52:28.512197 2038 raft_consensus.cc:740] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:28.512487 2038 consensus_queue.cc:260] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.512748 2038 raft_consensus.cc:3060] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:28.517964 1173 tablet_bootstrap.cc:654] T fe9ea4c0502f4976b440465992f2ea2c P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:28.521944 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35b3b4cdb6f646d5b28fae360ce82dad. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:28.499686 857 leader_election.cc:343] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:28.531486 857 leader_election.cc:304] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:28.531849 2170 raft_consensus.cc:2749] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:28.538282 2160 tablet.cc:1722] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:28.542415 1882 ts_tablet_manager.cc:1929] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260202 02:52:28.544500 715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 157090us
W20260202 02:52:28.548686 714 leader_election.cc:343] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: c25e329023c04f609d8a743b4f7a95a4
I20260202 02:52:28.552304 1882 log.cc:1199] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/0186572e7bb9499a932a636ab280f544
I20260202 02:52:28.552814 1882 ts_tablet_manager.cc:1950] T 0186572e7bb9499a932a636ab280f544 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:28.553189 1173 tablet_bootstrap.cc:492] T fe9ea4c0502f4976b440465992f2ea2c P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:28.553447 1173 ts_tablet_manager.cc:1403] T fe9ea4c0502f4976b440465992f2ea2c P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.088s user 0.002s sys 0.002s
I20260202 02:52:28.553663 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 386) took 5107 ms. Trace:
I20260202 02:52:28.554162 1173 raft_consensus.cc:359] T fe9ea4c0502f4976b440465992f2ea2c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:28.554267 1882 rpcz_store.cc:276] 0202 02:52:23.446418 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.446475 (+ 57us) service_pool.cc:225] Handling call
0202 02:52:28.503681 (+5057206us) ts_tablet_manager.cc:1151] Deleting tablet 0186572e7bb9499a932a636ab280f544
0202 02:52:28.503691 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.537677 (+ 33986us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.542315 (+ 4638us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.553647 (+ 11332us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5052469}
I20260202 02:52:28.555107 1882 tablet_replica.cc:333] T d67e91a022f1401ea724dff898c3b81b P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:28.555224 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 0186572e7bb9499a932a636ab280f544 (table default.Table_a15c97bb32284dcfb36cdc47921b2346 [id=dc77705cb5704fd58af1eed3edd45675]) successfully deleted
I20260202 02:52:28.555485 1882 raft_consensus.cc:2243] T d67e91a022f1401ea724dff898c3b81b P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:28.555879 1882 raft_consensus.cc:2272] T d67e91a022f1401ea724dff898c3b81b P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:28.556679 1882 ts_tablet_manager.cc:1916] T d67e91a022f1401ea724dff898c3b81b P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:28.554404 1173 raft_consensus.cc:385] T fe9ea4c0502f4976b440465992f2ea2c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:28.557982 1173 raft_consensus.cc:740] T fe9ea4c0502f4976b440465992f2ea2c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:28.558249 1173 consensus_queue.cc:260] T fe9ea4c0502f4976b440465992f2ea2c P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:28.558977 1173 ts_tablet_manager.cc:1434] T fe9ea4c0502f4976b440465992f2ea2c P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.005s user 0.000s sys 0.001s
I20260202 02:52:28.559273 1173 tablet_bootstrap.cc:492] T 26eeef6eebbf4f2cb44f69255fb37594 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:28.560065 920 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28a900bcaa8b41718a285e595886d805" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:28.561540 1058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "08d8d781bb4e4acf915f11afb4fb2e22" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:28.561728 1058 raft_consensus.cc:3060] T 08d8d781bb4e4acf915f11afb4fb2e22 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:28.564776 1058 raft_consensus.cc:2468] T 08d8d781bb4e4acf915f11afb4fb2e22 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:28.570487 1174 tablet_bootstrap.cc:492] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:28.570639 1174 ts_tablet_manager.cc:1403] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.112s user 0.003s sys 0.007s
I20260202 02:52:28.571390 1174 raft_consensus.cc:359] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.571568 1174 raft_consensus.cc:385] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:28.571616 1174 raft_consensus.cc:740] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:28.572796 1174 consensus_queue.cc:260] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.572899 520 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 4ec86e245f8c4d9e87c3ee3409a164b3
I20260202 02:52:28.573771 526 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 83d198d90a204c6396b569fc1b65abef
I20260202 02:52:28.574234 1174 ts_tablet_manager.cc:1434] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260202 02:52:28.574267 526 master_service.cc:954] ReplaceTablet: received request to replace tablet 6a3dac5cd1ce47488a50b4c32eca67c7 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:28.574496 1174 tablet_bootstrap.cc:492] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:28.575582 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet 83d198d90a204c6396b569fc1b65abef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:28 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:28.575878 1174 tablet_bootstrap.cc:654] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:28.576884 527 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 47edd9f65ef343b9a3be74c30708daca
I20260202 02:52:28.579377 522 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet b05008bde58d47f7adaaf35ac5949bdc
I20260202 02:52:28.579730 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c25e329023c04f609d8a743b4f7a95a4" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:28.579926 1062 raft_consensus.cc:3060] T c25e329023c04f609d8a743b4f7a95a4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:28.581027 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet 47edd9f65ef343b9a3be74c30708daca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:28 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:28.581703 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet b05008bde58d47f7adaaf35ac5949bdc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:28 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:28.582828 1062 raft_consensus.cc:2468] T c25e329023c04f609d8a743b4f7a95a4 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:28.583091 744 tablet_service.cc:1552] Processing DeleteTablet for tablet 47edd9f65ef343b9a3be74c30708daca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:28 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:28.583171 742 tablet_service.cc:1552] Processing DeleteTablet for tablet 83d198d90a204c6396b569fc1b65abef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:28 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:28.583137 752 tablet_service.cc:1552] Processing DeleteTablet for tablet b05008bde58d47f7adaaf35ac5949bdc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:28 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:28.584061 759 tablet_service.cc:1552] Processing DeleteTablet for tablet 4ec86e245f8c4d9e87c3ee3409a164b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:28 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:28.583781 1058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28a900bcaa8b41718a285e595886d805" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:28.585434 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 47edd9f65ef343b9a3be74c30708daca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:28 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:28.585646 896 tablet_service.cc:1552] Processing DeleteTablet for tablet b05008bde58d47f7adaaf35ac5949bdc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:28 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:28.585379 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 83d198d90a204c6396b569fc1b65abef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:28 UTC) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:28.586159 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 47edd9f65ef343b9a3be74c30708daca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 47edd9f65ef343b9a3be74c30708daca already in progress: creating tablet
I20260202 02:52:28.586970 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_d2f92c0892494da18afb7e3336faec7a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:28.587289 1058 raft_consensus.cc:2468] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 2.
W20260202 02:52:28.587653 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d2f92c0892494da18afb7e3336faec7a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:28.588912 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_edaccffa45e248b587c41ba28dda9dbc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:28.589126 715 leader_election.cc:343] T 75b97d7e848241cf8d894b0ae92a8f8f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Not found: Tablet not found: 75b97d7e848241cf8d894b0ae92a8f8f
I20260202 02:52:28.589244 715 leader_election.cc:304] T 75b97d7e848241cf8d894b0ae92a8f8f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
W20260202 02:52:28.589406 715 raft_consensus.cc:2686] T 75b97d7e848241cf8d894b0ae92a8f8f P 1f442ef85029461091de847118c34c0d: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260202 02:52:28.589426 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_edaccffa45e248b587c41ba28dda9dbc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:28.589649 715 leader_election.cc:304] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:28.591634 2151 raft_consensus.cc:2804] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:28.591758 2151 raft_consensus.cc:697] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:28.591997 2151 consensus_queue.cc:237] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.594472 1175 tablet_bootstrap.cc:492] T 550895b4010f4088b2cc34acff50e03f P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:28.594626 1175 ts_tablet_manager.cc:1403] T 550895b4010f4088b2cc34acff50e03f P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.160s user 0.005s sys 0.001s
I20260202 02:52:28.594910 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_65aa136d4433457aa4321915254761dc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:28.575582 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 4ec86e245f8c4d9e87c3ee3409a164b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:28 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:28.595340 1175 raft_consensus.cc:359] T 550895b4010f4088b2cc34acff50e03f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.595480 1175 raft_consensus.cc:385] T 550895b4010f4088b2cc34acff50e03f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:28.595521 1175 raft_consensus.cc:740] T 550895b4010f4088b2cc34acff50e03f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
W20260202 02:52:28.595577 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_65aa136d4433457aa4321915254761dc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:28.595724 715 leader_election.cc:304] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:28.595697 1175 consensus_queue.cc:260] T 550895b4010f4088b2cc34acff50e03f P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.596168 1175 ts_tablet_manager.cc:1434] T 550895b4010f4088b2cc34acff50e03f P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260202 02:52:28.596428 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 4ec86e245f8c4d9e87c3ee3409a164b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:28 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:28.597164 1180 ts_tablet_manager.cc:1929] T 75b97d7e848241cf8d894b0ae92a8f8f P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:28.597497 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_7a09e5be1e824c82bfd2d252cbd1043e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:28.598156 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7a09e5be1e824c82bfd2d252cbd1043e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:28.601421 1882 ts_tablet_manager.cc:1929] T d67e91a022f1401ea724dff898c3b81b P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:28.602624 519 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet 28a900bcaa8b41718a285e595886d805 (table default.Table_911bb209b4554ee8b9bc09edaf488d63 [id=cd07f9746ec94d9ebe956a7ca0d19b1c]). Expected version 1 got 0
W20260202 02:52:28.606298 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet b05008bde58d47f7adaaf35ac5949bdc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b05008bde58d47f7adaaf35ac5949bdc already in progress: creating tablet
I20260202 02:52:28.608292 1882 log.cc:1199] T d67e91a022f1401ea724dff898c3b81b P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/d67e91a022f1401ea724dff898c3b81b
I20260202 02:52:28.608577 1175 tablet_bootstrap.cc:492] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:28.608872 1882 ts_tablet_manager.cc:1950] T d67e91a022f1401ea724dff898c3b81b P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
W20260202 02:52:28.609004 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 83d198d90a204c6396b569fc1b65abef because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 83d198d90a204c6396b569fc1b65abef already in progress: creating tablet
I20260202 02:52:28.609687 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 387) took 5159 ms. Trace:
I20260202 02:52:28.609997 1175 tablet_bootstrap.cc:654] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:28.610129 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 4ec86e245f8c4d9e87c3ee3409a164b3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4ec86e245f8c4d9e87c3ee3409a164b3 already in progress: creating tablet
I20260202 02:52:28.609827 1882 rpcz_store.cc:276] 0202 02:52:23.449756 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.449839 (+ 83us) service_pool.cc:225] Handling call
0202 02:52:28.555059 (+5105220us) ts_tablet_manager.cc:1151] Deleting tablet d67e91a022f1401ea724dff898c3b81b
0202 02:52:28.555070 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.571823 (+ 16753us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.601311 (+ 29488us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.609665 (+ 8354us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5103073}
I20260202 02:52:28.610538 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet d67e91a022f1401ea724dff898c3b81b (table default.Table_c245b83c1987454fabbe8646d14252d6 [id=08498a654b6a4a9ca3f968d70daad9b6]) successfully deleted
I20260202 02:52:28.610597 1882 tablet_replica.cc:333] T b428356fba0c4fb5a6033c5d39740cc5 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:28.610761 1882 raft_consensus.cc:2243] T b428356fba0c4fb5a6033c5d39740cc5 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:28.610965 1882 raft_consensus.cc:2272] T b428356fba0c4fb5a6033c5d39740cc5 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:28.611532 1882 ts_tablet_manager.cc:1916] T b428356fba0c4fb5a6033c5d39740cc5 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:28.611958 1173 tablet_bootstrap.cc:654] T 26eeef6eebbf4f2cb44f69255fb37594 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:28.614704 923 raft_consensus.cc:1275] T af5aa68e47bb420eac300754d274fa73 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Refusing update from remote peer b1f05820b6034228bdad884cc348716d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260202 02:52:28.600725 2150 raft_consensus.cc:2804] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Leader election won for term 2
I20260202 02:52:28.616006 2150 raft_consensus.cc:697] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d [term 2 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
W20260202 02:52:28.615772 1906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
I20260202 02:52:28.616274 2150 consensus_queue.cc:237] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:28.616727 1906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
I20260202 02:52:28.617954 1180 log.cc:1199] T 75b97d7e848241cf8d894b0ae92a8f8f P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/75b97d7e848241cf8d894b0ae92a8f8f
I20260202 02:52:28.618383 1180 ts_tablet_manager.cc:1950] T 75b97d7e848241cf8d894b0ae92a8f8f P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:28.619225 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 416) took 5115 ms. Trace:
I20260202 02:52:28.608690 772 raft_consensus.cc:1275] T af5aa68e47bb420eac300754d274fa73 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Refusing update from remote peer b1f05820b6034228bdad884cc348716d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260202 02:52:28.619333 714 consensus_peers.cc:597] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b8ba9dd450948ada395207ae1c5b325. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:28.608194 715 leader_election.cc:304] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:28.620042 2126 consensus_queue.cc:1048] T af5aa68e47bb420eac300754d274fa73 P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:28.620474 2189 raft_consensus.cc:2804] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:28.620700 2189 raft_consensus.cc:697] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
W20260202 02:52:28.620832 1906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260202 02:52:28.620967 2189 consensus_queue.cc:237] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.624513 2126 consensus_queue.cc:1048] T af5aa68e47bb420eac300754d274fa73 P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:28.626081 1882 ts_tablet_manager.cc:1929] T b428356fba0c4fb5a6033c5d39740cc5 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:28.627226 1882 log.cc:1199] T b428356fba0c4fb5a6033c5d39740cc5 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/b428356fba0c4fb5a6033c5d39740cc5
I20260202 02:52:28.628708 1882 ts_tablet_manager.cc:1950] T b428356fba0c4fb5a6033c5d39740cc5 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:28.629634 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 385) took 5183 ms. Trace:
I20260202 02:52:28.629917 1175 tablet_bootstrap.cc:492] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:28.630066 1175 ts_tablet_manager.cc:1403] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.000s
I20260202 02:52:28.631397 1882 rpcz_store.cc:276] 0202 02:52:23.446022 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.446117 (+ 95us) service_pool.cc:225] Handling call
0202 02:52:28.610547 (+5164430us) ts_tablet_manager.cc:1151] Deleting tablet b428356fba0c4fb5a6033c5d39740cc5
0202 02:52:28.610557 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.617477 (+ 6920us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.625968 (+ 8491us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.629613 (+ 3645us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5150493}
I20260202 02:52:28.632016 1174 tablet_bootstrap.cc:492] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:28.632156 1174 ts_tablet_manager.cc:1403] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.058s user 0.000s sys 0.003s
I20260202 02:52:28.632196 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet b428356fba0c4fb5a6033c5d39740cc5 (table default.Table_c37ce6d5cf70417c83a68c31d02881ef [id=e87761e6b64a4476a05fcbcf73f0a2f1]) successfully deleted
I20260202 02:52:28.632658 1882 tablet_replica.cc:333] T 929eb48881b141718873d0e359e768a8 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:28.632928 1174 raft_consensus.cc:359] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:28.633091 1174 raft_consensus.cc:385] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:28.633138 1174 raft_consensus.cc:740] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:28.633366 1882 raft_consensus.cc:2243] T 929eb48881b141718873d0e359e768a8 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:28.633569 1882 raft_consensus.cc:2272] T 929eb48881b141718873d0e359e768a8 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:28.634485 1174 consensus_queue.cc:260] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:28.634996 1174 ts_tablet_manager.cc:1434] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260202 02:52:28.635260 1174 tablet_bootstrap.cc:492] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:28.642168 1174 tablet_bootstrap.cc:654] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:28.642735 1180 rpcz_store.cc:276] 0202 02:52:23.503670 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.503730 (+ 60us) service_pool.cc:225] Handling call
0202 02:52:28.462171 (+4958441us) ts_tablet_manager.cc:1151] Deleting tablet 75b97d7e848241cf8d894b0ae92a8f8f
0202 02:52:28.462182 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.528277 (+ 66095us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.597037 (+ 68760us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.619201 (+ 22164us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4941585}
I20260202 02:52:28.645324 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 75b97d7e848241cf8d894b0ae92a8f8f (table default.Table_5e366ca4b9744597add5ca93f7fe7850 [id=f2e865926cf5453292c34daa4af8be2a]) successfully deleted
I20260202 02:52:28.645447 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 417) took 5141 ms. Trace:
I20260202 02:52:28.645551 1180 rpcz_store.cc:276] 0202 02:52:23.503991 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.504038 (+ 47us) service_pool.cc:225] Handling call
0202 02:52:28.645327 (+5141289us) ts_tablet_manager.cc:1151] Deleting tablet f58a7b25815c4b5bac568aa11c19610c
0202 02:52:28.645338 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.645430 (+ 92us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5124691}
W20260202 02:52:28.646039 513 catalog_manager.cc:4952] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet f58a7b25815c4b5bac568aa11c19610c because the tablet was not found. No further retry: Not found: Tablet not found: f58a7b25815c4b5bac568aa11c19610c
I20260202 02:52:28.646039 1175 raft_consensus.cc:359] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.646301 1175 raft_consensus.cc:385] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:28.646361 1175 raft_consensus.cc:740] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:28.646570 1175 consensus_queue.cc:260] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.647352 1175 ts_tablet_manager.cc:1434] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.017s user 0.002s sys 0.000s
I20260202 02:52:28.647500 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 423) took 5140 ms. Trace:
I20260202 02:52:28.647742 1180 rpcz_store.cc:276] 0202 02:52:23.507478 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.510274 (+ 2796us) service_pool.cc:225] Handling call
0202 02:52:28.646658 (+5136384us) ts_tablet_manager.cc:1151] Deleting tablet 1f379a29e2454f689d68f5951837512b
0202 02:52:28.646669 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.647484 (+ 815us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5109795}
W20260202 02:52:28.648221 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 1f379a29e2454f689d68f5951837512b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1f379a29e2454f689d68f5951837512b already in progress: creating tablet
I20260202 02:52:28.648658 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 425) took 5137 ms. Trace:
I20260202 02:52:28.650472 2161 tablet.cc:1722] T af5aa68e47bb420eac300754d274fa73 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:28.649039 2173 tablet.cc:1722] T af5aa68e47bb420eac300754d274fa73 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:28.648777 1180 rpcz_store.cc:276] 0202 02:52:23.511236 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.511287 (+ 51us) service_pool.cc:225] Handling call
0202 02:52:28.648526 (+5137239us) ts_tablet_manager.cc:1151] Deleting tablet 874ca4ba4ebc4721bb4b60cba8e01841
0202 02:52:28.648536 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.648642 (+ 106us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5111602}
I20260202 02:52:28.656175 1882 ts_tablet_manager.cc:1916] T 929eb48881b141718873d0e359e768a8 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:28.656617 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 874ca4ba4ebc4721bb4b60cba8e01841 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 874ca4ba4ebc4721bb4b60cba8e01841 already in progress: creating tablet
I20260202 02:52:28.657022 1180 tablet_replica.cc:333] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:28.657024 524 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_79cf9f546b684c598c9447efe6997ca7" } new_table_name: "default.Table_0eedfe3d0e4c4ba08086d0cf45e438ae" modify_external_catalogs: true
I20260202 02:52:28.657255 1180 raft_consensus.cc:2243] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:28.657433 1180 raft_consensus.cc:2272] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:28.657925 1180 ts_tablet_manager.cc:1916] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:28.659886 525 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_d9237a0573e5464a995da7c7bafc08a6" } new_table_name: "default.Table_7ef2d38342484a3fb1cca1ced25130a2" modify_external_catalogs: true
I20260202 02:52:28.660449 1175 tablet_bootstrap.cc:492] T a0b5d82e5fb94b60b4aa324b978a5378 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:28.661818 1175 tablet_bootstrap.cc:654] T a0b5d82e5fb94b60b4aa324b978a5378 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:28.662665 527 master_service.cc:954] ReplaceTablet: received request to replace tablet 940e656cd34e4e53896fda6d38132b6e from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:28.664855 522 master_service.cc:954] ReplaceTablet: received request to replace tablet b4a4c75a906f40beb896bf6fff705ae6 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:28.668416 1180 ts_tablet_manager.cc:1929] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
W20260202 02:52:28.671367 1906 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260202 02:52:28.671499 1906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260202 02:52:28.671598 1906 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260202 02:52:28.671666 1906 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260202 02:52:28.672607 752 tablet_service.cc:1505] Processing CreateTablet for tablet 57df8eb7aa164138845a453391e091da (DEFAULT_TABLE table=default.Table_7d832c4a3a854a09909c552c8f6229a8 [id=f4a659e2315743879f343a6cbb17020b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:28.672545 750 tablet_service.cc:1505] Processing CreateTablet for tablet 5ec7d5c6cf834f8190437948f5ff2b8e (DEFAULT_TABLE table=default.Table_c826a60439364f9eb3cbcd4a5cd99527 [id=968501dc0afb40ae97b714bd3f7cb727]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:28.673331 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57df8eb7aa164138845a453391e091da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.673359 1180 log.cc:1199] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/938813ee0a8a40b6bf0e537e3ec1a0d4
I20260202 02:52:28.673692 1180 ts_tablet_manager.cc:1950] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:28.673954 523 catalog_manager.cc:5654] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d reported cstate change: term changed from 0 to 1, leader changed from <none> to b1f05820b6034228bdad884cc348716d (127.0.104.131). New cstate: current_term: 1 leader_uuid: "b1f05820b6034228bdad884cc348716d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } }
I20260202 02:52:28.674365 523 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 16b47a2831f54d809ed0b16deaec7e3d (table default.Table_e4018d1c57714a24be5d3a20c1ccaef4 [id=1e291b35a6764638acfb781bb4e6adb3]). Expected version 2 got 0
I20260202 02:52:28.674397 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 426) took 5162 ms. Trace:
I20260202 02:52:28.674477 1180 rpcz_store.cc:276] 0202 02:52:23.511512 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.511552 (+ 40us) service_pool.cc:225] Handling call
0202 02:52:28.656959 (+5145407us) ts_tablet_manager.cc:1151] Deleting tablet 938813ee0a8a40b6bf0e537e3ec1a0d4
0202 02:52:28.656969 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.663601 (+ 6632us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.668268 (+ 4667us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.674378 (+ 6110us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5119988}
I20260202 02:52:28.675540 1175 tablet_bootstrap.cc:492] T a0b5d82e5fb94b60b4aa324b978a5378 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:28.675540 1882 ts_tablet_manager.cc:1929] T 929eb48881b141718873d0e359e768a8 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:28.675652 1882 log.cc:1199] T 929eb48881b141718873d0e359e768a8 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/929eb48881b141718873d0e359e768a8
I20260202 02:52:28.675727 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 938813ee0a8a40b6bf0e537e3ec1a0d4 (table default.Table_cafbdde19f984e0fa22b1fde486f8e7e [id=1229c6d37b994a669d3363fdf6f8c6c8]) successfully deleted
I20260202 02:52:28.675969 521 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_026b5574b3af4ff08f92f7ef3b3025ff [id=255f4d94812f45d0b4423150098460ff] alter complete (version 2)
I20260202 02:52:28.676023 1882 ts_tablet_manager.cc:1950] T 929eb48881b141718873d0e359e768a8 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:28.676379 521 master_service.cc:954] ReplaceTablet: received request to replace tablet ac2ab12f5040488aa306cb69b107b485 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:28.676755 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 388) took 5220 ms. Trace:
I20260202 02:52:28.677809 1042 tablet_service.cc:1505] Processing CreateTablet for tablet aed1081136f7431b9b14784c21112d1c (DEFAULT_TABLE table=default.Table_5db99844c1bb484a863a9e289a19da3e [id=f1f6c97fc88340da8bab0347ad6430ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:28.678004 1041 tablet_service.cc:1505] Processing CreateTablet for tablet 57df8eb7aa164138845a453391e091da (DEFAULT_TABLE table=default.Table_7d832c4a3a854a09909c552c8f6229a8 [id=f4a659e2315743879f343a6cbb17020b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:28.678197 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aed1081136f7431b9b14784c21112d1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.678407 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57df8eb7aa164138845a453391e091da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.677809 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 5ec7d5c6cf834f8190437948f5ff2b8e (DEFAULT_TABLE table=default.Table_c826a60439364f9eb3cbcd4a5cd99527 [id=968501dc0afb40ae97b714bd3f7cb727]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:28.678839 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ec7d5c6cf834f8190437948f5ff2b8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.682147 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ec7d5c6cf834f8190437948f5ff2b8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.682543 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 427) took 5170 ms. Trace:
I20260202 02:52:28.682614 1180 rpcz_store.cc:276] 0202 02:52:23.511766 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.511811 (+ 45us) service_pool.cc:225] Handling call
0202 02:52:28.682409 (+5170598us) ts_tablet_manager.cc:1151] Deleting tablet e95153ee8f2d40de955e3389d3a58644
0202 02:52:28.682419 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.682527 (+ 108us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5140683}
W20260202 02:52:28.683051 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet e95153ee8f2d40de955e3389d3a58644 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e95153ee8f2d40de955e3389d3a58644 already in progress: creating tablet
I20260202 02:52:28.683332 1175 ts_tablet_manager.cc:1403] T a0b5d82e5fb94b60b4aa324b978a5378 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.036s user 0.000s sys 0.004s
I20260202 02:52:28.684085 1175 raft_consensus.cc:359] T a0b5d82e5fb94b60b4aa324b978a5378 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.684237 1175 raft_consensus.cc:385] T a0b5d82e5fb94b60b4aa324b978a5378 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:28.684279 1175 raft_consensus.cc:740] T a0b5d82e5fb94b60b4aa324b978a5378 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:28.684801 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 414) took 5181 ms. Trace:
I20260202 02:52:28.684935 1039 tablet_service.cc:1505] Processing CreateTablet for tablet f50d75950af14559a68d0d6be871f23c (DEFAULT_TABLE table=default.Table_56abcf537bc74585bad14f2d06769ccf [id=277be7d2bf36417f81622888421f6283]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:28.684877 1180 rpcz_store.cc:276] 0202 02:52:23.503177 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.547877 (+ 44700us) service_pool.cc:225] Handling call
0202 02:52:28.684677 (+5136800us) ts_tablet_manager.cc:1151] Deleting tablet 07580ad1d049475a9645acfad0b1d219
0202 02:52:28.684687 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.684782 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5086904}
I20260202 02:52:28.685671 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f50d75950af14559a68d0d6be871f23c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:28.685654 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 07580ad1d049475a9645acfad0b1d219 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 07580ad1d049475a9645acfad0b1d219 already in progress: creating tablet
I20260202 02:52:28.686034 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 428) took 5025 ms. Trace:
I20260202 02:52:28.686100 1180 rpcz_store.cc:276] 0202 02:52:23.660544 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.660601 (+ 57us) service_pool.cc:225] Handling call
0202 02:52:28.685895 (+5025294us) ts_tablet_manager.cc:1151] Deleting tablet 33150e1456144b8e9c64cc2a2ac6dcca
0202 02:52:28.685904 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.686019 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5025026}
W20260202 02:52:28.686497 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 33150e1456144b8e9c64cc2a2ac6dcca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 33150e1456144b8e9c64cc2a2ac6dcca already in progress: creating tablet
I20260202 02:52:28.684484 1175 consensus_queue.cc:260] T a0b5d82e5fb94b60b4aa324b978a5378 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.687111 1175 ts_tablet_manager.cc:1434] T a0b5d82e5fb94b60b4aa324b978a5378 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260202 02:52:28.687731 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 430) took 5019 ms. Trace:
I20260202 02:52:28.687810 1180 rpcz_store.cc:276] 0202 02:52:23.668227 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.668272 (+ 45us) service_pool.cc:225] Handling call
0202 02:52:28.687614 (+5019342us) ts_tablet_manager.cc:1151] Deleting tablet b4a4c75a906f40beb896bf6fff705ae6
0202 02:52:28.687624 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.687714 (+ 90us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5019101}
W20260202 02:52:28.688225 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet b4a4c75a906f40beb896bf6fff705ae6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b4a4c75a906f40beb896bf6fff705ae6 already in progress: creating tablet
I20260202 02:52:28.688534 1180 tablet_replica.cc:333] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:28.688730 1180 raft_consensus.cc:2243] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:28.676890 1882 rpcz_store.cc:276] 0202 02:52:23.456784 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.456851 (+ 67us) service_pool.cc:225] Handling call
0202 02:52:28.632601 (+5175750us) ts_tablet_manager.cc:1151] Deleting tablet 929eb48881b141718873d0e359e768a8
0202 02:52:28.632614 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.665996 (+ 33382us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.675334 (+ 9338us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.676736 (+ 1402us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5161589}
I20260202 02:52:28.689072 1180 raft_consensus.cc:2272] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:28.689375 742 tablet_service.cc:1505] Processing CreateTablet for tablet aed1081136f7431b9b14784c21112d1c (DEFAULT_TABLE table=default.Table_5db99844c1bb484a863a9e289a19da3e [id=f1f6c97fc88340da8bab0347ad6430ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:28.689708 1175 tablet_bootstrap.cc:492] T 2c31b45dc8004eae993e972e9faa3ff8 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:28.690105 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aed1081136f7431b9b14784c21112d1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.691010 1175 tablet_bootstrap.cc:654] T 2c31b45dc8004eae993e972e9faa3ff8 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:28.689742 1882 tablet_replica.cc:333] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:28.691279 1882 raft_consensus.cc:2243] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [term 2 LEADER]: Raft consensus shutting down.
I20260202 02:52:28.691541 1882 pending_rounds.cc:70] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d: Trying to abort 1 pending ops.
I20260202 02:52:28.691633 1882 pending_rounds.cc:77] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d: Aborting op as it isn't in flight: id { term: 2 index: 1 } timestamp: 7249923065665110016 op_type: NO_OP noop_request { }
I20260202 02:52:28.691794 1882 raft_consensus.cc:2889] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:28.691937 1882 raft_consensus.cc:2272] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:28.692203 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 929eb48881b141718873d0e359e768a8 (table default.Table_b0e344dc1a824cd9b41aa8991b7911dd [id=8e640d1261014706a067f522bda7ea61]) successfully deleted
I20260202 02:52:28.692997 1174 tablet_bootstrap.cc:492] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:28.693131 1174 ts_tablet_manager.cc:1403] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.058s user 0.003s sys 0.001s
I20260202 02:52:28.693478 1882 ts_tablet_manager.cc:1916] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:28.693902 1174 raft_consensus.cc:359] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.694073 1174 raft_consensus.cc:385] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:28.694128 1174 raft_consensus.cc:740] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:28.694336 1174 consensus_queue.cc:260] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.694809 1174 ts_tablet_manager.cc:1434] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:28.695089 1174 tablet_bootstrap.cc:492] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:28.695358 759 tablet_service.cc:1505] Processing CreateTablet for tablet f50d75950af14559a68d0d6be871f23c (DEFAULT_TABLE table=default.Table_56abcf537bc74585bad14f2d06769ccf [id=277be7d2bf36417f81622888421f6283]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:28.695797 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f50d75950af14559a68d0d6be871f23c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.696760 1174 tablet_bootstrap.cc:654] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:28.697639 1721 raft_consensus.cc:493] T fe9ea4c0502f4976b440465992f2ea2c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:28.697774 1721 raft_consensus.cc:515] T fe9ea4c0502f4976b440465992f2ea2c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:28.698364 1721 leader_election.cc:290] T fe9ea4c0502f4976b440465992f2ea2c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:28.699054 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fe9ea4c0502f4976b440465992f2ea2c" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:28.699537 998 leader_election.cc:343] T fe9ea4c0502f4976b440465992f2ea2c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:28.708005 522 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_df0a1db6c8b048938a0bbd425a0a65dd" } new_table_name: "default.Table_0c357b5eb1e14dbba252f4edfc630705" modify_external_catalogs: true
I20260202 02:52:28.712352 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_881157f1bd2945a5be1340a99a941bef" } new_table_name: "default.Table_971daa4f561047c5a129d2937ba0cd56" modify_external_catalogs: true
I20260202 02:52:28.712455 1882 ts_tablet_manager.cc:1929] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260202 02:52:28.712559 1882 log.cc:1199] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/94bada52db6a47179415d302b541d01f
I20260202 02:52:28.712921 1882 ts_tablet_manager.cc:1950] T 94bada52db6a47179415d302b541d01f P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:28.713716 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 383) took 5340 ms. Trace:
I20260202 02:52:28.713801 1882 rpcz_store.cc:276] 0202 02:52:23.373348 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.373898 (+ 550us) service_pool.cc:225] Handling call
0202 02:52:28.689692 (+5315794us) ts_tablet_manager.cc:1151] Deleting tablet 94bada52db6a47179415d302b541d01f
0202 02:52:28.689703 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.703744 (+ 14041us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.712289 (+ 8545us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.713692 (+ 1403us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5172559}
I20260202 02:52:28.714303 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 94bada52db6a47179415d302b541d01f (table default.Table_086abcdd67e94b38ac7f5d98777f5c6b [id=32fe1d8734de41cfae07a4662814987e]) successfully deleted
I20260202 02:52:28.714709 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 397) took 5157 ms. Trace:
I20260202 02:52:28.714829 1882 rpcz_store.cc:276] 0202 02:52:23.557013 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.557062 (+ 49us) service_pool.cc:225] Handling call
0202 02:52:28.714556 (+5157494us) ts_tablet_manager.cc:1151] Deleting tablet 07580ad1d049475a9645acfad0b1d219
0202 02:52:28.714569 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.714693 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5157288}
I20260202 02:52:28.715304 1175 tablet_bootstrap.cc:492] T 2c31b45dc8004eae993e972e9faa3ff8 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
W20260202 02:52:28.715396 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 07580ad1d049475a9645acfad0b1d219 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 07580ad1d049475a9645acfad0b1d219 already in progress: creating tablet
I20260202 02:52:28.715514 1175 ts_tablet_manager.cc:1403] T 2c31b45dc8004eae993e972e9faa3ff8 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.028s user 0.010s sys 0.004s
I20260202 02:52:28.715811 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 402) took 5149 ms. Trace:
I20260202 02:52:28.715912 1882 rpcz_store.cc:276] 0202 02:52:23.565903 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.566053 (+ 150us) service_pool.cc:225] Handling call
0202 02:52:28.715668 (+5149615us) ts_tablet_manager.cc:1151] Deleting tablet 1f379a29e2454f689d68f5951837512b
0202 02:52:28.715677 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.715794 (+ 117us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5149221}
W20260202 02:52:28.716379 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 1f379a29e2454f689d68f5951837512b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1f379a29e2454f689d68f5951837512b already in progress: creating tablet
I20260202 02:52:28.716833 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 405) took 5144 ms. Trace:
I20260202 02:52:28.716816 521 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_f0ad9b13e025411d844149bea0d37e2f" } modify_external_catalogs: true
I20260202 02:52:28.716928 1882 rpcz_store.cc:276] 0202 02:52:23.572719 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.614128 (+ 41409us) service_pool.cc:225] Handling call
0202 02:52:28.716733 (+5102605us) ts_tablet_manager.cc:1151] Deleting tablet 874ca4ba4ebc4721bb4b60cba8e01841
0202 02:52:28.716742 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.716818 (+ 76us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5102056}
I20260202 02:52:28.717020 521 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_f0ad9b13e025411d844149bea0d37e2f" } modify_external_catalogs: true
W20260202 02:52:28.717331 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 874ca4ba4ebc4721bb4b60cba8e01841 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 874ca4ba4ebc4721bb4b60cba8e01841 already in progress: creating tablet
I20260202 02:52:28.717597 1882 tablet_replica.cc:333] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:28.717767 1882 raft_consensus.cc:2243] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:28.717937 1882 raft_consensus.cc:2272] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:28.718434 1882 ts_tablet_manager.cc:1916] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:28.719005 1175 raft_consensus.cc:359] T 2c31b45dc8004eae993e972e9faa3ff8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:28.719187 1175 raft_consensus.cc:385] T 2c31b45dc8004eae993e972e9faa3ff8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:28.719241 1175 raft_consensus.cc:740] T 2c31b45dc8004eae993e972e9faa3ff8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:28.719442 1175 consensus_queue.cc:260] T 2c31b45dc8004eae993e972e9faa3ff8 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:28.689591 1180 ts_tablet_manager.cc:1916] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:28.719884 1175 ts_tablet_manager.cc:1434] T 2c31b45dc8004eae993e972e9faa3ff8 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.004s user 0.000s sys 0.000s
I20260202 02:52:28.720160 1175 tablet_bootstrap.cc:492] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:28.720517 1174 tablet_bootstrap.cc:492] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:28.720902 1174 ts_tablet_manager.cc:1403] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.000s
I20260202 02:52:28.721714 1174 raft_consensus.cc:359] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.721874 1174 raft_consensus.cc:385] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:28.721936 1174 raft_consensus.cc:740] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:28.722043 1175 tablet_bootstrap.cc:654] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:28.722137 1174 consensus_queue.cc:260] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.722589 1174 ts_tablet_manager.cc:1434] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260202 02:52:28.726441 885 tablet_service.cc:1505] Processing CreateTablet for tablet 5ec7d5c6cf834f8190437948f5ff2b8e (DEFAULT_TABLE table=default.Table_c826a60439364f9eb3cbcd4a5cd99527 [id=968501dc0afb40ae97b714bd3f7cb727]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:28.726828 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ec7d5c6cf834f8190437948f5ff2b8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.733337 1175 tablet_bootstrap.cc:492] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:28.733570 1175 ts_tablet_manager.cc:1403] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.006s
I20260202 02:52:28.734277 1175 raft_consensus.cc:359] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.734457 1175 raft_consensus.cc:385] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:28.734519 1175 raft_consensus.cc:740] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:28.734398 885 tablet_service.cc:1505] Processing CreateTablet for tablet f50d75950af14559a68d0d6be871f23c (DEFAULT_TABLE table=default.Table_56abcf537bc74585bad14f2d06769ccf [id=277be7d2bf36417f81622888421f6283]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:28.734730 1175 consensus_queue.cc:260] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.735031 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f50d75950af14559a68d0d6be871f23c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.735181 1175 ts_tablet_manager.cc:1434] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260202 02:52:28.735447 1175 tablet_bootstrap.cc:492] T 6df3d11cf578404ca41807f0777b6c53 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:28.737278 1175 tablet_bootstrap.cc:654] T 6df3d11cf578404ca41807f0777b6c53 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:28.737290 1882 ts_tablet_manager.cc:1929] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20260202 02:52:28.737392 1882 log.cc:1199] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/938813ee0a8a40b6bf0e537e3ec1a0d4
I20260202 02:52:28.736392 1906 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:45945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-0/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.190:33981,127.0.104.189:46271,127.0.104.188:39077
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.190:33981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.104.190
--webserver_port=40497
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3722ed49c19ce54c7feae2fa944e32b1d4ed8d2d
build type FASTDEBUG
built by None at 02 Feb 2026 02:43:23 UTC on 24a791456cd2
build id 10304
ASAN enabled
I20260202 02:52:28.740006 1882 ts_tablet_manager.cc:1950] T 938813ee0a8a40b6bf0e537e3ec1a0d4 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:28.741032 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 406) took 5165 ms. Trace:
I20260202 02:52:28.741207 1882 rpcz_store.cc:276] 0202 02:52:23.575685 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.614728 (+ 39043us) service_pool.cc:225] Handling call
0202 02:52:28.717554 (+5102826us) ts_tablet_manager.cc:1151] Deleting tablet 938813ee0a8a40b6bf0e537e3ec1a0d4
0202 02:52:28.717562 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.725590 (+ 8028us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.737172 (+ 11582us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.741012 (+ 3840us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5102477}
I20260202 02:52:28.741416 2189 consensus_queue.cc:1048] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:28.741700 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 938813ee0a8a40b6bf0e537e3ec1a0d4 (table default.Table_cafbdde19f984e0fa22b1fde486f8e7e [id=1229c6d37b994a669d3363fdf6f8c6c8]) successfully deleted
I20260202 02:52:28.742240 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 407) took 5158 ms. Trace:
I20260202 02:52:28.742493 1906 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260202 02:52:28.745169 1906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20260202 02:52:28.745206 1175 tablet_bootstrap.cc:492] T 6df3d11cf578404ca41807f0777b6c53 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:28.745342 1175 ts_tablet_manager.cc:1403] T 6df3d11cf578404ca41807f0777b6c53 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.003s
I20260202 02:52:28.745375 1882 rpcz_store.cc:276] 0202 02:52:23.584299 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.615126 (+ 30827us) service_pool.cc:225] Handling call
0202 02:52:28.742115 (+5126989us) ts_tablet_manager.cc:1151] Deleting tablet e95153ee8f2d40de955e3389d3a58644
0202 02:52:28.742126 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.742223 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5126231}
W20260202 02:52:28.746071 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet e95153ee8f2d40de955e3389d3a58644 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e95153ee8f2d40de955e3389d3a58644 already in progress: creating tablet
I20260202 02:52:28.747056 1882 tablet_replica.cc:333] T bd7b0a0fcb7645fdb9edfeccb3d18268 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:28.747283 1882 raft_consensus.cc:2243] T bd7b0a0fcb7645fdb9edfeccb3d18268 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:28.747541 1882 raft_consensus.cc:2272] T bd7b0a0fcb7645fdb9edfeccb3d18268 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:28.748140 1882 ts_tablet_manager.cc:1916] T bd7b0a0fcb7645fdb9edfeccb3d18268 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:28.751044 1175 raft_consensus.cc:359] T 6df3d11cf578404ca41807f0777b6c53 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.751236 1175 raft_consensus.cc:385] T 6df3d11cf578404ca41807f0777b6c53 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:28.751287 1175 raft_consensus.cc:740] T 6df3d11cf578404ca41807f0777b6c53 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:28.751479 1175 consensus_queue.cc:260] T 6df3d11cf578404ca41807f0777b6c53 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.753717 1721 raft_consensus.cc:493] T 79f694b740954d429145fba9157338ca P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:28.753881 1721 raft_consensus.cc:515] T 79f694b740954d429145fba9157338ca P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.754550 1721 leader_election.cc:290] T 79f694b740954d429145fba9157338ca P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:28.755689 998 leader_election.cc:343] T 79f694b740954d429145fba9157338ca P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Not found: Tablet not found: 79f694b740954d429145fba9157338ca
I20260202 02:52:28.756614 1175 ts_tablet_manager.cc:1434] T 6df3d11cf578404ca41807f0777b6c53 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20260202 02:52:28.757174 1175 tablet_bootstrap.cc:492] T fbcb4a522df648ba9542250450c08f1b P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:28.757632 887 tablet_service.cc:1505] Processing CreateTablet for tablet aed1081136f7431b9b14784c21112d1c (DEFAULT_TABLE table=default.Table_5db99844c1bb484a863a9e289a19da3e [id=f1f6c97fc88340da8bab0347ad6430ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:28.758051 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aed1081136f7431b9b14784c21112d1c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:28.758551 999 leader_election.cc:343] T fe9ea4c0502f4976b440465992f2ea2c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: fe9ea4c0502f4976b440465992f2ea2c
I20260202 02:52:28.758658 999 leader_election.cc:304] T fe9ea4c0502f4976b440465992f2ea2c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:28.759234 2038 raft_consensus.cc:2749] T fe9ea4c0502f4976b440465992f2ea2c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:28.760532 999 leader_election.cc:343] T 79f694b740954d429145fba9157338ca P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 79f694b740954d429145fba9157338ca
I20260202 02:52:28.760649 999 leader_election.cc:304] T 79f694b740954d429145fba9157338ca P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:28.761132 2038 raft_consensus.cc:2749] T 79f694b740954d429145fba9157338ca P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:28.762446 1175 tablet_bootstrap.cc:654] T fbcb4a522df648ba9542250450c08f1b P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:28.762595 2197 tablet.cc:1722] T af5aa68e47bb420eac300754d274fa73 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:28.763087 1882 ts_tablet_manager.cc:1929] T bd7b0a0fcb7645fdb9edfeccb3d18268 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260202 02:52:28.763226 1882 log.cc:1199] T bd7b0a0fcb7645fdb9edfeccb3d18268 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/bd7b0a0fcb7645fdb9edfeccb3d18268
I20260202 02:52:28.764199 1882 ts_tablet_manager.cc:1950] T bd7b0a0fcb7645fdb9edfeccb3d18268 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:28.765041 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 409) took 5098 ms. Trace:
I20260202 02:52:28.765357 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_af8ce4ca28bf4200b52f066fc4c30d63"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:28.765694 1882 rpcz_store.cc:276] 0202 02:52:23.666540 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.666588 (+ 48us) service_pool.cc:225] Handling call
0202 02:52:28.747011 (+5080423us) ts_tablet_manager.cc:1151] Deleting tablet bd7b0a0fcb7645fdb9edfeccb3d18268
0202 02:52:28.747020 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.753738 (+ 6718us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.762986 (+ 9248us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.765020 (+ 2034us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4943663}
W20260202 02:52:28.766036 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_af8ce4ca28bf4200b52f066fc4c30d63 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:28.775627 1174 tablet_bootstrap.cc:492] T a1f39e5b02f747819e636cddb4292eb6 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:28.777273 885 tablet_service.cc:1505] Processing CreateTablet for tablet 57df8eb7aa164138845a453391e091da (DEFAULT_TABLE table=default.Table_7d832c4a3a854a09909c552c8f6229a8 [id=f4a659e2315743879f343a6cbb17020b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:28.777561 1174 tablet_bootstrap.cc:654] T a1f39e5b02f747819e636cddb4292eb6 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:28.777679 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57df8eb7aa164138845a453391e091da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:28.780411 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet bd7b0a0fcb7645fdb9edfeccb3d18268 (table default.Table_18fec323d6b04e10a687de6361b643b2 [id=1ffcc11561524fe8a7b09a7ddc92d5b2]) successfully deleted
I20260202 02:52:28.781563 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 408) took 5117 ms. Trace:
I20260202 02:52:28.782151 1882 rpcz_store.cc:276] 0202 02:52:23.666357 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.666979 (+ 622us) service_pool.cc:225] Handling call
0202 02:52:28.781413 (+5114434us) ts_tablet_manager.cc:1151] Deleting tablet 33150e1456144b8e9c64cc2a2ac6dcca
0202 02:52:28.781425 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.781538 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4978013}
W20260202 02:52:28.783093 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 33150e1456144b8e9c64cc2a2ac6dcca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 33150e1456144b8e9c64cc2a2ac6dcca already in progress: creating tablet
I20260202 02:52:28.784202 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 410) took 5117 ms. Trace:
I20260202 02:52:28.784754 1882 rpcz_store.cc:276] 0202 02:52:23.666929 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.667153 (+ 224us) service_pool.cc:225] Handling call
0202 02:52:28.783589 (+5116436us) ts_tablet_manager.cc:1151] Deleting tablet b4a4c75a906f40beb896bf6fff705ae6
0202 02:52:28.783600 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.784183 (+ 583us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4980143}
W20260202 02:52:28.786252 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet b4a4c75a906f40beb896bf6fff705ae6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b4a4c75a906f40beb896bf6fff705ae6 already in progress: creating tablet
I20260202 02:52:28.787187 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 411) took 4971 ms. Trace:
I20260202 02:52:28.787568 1882 rpcz_store.cc:276] 0202 02:52:23.816025 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.816086 (+ 61us) service_pool.cc:225] Handling call
0202 02:52:28.786844 (+4970758us) ts_tablet_manager.cc:1151] Deleting tablet 65bd1c1a382e42e4a68077264563c55b
0202 02:52:28.786967 (+ 123us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.787170 (+ 203us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4970554}
I20260202 02:52:28.787976 1175 tablet_bootstrap.cc:492] T fbcb4a522df648ba9542250450c08f1b P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:28.788111 1175 ts_tablet_manager.cc:1403] T fbcb4a522df648ba9542250450c08f1b P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.031s user 0.003s sys 0.001s
W20260202 02:52:28.788619 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 65bd1c1a382e42e4a68077264563c55b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 65bd1c1a382e42e4a68077264563c55b already in progress: creating tablet
I20260202 02:52:28.788964 1175 raft_consensus.cc:359] T fbcb4a522df648ba9542250450c08f1b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.789131 1175 raft_consensus.cc:385] T fbcb4a522df648ba9542250450c08f1b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:28.789183 1175 raft_consensus.cc:740] T fbcb4a522df648ba9542250450c08f1b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:28.789368 1175 consensus_queue.cc:260] T fbcb4a522df648ba9542250450c08f1b P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.789655 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 412) took 4932 ms. Trace:
I20260202 02:52:28.789835 1175 ts_tablet_manager.cc:1434] T fbcb4a522df648ba9542250450c08f1b P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:28.789937 1882 rpcz_store.cc:276] 0202 02:52:23.857381 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.857441 (+ 60us) service_pool.cc:225] Handling call
0202 02:52:28.789012 (+4931571us) ts_tablet_manager.cc:1151] Deleting tablet 7ff6d2fa085343a6b11be0898da6233d
0202 02:52:28.789023 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.789625 (+ 602us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4931341}
I20260202 02:52:28.790113 1175 tablet_bootstrap.cc:492] T ccd6bc8bd0504bf18d108c73685ef9ff P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
W20260202 02:52:28.790196 714 consensus_peers.cc:597] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 68823eff1bc04108ba83487605aec0ea. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:28.790508 714 consensus_peers.cc:597] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 042d1ebf357b4abe8814a407891ce265. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:28.790913 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 7ff6d2fa085343a6b11be0898da6233d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7ff6d2fa085343a6b11be0898da6233d already in progress: creating tablet
I20260202 02:52:28.791817 1175 tablet_bootstrap.cc:654] T ccd6bc8bd0504bf18d108c73685ef9ff P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:28.791314 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 413) took 4927 ms. Trace:
I20260202 02:52:28.791994 1882 rpcz_store.cc:276] 0202 02:52:23.864130 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.864191 (+ 61us) service_pool.cc:225] Handling call
0202 02:52:28.791190 (+4926999us) ts_tablet_manager.cc:1151] Deleting tablet b7880cb051c24bc998003f0effed3cf8
0202 02:52:28.791201 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.791299 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4926766}
W20260202 02:52:28.792528 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet b7880cb051c24bc998003f0effed3cf8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b7880cb051c24bc998003f0effed3cf8 already in progress: creating tablet
I20260202 02:52:28.793257 1882 tablet_replica.cc:333] T 5986676f5484498da927b4c6da3f47ff P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:28.794204 1882 raft_consensus.cc:2243] T 5986676f5484498da927b4c6da3f47ff P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:28.794459 1882 raft_consensus.cc:2272] T 5986676f5484498da927b4c6da3f47ff P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:28.795068 1882 ts_tablet_manager.cc:1916] T 5986676f5484498da927b4c6da3f47ff P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:28.796295 524 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bb145df67e1a44569a3dbc91c024b584 consensus state has no leader
I20260202 02:52:28.796684 524 master_service.cc:954] ReplaceTablet: received request to replace tablet 23671408fcbc454cbd918328015cf6e6 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:28.810165 524 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_005b6c9953134fc3a42de2403722bdfe" } new_table_name: "default.Table_d2348bd2d13c4d75ab57c942a75df58c" modify_external_catalogs: true
I20260202 02:52:28.815032 1180 ts_tablet_manager.cc:1929] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260202 02:52:28.816413 1173 tablet_bootstrap.cc:492] T 26eeef6eebbf4f2cb44f69255fb37594 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:28.816658 1173 ts_tablet_manager.cc:1403] T 26eeef6eebbf4f2cb44f69255fb37594 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.257s user 0.003s sys 0.002s
I20260202 02:52:28.817443 1173 raft_consensus.cc:359] T 26eeef6eebbf4f2cb44f69255fb37594 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:28.817724 1173 raft_consensus.cc:385] T 26eeef6eebbf4f2cb44f69255fb37594 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:28.817839 1173 raft_consensus.cc:740] T 26eeef6eebbf4f2cb44f69255fb37594 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:28.818096 1173 consensus_queue.cc:260] T 26eeef6eebbf4f2cb44f69255fb37594 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:28.818900 1173 ts_tablet_manager.cc:1434] T 26eeef6eebbf4f2cb44f69255fb37594 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
W20260202 02:52:28.823686 525 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 767c004aa03d493c8d2301e1a49787e7 consensus state has no leader
I20260202 02:52:28.828418 2170 raft_consensus.cc:493] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:28.828580 2170 raft_consensus.cc:515] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.829232 2170 leader_election.cc:290] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:28.829171 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_6b5c3a97133940d98d66d07f6be3d55a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:28.829836 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6b5c3a97133940d98d66d07f6be3d55a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:28.830191 2227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260202 02:52:28.830839 1173 tablet_bootstrap.cc:492] T 3d1a7578e83447669e3c3972a426ad44 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:28.832084 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1116872cc25a4ab5b1a3a6bbba1f0da7" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:28.832670 1173 tablet_bootstrap.cc:654] T 3d1a7578e83447669e3c3972a426ad44 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:28.832717 859 leader_election.cc:343] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:28.837719 2223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:52:28.838179 2224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260202 02:52:28.839391 1882 ts_tablet_manager.cc:1929] T 5986676f5484498da927b4c6da3f47ff P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260202 02:52:28.839509 1175 tablet_bootstrap.cc:492] T ccd6bc8bd0504bf18d108c73685ef9ff P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:28.839661 1175 ts_tablet_manager.cc:1403] T ccd6bc8bd0504bf18d108c73685ef9ff P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.050s user 0.019s sys 0.010s
I20260202 02:52:28.839927 1906 server_base.cc:1060] running on GCE node
I20260202 02:52:28.840066 526 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 6a3dac5cd1ce47488a50b4c32eca67c7
I20260202 02:52:28.840659 526 catalog_manager.cc:6724] ReplaceTablet: tablet 6a3dac5cd1ce47488a50b4c32eca67c7 deleted and replaced by tablet 365417e050304073b5de887082fc699b
I20260202 02:52:28.840703 1175 raft_consensus.cc:359] T ccd6bc8bd0504bf18d108c73685ef9ff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.840864 1175 raft_consensus.cc:385] T ccd6bc8bd0504bf18d108c73685ef9ff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:28.840920 1175 raft_consensus.cc:740] T ccd6bc8bd0504bf18d108c73685ef9ff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:28.841123 1175 consensus_queue.cc:260] T ccd6bc8bd0504bf18d108c73685ef9ff P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.841588 1175 ts_tablet_manager.cc:1434] T ccd6bc8bd0504bf18d108c73685ef9ff P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:28.842033 1175 tablet_bootstrap.cc:492] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:28.843264 1906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260202 02:52:28.843679 1175 tablet_bootstrap.cc:654] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:28.840924 1039 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a3dac5cd1ce47488a50b4c32eca67c7 with delete_type TABLET_DATA_DELETED (replaced by tablet 365417e050304073b5de887082fc699b) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:28.846053 1882 log.cc:1199] T 5986676f5484498da927b4c6da3f47ff P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/5986676f5484498da927b4c6da3f47ff
I20260202 02:52:28.846541 1882 ts_tablet_manager.cc:1950] T 5986676f5484498da927b4c6da3f47ff P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:28.847385 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 414) took 4968 ms. Trace:
I20260202 02:52:28.847628 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a3dac5cd1ce47488a50b4c32eca67c7 with delete_type TABLET_DATA_DELETED (replaced by tablet 365417e050304073b5de887082fc699b) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:28.847926 1882 rpcz_store.cc:276] 0202 02:52:23.878595 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.878663 (+ 68us) service_pool.cc:225] Handling call
0202 02:52:28.793207 (+4914544us) ts_tablet_manager.cc:1151] Deleting tablet 5986676f5484498da927b4c6da3f47ff
0202 02:52:28.793217 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.820624 (+ 27407us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.839258 (+ 18634us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.847365 (+ 8107us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4913614}
I20260202 02:52:28.848562 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 5986676f5484498da927b4c6da3f47ff (table default.Table_755b6ce0796b45b8be9df04e59dfe565 [id=d8485f23f63b493781bea557dbe8a0c1]) successfully deleted
I20260202 02:52:28.848668 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_d05a522f656546f9a12e61fe94ca4873"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:28.849223 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 415) took 4967 ms. Trace:
W20260202 02:52:28.849310 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d05a522f656546f9a12e61fe94ca4873 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:28.849298 1882 rpcz_store.cc:276] 0202 02:52:23.881443 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.881504 (+ 61us) service_pool.cc:225] Handling call
0202 02:52:28.849096 (+4967592us) ts_tablet_manager.cc:1151] Deleting tablet d0ce2121d65b4a228596e0ca9018883d
0202 02:52:28.849106 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.849206 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4967360}
W20260202 02:52:28.849752 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet d0ce2121d65b4a228596e0ca9018883d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d0ce2121d65b4a228596e0ca9018883d already in progress: creating tablet
I20260202 02:52:28.851050 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 418) took 4855 ms. Trace:
I20260202 02:52:28.851423 1882 rpcz_store.cc:276] 0202 02:52:23.995493 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.995550 (+ 57us) service_pool.cc:225] Handling call
0202 02:52:28.850940 (+4855390us) ts_tablet_manager.cc:1151] Deleting tablet c12ff20ca6b44ddba0c936a28cfb2eb9
0202 02:52:28.850962 (+ 22us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.851032 (+ 70us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4850184}
W20260202 02:52:28.852144 515 catalog_manager.cc:4952] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet c12ff20ca6b44ddba0c936a28cfb2eb9 because the tablet was not found. No further retry: Not found: Tablet not found: c12ff20ca6b44ddba0c936a28cfb2eb9
I20260202 02:52:28.852641 1882 tablet_replica.cc:333] T 2b8ba9dd450948ada395207ae1c5b325 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
W20260202 02:52:28.852746 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 6a3dac5cd1ce47488a50b4c32eca67c7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6a3dac5cd1ce47488a50b4c32eca67c7 already in progress: creating tablet
I20260202 02:52:28.853322 1906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260202 02:52:28.853648 1882 raft_consensus.cc:2243] T 2b8ba9dd450948ada395207ae1c5b325 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:28.854104 1882 raft_consensus.cc:2272] T 2b8ba9dd450948ada395207ae1c5b325 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:28.854480 1173 tablet_bootstrap.cc:492] T 3d1a7578e83447669e3c3972a426ad44 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:28.855049 1173 ts_tablet_manager.cc:1403] T 3d1a7578e83447669e3c3972a426ad44 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.024s user 0.006s sys 0.004s
I20260202 02:52:28.826228 1174 tablet_bootstrap.cc:492] T a1f39e5b02f747819e636cddb4292eb6 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:28.833158 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1116872cc25a4ab5b1a3a6bbba1f0da7" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:28.855702 1882 ts_tablet_manager.cc:1916] T 2b8ba9dd450948ada395207ae1c5b325 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:28.855767 1180 log.cc:1199] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/bd7b0a0fcb7645fdb9edfeccb3d18268
I20260202 02:52:28.855830 1174 ts_tablet_manager.cc:1403] T a1f39e5b02f747819e636cddb4292eb6 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.080s user 0.000s sys 0.009s
I20260202 02:52:28.844002 759 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a3dac5cd1ce47488a50b4c32eca67c7 with delete_type TABLET_DATA_DELETED (replaced by tablet 365417e050304073b5de887082fc699b) from {username='slave'} at 127.0.0.1:42950
W20260202 02:52:28.856247 857 leader_election.cc:343] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:28.856405 857 leader_election.cc:304] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:28.856581 1173 raft_consensus.cc:359] T 3d1a7578e83447669e3c3972a426ad44 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:28.856634 1174 raft_consensus.cc:359] T a1f39e5b02f747819e636cddb4292eb6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.856777 1174 raft_consensus.cc:385] T a1f39e5b02f747819e636cddb4292eb6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:28.856824 1174 raft_consensus.cc:740] T a1f39e5b02f747819e636cddb4292eb6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:28.856912 1173 raft_consensus.cc:385] T 3d1a7578e83447669e3c3972a426ad44 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:28.857008 1173 raft_consensus.cc:740] T 3d1a7578e83447669e3c3972a426ad44 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:28.857028 1174 consensus_queue.cc:260] T a1f39e5b02f747819e636cddb4292eb6 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.857311 1173 consensus_queue.cc:260] T 3d1a7578e83447669e3c3972a426ad44 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:28.857476 1174 ts_tablet_manager.cc:1434] T a1f39e5b02f747819e636cddb4292eb6 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:28.857769 1174 tablet_bootstrap.cc:492] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:28.859396 1174 tablet_bootstrap.cc:654] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:28.860273 2170 raft_consensus.cc:2749] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:28.857031 1180 ts_tablet_manager.cc:1950] T bd7b0a0fcb7645fdb9edfeccb3d18268 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:28.868348 1173 ts_tablet_manager.cc:1434] T 3d1a7578e83447669e3c3972a426ad44 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.013s user 0.002s sys 0.000s
I20260202 02:52:28.869028 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 429) took 5200 ms. Trace:
I20260202 02:52:28.869341 1180 rpcz_store.cc:276] 0202 02:52:23.668078 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.668281 (+ 203us) service_pool.cc:225] Handling call
0202 02:52:28.688466 (+5020185us) ts_tablet_manager.cc:1151] Deleting tablet bd7b0a0fcb7645fdb9edfeccb3d18268
0202 02:52:28.688476 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.762981 (+ 74505us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.814903 (+ 51922us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.868020 (+ 53117us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5019812}
I20260202 02:52:28.869495 1906 hybrid_clock.cc:648] HybridClock initialized: now 1770000748868620 us; error 907 us; skew 500 ppm
I20260202 02:52:28.870042 1906 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260202 02:52:28.870292 520 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet af5aa68e47bb420eac300754d274fa73 (table default.Table_12c352712c374a45b7de7a0e634a1327 [id=feb8b9f926434cec8b119843413f8c9b]). Expected version 1 got 0
I20260202 02:52:28.870465 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 432) took 5055 ms. Trace:
I20260202 02:52:28.870550 1180 rpcz_store.cc:276] 0202 02:52:23.815433 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.815488 (+ 55us) service_pool.cc:225] Handling call
0202 02:52:28.870004 (+5054516us) ts_tablet_manager.cc:1151] Deleting tablet 65bd1c1a382e42e4a68077264563c55b
0202 02:52:28.870016 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.870443 (+ 427us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5054288}
I20260202 02:52:28.870636 520 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 28a900bcaa8b41718a285e595886d805 (table default.Table_911bb209b4554ee8b9bc09edaf488d63 [id=cd07f9746ec94d9ebe956a7ca0d19b1c]). Expected version 1 got 0
I20260202 02:52:28.870870 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 433) took 5013 ms. Trace:
I20260202 02:52:28.870944 1180 rpcz_store.cc:276] 0202 02:52:23.857064 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.857130 (+ 66us) service_pool.cc:225] Handling call
0202 02:52:28.870767 (+5013637us) ts_tablet_manager.cc:1151] Deleting tablet 7ff6d2fa085343a6b11be0898da6233d
0202 02:52:28.870776 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.870854 (+ 78us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5013391}
I20260202 02:52:28.871223 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 434) took 5008 ms. Trace:
I20260202 02:52:28.871282 1180 rpcz_store.cc:276] 0202 02:52:23.862660 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.862714 (+ 54us) service_pool.cc:225] Handling call
0202 02:52:28.871129 (+5008415us) ts_tablet_manager.cc:1151] Deleting tablet b7880cb051c24bc998003f0effed3cf8
0202 02:52:28.871137 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.871208 (+ 71us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5008202}
I20260202 02:52:28.871505 1180 tablet_replica.cc:333] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:28.871671 1180 raft_consensus.cc:2243] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:28.871948 519 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_3b335a0806154f98a907f551596ba353 [id=0da4db61b3fe4cf4846edd5530b9327f] alter complete (version 1)
I20260202 02:52:28.874505 1173 tablet_bootstrap.cc:492] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:28.875936 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet bd7b0a0fcb7645fdb9edfeccb3d18268 (table default.Table_18fec323d6b04e10a687de6361b643b2 [id=1ffcc11561524fe8a7b09a7ddc92d5b2]) successfully deleted
W20260202 02:52:28.876197 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 65bd1c1a382e42e4a68077264563c55b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 65bd1c1a382e42e4a68077264563c55b already in progress: creating tablet
I20260202 02:52:28.876422 1173 tablet_bootstrap.cc:654] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:28.876415 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 7ff6d2fa085343a6b11be0898da6233d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7ff6d2fa085343a6b11be0898da6233d already in progress: creating tablet
W20260202 02:52:28.876590 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet b7880cb051c24bc998003f0effed3cf8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b7880cb051c24bc998003f0effed3cf8 already in progress: creating tablet
I20260202 02:52:28.878037 1180 raft_consensus.cc:2272] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:28.878657 1180 ts_tablet_manager.cc:1916] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:28.880388 759 tablet_service.cc:1552] Processing DeleteTablet for tablet 9c1271ce0f194021b588b82ac9d3db79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:28.880563 742 tablet_service.cc:1552] Processing DeleteTablet for tablet 08d8d781bb4e4acf915f11afb4fb2e22 with delete_type TABLET_DATA_DELETED (replaced by tablet 7921daa000f54c8dbed04473e21cf0a5) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:28.883572 1882 ts_tablet_manager.cc:1929] T 2b8ba9dd450948ada395207ae1c5b325 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:28.884403 1882 log.cc:1199] T 2b8ba9dd450948ada395207ae1c5b325 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/2b8ba9dd450948ada395207ae1c5b325
I20260202 02:52:28.887542 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_521f05f94e4b4b90a7304648efdd4919"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:28.888185 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_521f05f94e4b4b90a7304648efdd4919 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:28.888757 1882 ts_tablet_manager.cc:1950] T 2b8ba9dd450948ada395207ae1c5b325 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:28.890278 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 419) took 4809 ms. Trace:
I20260202 02:52:28.892454 1882 rpcz_store.cc:276] 0202 02:52:24.080901 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.081168 (+ 267us) service_pool.cc:225] Handling call
0202 02:52:28.852560 (+4771392us) ts_tablet_manager.cc:1151] Deleting tablet 2b8ba9dd450948ada395207ae1c5b325
0202 02:52:28.852570 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.865562 (+ 12992us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.883434 (+ 17872us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.890255 (+ 6821us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4771152}
I20260202 02:52:28.893131 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 2b8ba9dd450948ada395207ae1c5b325 (table default.Table_b88bd2e65dd140d5a1c208c9d4aa9430 [id=29f81d15b06d4f8abe2aba83493cfa6a]) successfully deleted
I20260202 02:52:28.893529 1882 tablet_replica.cc:333] T d21fca9a5b234c26879717f51c8ea0f0 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:28.893740 1882 raft_consensus.cc:2243] T d21fca9a5b234c26879717f51c8ea0f0 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:28.894157 1882 raft_consensus.cc:2272] T d21fca9a5b234c26879717f51c8ea0f0 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:28.894440 1174 tablet_bootstrap.cc:492] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:28.894593 1174 ts_tablet_manager.cc:1403] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.037s user 0.002s sys 0.004s
I20260202 02:52:28.895081 1882 ts_tablet_manager.cc:1916] T d21fca9a5b234c26879717f51c8ea0f0 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:28.895308 1174 raft_consensus.cc:359] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:28.895475 1174 raft_consensus.cc:385] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:28.895529 1174 raft_consensus.cc:740] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:28.895735 1174 consensus_queue.cc:260] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:28.896174 1174 ts_tablet_manager.cc:1434] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:28.896485 1174 tablet_bootstrap.cc:492] T 3d1a7578e83447669e3c3972a426ad44 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:28.898209 1174 tablet_bootstrap.cc:654] T 3d1a7578e83447669e3c3972a426ad44 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:28.898949 1180 ts_tablet_manager.cc:1929] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260202 02:52:28.899053 1180 log.cc:1199] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/5986676f5484498da927b4c6da3f47ff
I20260202 02:52:28.899375 1180 ts_tablet_manager.cc:1950] T 5986676f5484498da927b4c6da3f47ff P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:28.899667 1906 webserver.cc:492] Webserver started at http://127.0.104.190:40497/ using document root <none> and password file <none>
W20260202 02:52:28.899770 522 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d594295561ff4b01bf13451533a5743c consensus state has no leader
I20260202 02:52:28.900164 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 435) took 5029 ms. Trace:
I20260202 02:52:28.901064 1175 tablet_bootstrap.cc:492] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:28.901217 1175 ts_tablet_manager.cc:1403] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.059s user 0.000s sys 0.007s
I20260202 02:52:28.902065 1906 fs_manager.cc:362] Metadata directory not provided
I20260202 02:52:28.901993 1175 raft_consensus.cc:359] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.902179 1175 raft_consensus.cc:385] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:28.902235 1175 raft_consensus.cc:740] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:28.902398 1906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260202 02:52:28.900244 1180 rpcz_store.cc:276] 0202 02:52:23.871214 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.871267 (+ 53us) service_pool.cc:225] Handling call
0202 02:52:28.871462 (+5000195us) ts_tablet_manager.cc:1151] Deleting tablet 5986676f5484498da927b4c6da3f47ff
0202 02:52:28.871469 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.888407 (+ 16938us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.898828 (+ 10421us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.900144 (+ 1316us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4999961}
I20260202 02:52:28.904414 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 436) took 5021 ms. Trace:
I20260202 02:52:28.904492 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 5986676f5484498da927b4c6da3f47ff (table default.Table_755b6ce0796b45b8be9df04e59dfe565 [id=d8485f23f63b493781bea557dbe8a0c1]) successfully deleted
I20260202 02:52:28.904515 1180 rpcz_store.cc:276] 0202 02:52:23.883184 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:23.883246 (+ 62us) service_pool.cc:225] Handling call
0202 02:52:28.904245 (+5020999us) ts_tablet_manager.cc:1151] Deleting tablet d0ce2121d65b4a228596e0ca9018883d
0202 02:52:28.904257 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.904397 (+ 140us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5020777}
I20260202 02:52:28.904793 1180 tablet_replica.cc:333] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
W20260202 02:52:28.905081 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet d0ce2121d65b4a228596e0ca9018883d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d0ce2121d65b4a228596e0ca9018883d already in progress: creating tablet
I20260202 02:52:28.908164 1180 raft_consensus.cc:2243] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:28.908285 1175 consensus_queue.cc:260] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.909090 2170 raft_consensus.cc:493] T a0b5d82e5fb94b60b4aa324b978a5378 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:28.909224 2170 raft_consensus.cc:515] T a0b5d82e5fb94b60b4aa324b978a5378 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.909850 2170 leader_election.cc:290] T a0b5d82e5fb94b60b4aa324b978a5378 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:28.910326 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a0b5d82e5fb94b60b4aa324b978a5378" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:28.911222 859 leader_election.cc:343] T a0b5d82e5fb94b60b4aa324b978a5378 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:28.911825 1180 raft_consensus.cc:2272] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:28.913318 1180 ts_tablet_manager.cc:1916] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:28.914469 522 catalog_manager.cc:5654] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d reported cstate change: term changed from 1 to 2, leader changed from b1f05820b6034228bdad884cc348716d (127.0.104.131) to 1f442ef85029461091de847118c34c0d (127.0.104.129). New cstate: current_term: 2 leader_uuid: "1f442ef85029461091de847118c34c0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } }
I20260202 02:52:28.917002 522 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet 28a900bcaa8b41718a285e595886d805 (table default.Table_911bb209b4554ee8b9bc09edaf488d63 [id=cd07f9746ec94d9ebe956a7ca0d19b1c]). Expected version 1 got 0
I20260202 02:52:28.917529 522 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet cb634d4bae554ef3b39eeaa9958adc6c (table default.Table_e5f4ac82d1ef4b85999695997eb9df2a [id=c97e407e519d4a538860f66931f6aefd]). Expected version 2 got 0
I20260202 02:52:28.918840 2170 raft_consensus.cc:493] T 072bb7f0e4ab4935be49461bc8a3c163 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:28.918989 2170 raft_consensus.cc:515] T 072bb7f0e4ab4935be49461bc8a3c163 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.919502 2170 leader_election.cc:290] T 072bb7f0e4ab4935be49461bc8a3c163 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:28.920461 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "072bb7f0e4ab4935be49461bc8a3c163" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:28.921041 859 leader_election.cc:343] T 072bb7f0e4ab4935be49461bc8a3c163 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:28.921150 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "072bb7f0e4ab4935be49461bc8a3c163" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:28.916302 2157 raft_consensus.cc:493] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:28.921505 2157 raft_consensus.cc:515] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:28.921653 857 leader_election.cc:343] T 072bb7f0e4ab4935be49461bc8a3c163 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:28.921758 857 leader_election.cc:304] T 072bb7f0e4ab4935be49461bc8a3c163 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:28.922173 2157 leader_election.cc:290] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:28.917555 2150 consensus_queue.cc:1048] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:28.920521 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a0b5d82e5fb94b60b4aa324b978a5378" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:28.922912 857 leader_election.cc:343] T a0b5d82e5fb94b60b4aa324b978a5378 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:28.923017 857 leader_election.cc:304] T a0b5d82e5fb94b60b4aa324b978a5378 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:28.923435 2170 raft_consensus.cc:2749] T 072bb7f0e4ab4935be49461bc8a3c163 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:28.923540 1882 ts_tablet_manager.cc:1929] T d21fca9a5b234c26879717f51c8ea0f0 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:28.923601 2170 raft_consensus.cc:2749] T a0b5d82e5fb94b60b4aa324b978a5378 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:28.923651 1882 log.cc:1199] T d21fca9a5b234c26879717f51c8ea0f0 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/d21fca9a5b234c26879717f51c8ea0f0
I20260202 02:52:28.923969 1882 ts_tablet_manager.cc:1950] T d21fca9a5b234c26879717f51c8ea0f0 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:28.924422 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9c1271ce0f194021b588b82ac9d3db79" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:28.924605 1062 raft_consensus.cc:2468] T 9c1271ce0f194021b588b82ac9d3db79 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:28.925220 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 420) took 4844 ms. Trace:
I20260202 02:52:28.925372 1174 tablet_bootstrap.cc:492] T 3d1a7578e83447669e3c3972a426ad44 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:28.925513 1174 ts_tablet_manager.cc:1403] T 3d1a7578e83447669e3c3972a426ad44 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.029s user 0.005s sys 0.002s
I20260202 02:52:28.925849 715 leader_election.cc:304] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:28.926108 2150 raft_consensus.cc:2804] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:28.926180 2150 raft_consensus.cc:493] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:28.926239 2150 raft_consensus.cc:3060] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:28.926285 1174 raft_consensus.cc:359] T 3d1a7578e83447669e3c3972a426ad44 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:28.926422 1174 raft_consensus.cc:385] T 3d1a7578e83447669e3c3972a426ad44 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:28.926465 1174 raft_consensus.cc:740] T 3d1a7578e83447669e3c3972a426ad44 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:28.926671 1174 consensus_queue.cc:260] T 3d1a7578e83447669e3c3972a426ad44 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:28.927150 1882 rpcz_store.cc:276] 0202 02:52:24.081095 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.081826 (+ 731us) service_pool.cc:225] Handling call
0202 02:52:28.893474 (+4811648us) ts_tablet_manager.cc:1151] Deleting tablet d21fca9a5b234c26879717f51c8ea0f0
0202 02:52:28.893486 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.908899 (+ 15413us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.923430 (+ 14531us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.925202 (+ 1772us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4810211}
I20260202 02:52:28.927583 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet d21fca9a5b234c26879717f51c8ea0f0 (table default.Table_b5dcfcce852a48a0b1fbd3deed34f4d2 [id=a77fa0c30d27405294adaef676bc76f7]) successfully deleted
I20260202 02:52:28.927978 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 421) took 4828 ms. Trace:
I20260202 02:52:28.928050 1882 rpcz_store.cc:276] 0202 02:52:24.099576 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.101909 (+ 2333us) service_pool.cc:225] Handling call
0202 02:52:28.927837 (+4825928us) ts_tablet_manager.cc:1151] Deleting tablet dc4c3f52f4f44a84b0d0e3c9a4a49f09
0202 02:52:28.927848 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.927960 (+ 112us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4825582}
W20260202 02:52:28.928269 527 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8153510d5d154c7895b1ee794585612e consensus state has no leader
W20260202 02:52:28.928493 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet dc4c3f52f4f44a84b0d0e3c9a4a49f09 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dc4c3f52f4f44a84b0d0e3c9a4a49f09 already in progress: creating tablet
I20260202 02:52:28.929032 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 422) took 4810 ms. Trace:
I20260202 02:52:28.929060 1174 ts_tablet_manager.cc:1434] T 3d1a7578e83447669e3c3972a426ad44 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260202 02:52:28.929103 1882 rpcz_store.cc:276] 0202 02:52:24.119024 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.119744 (+ 720us) service_pool.cc:225] Handling call
0202 02:52:28.928910 (+4809166us) ts_tablet_manager.cc:1151] Deleting tablet da5435f969074364bc8f8a6ddc5e4f8a
0202 02:52:28.928920 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.929016 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4808881}
I20260202 02:52:28.929340 1174 tablet_bootstrap.cc:492] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
W20260202 02:52:28.929448 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet da5435f969074364bc8f8a6ddc5e4f8a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet da5435f969074364bc8f8a6ddc5e4f8a already in progress: creating tablet
I20260202 02:52:28.930593 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 423) took 4766 ms. Trace:
I20260202 02:52:28.930754 2150 raft_consensus.cc:515] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.930961 1174 tablet_bootstrap.cc:654] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:28.931536 2150 leader_election.cc:290] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:28.932046 1882 rpcz_store.cc:276] 0202 02:52:24.164035 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.164095 (+ 60us) service_pool.cc:225] Handling call
0202 02:52:28.930481 (+4766386us) ts_tablet_manager.cc:1151] Deleting tablet c9ef08b5830147a3a55049285a40535b
0202 02:52:28.930490 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.930574 (+ 84us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4766151}
I20260202 02:52:28.932116 527 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_d5007434a43d4bed87b905c21243d3aa" } modify_external_catalogs: true
I20260202 02:52:28.932281 527 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_d5007434a43d4bed87b905c21243d3aa" } modify_external_catalogs: true
I20260202 02:52:28.932268 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9c1271ce0f194021b588b82ac9d3db79" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:28.932585 1062 raft_consensus.cc:3060] T 9c1271ce0f194021b588b82ac9d3db79 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:28.935372 1062 raft_consensus.cc:2468] T 9c1271ce0f194021b588b82ac9d3db79 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
W20260202 02:52:28.935633 512 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 6a3dac5cd1ce47488a50b4c32eca67c7 (table default.Table_25f0023dd22a4bb0a7c2863da66d0002 [id=e24ffad12f514d60907b6cadfd09dbe6], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 6a3dac5cd1ce47488a50b4c32eca67c7 consensus state has no leader
I20260202 02:52:28.936177 715 leader_election.cc:304] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:28.936487 2150 raft_consensus.cc:2804] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
W20260202 02:52:28.936625 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet c9ef08b5830147a3a55049285a40535b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c9ef08b5830147a3a55049285a40535b already in progress: creating tablet
I20260202 02:52:28.936798 2150 raft_consensus.cc:697] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:28.937090 2150 consensus_queue.cc:237] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.937263 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 433) took 4511 ms. Trace:
I20260202 02:52:28.937706 1882 rpcz_store.cc:276] 0202 02:52:24.425954 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.426144 (+ 190us) service_pool.cc:225] Handling call
0202 02:52:28.937118 (+4510974us) ts_tablet_manager.cc:1151] Deleting tablet 587cfd0ae62f4cfd937faf6890e78f4a
0202 02:52:28.937131 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.937244 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4501939}
W20260202 02:52:28.940167 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 587cfd0ae62f4cfd937faf6890e78f4a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 587cfd0ae62f4cfd937faf6890e78f4a already in progress: creating tablet
I20260202 02:52:28.940886 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 436) took 4341 ms. Trace:
I20260202 02:52:28.941455 1882 rpcz_store.cc:276] 0202 02:52:24.598955 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.604607 (+ 5652us) service_pool.cc:225] Handling call
0202 02:52:28.940715 (+4336108us) ts_tablet_manager.cc:1151] Deleting tablet 75569511179b41038e521fa3346a92ae
0202 02:52:28.940731 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.940862 (+ 131us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4335885}
I20260202 02:52:28.940167 521 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 79e65a55f29449a0b13349bc152c651b
I20260202 02:52:28.944363 1906 fs_manager.cc:714] Time spent opening directory manager: real 0.039s user 0.003s sys 0.003s
I20260202 02:52:28.946098 742 tablet_service.cc:1552] Processing DeleteTablet for tablet 79e65a55f29449a0b13349bc152c651b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:28 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:28.945470 1039 tablet_service.cc:1552] Processing DeleteTablet for tablet 79e65a55f29449a0b13349bc152c651b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:28 UTC) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:28.948189 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 75569511179b41038e521fa3346a92ae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 75569511179b41038e521fa3346a92ae already in progress: creating tablet
I20260202 02:52:28.948546 1882 tablet_replica.cc:333] T a1f39e5b02f747819e636cddb4292eb6 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:28.952143 1882 raft_consensus.cc:2243] T a1f39e5b02f747819e636cddb4292eb6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:28.952448 1882 raft_consensus.cc:2272] T a1f39e5b02f747819e636cddb4292eb6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:28.952704 1173 tablet_bootstrap.cc:492] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:28.952920 1173 ts_tablet_manager.cc:1403] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.078s user 0.007s sys 0.000s
I20260202 02:52:28.953094 1882 ts_tablet_manager.cc:1916] T a1f39e5b02f747819e636cddb4292eb6 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:28.953869 1173 raft_consensus.cc:359] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:28.954144 1173 raft_consensus.cc:385] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:28.954120 521 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ea358e346aa9402ba015964e2a8a906d" } modify_external_catalogs: true
I20260202 02:52:28.954308 521 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ea358e346aa9402ba015964e2a8a906d" } modify_external_catalogs: true
I20260202 02:52:28.954334 1173 raft_consensus.cc:740] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:28.954818 1173 consensus_queue.cc:260] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:28.964366 528 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_24d7cc37a3884cda9a1e269515d3877d" } modify_external_catalogs: true
I20260202 02:52:28.964599 528 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_24d7cc37a3884cda9a1e269515d3877d" } modify_external_catalogs: true
I20260202 02:52:28.965494 1173 ts_tablet_manager.cc:1434] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.012s user 0.001s sys 0.000s
I20260202 02:52:28.965927 1173 tablet_bootstrap.cc:492] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:28.967470 2127 raft_consensus.cc:493] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:28.967608 2127 raft_consensus.cc:515] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.968209 2127 leader_election.cc:290] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:28.968919 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8774c6bc0fe9414ea2f17ed859d9392a" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:28.969135 772 raft_consensus.cc:2468] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:28.969512 998 leader_election.cc:304] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:28.969921 2127 raft_consensus.cc:2804] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260202 02:52:28.970008 2127 raft_consensus.cc:493] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:28.970130 2127 raft_consensus.cc:3060] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Advancing to term 2
W20260202 02:52:28.971269 524 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c66f415bf66b48a9af192301b982f8dd consensus state has no leader
I20260202 02:52:28.972594 2127 raft_consensus.cc:515] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.973655 2127 leader_election.cc:290] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:28.973866 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8774c6bc0fe9414ea2f17ed859d9392a" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:28.974064 772 raft_consensus.cc:3060] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:28.975167 524 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_9465647c983f4c85b7b2b5454fcc80e7" } modify_external_catalogs: true
I20260202 02:52:28.975364 524 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_9465647c983f4c85b7b2b5454fcc80e7" } modify_external_catalogs: true
I20260202 02:52:28.970006 1882 ts_tablet_manager.cc:1929] T a1f39e5b02f747819e636cddb4292eb6 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:28.975580 1882 log.cc:1199] T a1f39e5b02f747819e636cddb4292eb6 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/a1f39e5b02f747819e636cddb4292eb6
I20260202 02:52:28.975878 1882 ts_tablet_manager.cc:1950] T a1f39e5b02f747819e636cddb4292eb6 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:28.976616 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 439) took 4362 ms. Trace:
I20260202 02:52:28.977418 772 raft_consensus.cc:2468] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 2.
I20260202 02:52:28.978570 1174 tablet_bootstrap.cc:492] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:28.979240 1174 ts_tablet_manager.cc:1403] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.050s user 0.004s sys 0.012s
I20260202 02:52:28.980020 1174 raft_consensus.cc:359] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:28.980180 1174 raft_consensus.cc:385] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:28.980230 1174 raft_consensus.cc:740] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:28.977843 998 leader_election.cc:304] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:28.982857 2127 raft_consensus.cc:2804] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Leader election won for term 2
I20260202 02:52:28.982419 1882 rpcz_store.cc:276] 0202 02:52:24.614269 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.614399 (+ 130us) service_pool.cc:225] Handling call
0202 02:52:28.948490 (+4334091us) ts_tablet_manager.cc:1151] Deleting tablet a1f39e5b02f747819e636cddb4292eb6
0202 02:52:28.948503 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:28.958604 (+ 10101us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.969888 (+ 11284us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:28.976597 (+ 6709us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4333837}
I20260202 02:52:28.982995 2127 raft_consensus.cc:697] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [term 2 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:28.983191 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet a1f39e5b02f747819e636cddb4292eb6 (table default.Table_61c2ba441aa746edbb73f8b5de4204ba [id=abceac85d3524c96834546f67f41f933]) successfully deleted
I20260202 02:52:28.983253 2127 consensus_queue.cc:237] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:28.983621 1882 tablet_replica.cc:333] T fe9ea4c0502f4976b440465992f2ea2c P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:28.983806 1882 raft_consensus.cc:2243] T fe9ea4c0502f4976b440465992f2ea2c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:28.983811 1174 consensus_queue.cc:260] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:28.985001 1174 ts_tablet_manager.cc:1434] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.005s user 0.000s sys 0.001s
I20260202 02:52:28.985283 1174 tablet_bootstrap.cc:492] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:28.985886 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_22af4f1f5edd4a0cba0252f162e49241"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:28.986506 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_22af4f1f5edd4a0cba0252f162e49241 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:28.992059 1882 raft_consensus.cc:2272] T fe9ea4c0502f4976b440465992f2ea2c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:28.993798 1174 tablet_bootstrap.cc:654] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:28.993810 1721 raft_consensus.cc:493] T d30557fbe9084cd3860820f69dda8752 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:28.993938 1721 raft_consensus.cc:515] T d30557fbe9084cd3860820f69dda8752 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:28.994537 1721 leader_election.cc:290] T d30557fbe9084cd3860820f69dda8752 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:28.995426 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d30557fbe9084cd3860820f69dda8752" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:28.995879 998 leader_election.cc:343] T d30557fbe9084cd3860820f69dda8752 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:28.996184 1882 ts_tablet_manager.cc:1916] T fe9ea4c0502f4976b440465992f2ea2c P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:29.001524 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_987beb2b868c44dd90fb8669aab44ca7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:29.002199 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_987beb2b868c44dd90fb8669aab44ca7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:29.008278 1174 tablet_bootstrap.cc:492] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:29.008934 1174 ts_tablet_manager.cc:1403] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.024s user 0.006s sys 0.001s
I20260202 02:52:29.009366 1173 tablet_bootstrap.cc:654] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:29.009953 1174 raft_consensus.cc:359] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.010562 1882 ts_tablet_manager.cc:1929] T fe9ea4c0502f4976b440465992f2ea2c P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:29.011471 1174 raft_consensus.cc:385] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:29.011737 1174 raft_consensus.cc:740] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:29.012135 1882 log.cc:1199] T fe9ea4c0502f4976b440465992f2ea2c P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/fe9ea4c0502f4976b440465992f2ea2c
I20260202 02:52:29.012184 1174 consensus_queue.cc:260] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.012653 1882 ts_tablet_manager.cc:1950] T fe9ea4c0502f4976b440465992f2ea2c P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:29.013789 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 438) took 4399 ms. Trace:
I20260202 02:52:29.015107 523 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_911bb209b4554ee8b9bc09edaf488d63 [id=cd07f9746ec94d9ebe956a7ca0d19b1c] alter complete (version 1)
I20260202 02:52:29.016960 1882 rpcz_store.cc:276] 0202 02:52:24.614125 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.681168 (+ 67043us) service_pool.cc:225] Handling call
0202 02:52:28.983569 (+4302401us) ts_tablet_manager.cc:1151] Deleting tablet fe9ea4c0502f4976b440465992f2ea2c
0202 02:52:28.983580 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.002183 (+ 18603us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.010438 (+ 8255us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.013770 (+ 3332us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4301938}
I20260202 02:52:29.017503 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet fe9ea4c0502f4976b440465992f2ea2c (table default.Table_9181c8b857b540ff846c397f9f1a5484 [id=d2fb3868962547c48e29f6353e5747a9]) successfully deleted
I20260202 02:52:29.018107 1882 tablet_replica.cc:333] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:29.018306 1882 raft_consensus.cc:2243] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:29.021821 1882 raft_consensus.cc:2272] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:29.022800 1882 ts_tablet_manager.cc:1916] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:29.023445 1174 ts_tablet_manager.cc:1434] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.014s user 0.002s sys 0.000s
I20260202 02:52:29.023885 2241 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260202 02:52:29.023798 1174 tablet_bootstrap.cc:492] T 16b47a2831f54d809ed0b16deaec7e3d P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:29.026603 1174 tablet_bootstrap.cc:654] T 16b47a2831f54d809ed0b16deaec7e3d P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:29.027822 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 79e65a55f29449a0b13349bc152c651b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:28 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:29.029959 520 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_e5f4ac82d1ef4b85999695997eb9df2a [id=c97e407e519d4a538860f66931f6aefd] alter complete (version 2)
I20260202 02:52:29.030923 1180 ts_tablet_manager.cc:1929] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:29.031051 1180 log.cc:1199] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/2b8ba9dd450948ada395207ae1c5b325
I20260202 02:52:29.031369 1180 ts_tablet_manager.cc:1950] T 2b8ba9dd450948ada395207ae1c5b325 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:29.032151 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 439) took 4963 ms. Trace:
I20260202 02:52:29.032238 1180 rpcz_store.cc:276] 0202 02:52:24.068641 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.068714 (+ 73us) service_pool.cc:225] Handling call
0202 02:52:28.904745 (+4836031us) ts_tablet_manager.cc:1151] Deleting tablet 2b8ba9dd450948ada395207ae1c5b325
0202 02:52:28.904754 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.021612 (+116858us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.030817 (+ 9205us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.032130 (+ 1313us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4835747}
I20260202 02:52:29.032604 1180 tablet_replica.cc:333] T d21fca9a5b234c26879717f51c8ea0f0 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:29.032761 1180 raft_consensus.cc:2243] T d21fca9a5b234c26879717f51c8ea0f0 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:29.032895 1180 raft_consensus.cc:2272] T d21fca9a5b234c26879717f51c8ea0f0 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:29.033386 1180 ts_tablet_manager.cc:1916] T d21fca9a5b234c26879717f51c8ea0f0 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:29.033514 1042 tablet_service.cc:1552] Processing DeleteTablet for tablet 02b434fa058b45529ff3938b053263c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:29.033710 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet 94bada52db6a47179415d302b541d01f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:29.033880 1039 tablet_service.cc:1552] Processing DeleteTablet for tablet d30557fbe9084cd3860820f69dda8752 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:29.034318 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 2b8ba9dd450948ada395207ae1c5b325 (table default.Table_b88bd2e65dd140d5a1c208c9d4aa9430 [id=29f81d15b06d4f8abe2aba83493cfa6a]) successfully deleted
W20260202 02:52:29.038811 714 leader_election.cc:343] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 9c1271ce0f194021b588b82ac9d3db79
I20260202 02:52:29.039379 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_6da2058f0c8c44feb77c1106bda2bffe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:29.040022 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6da2058f0c8c44feb77c1106bda2bffe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:29.041162 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 79e65a55f29449a0b13349bc152c651b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 79e65a55f29449a0b13349bc152c651b already in progress: creating tablet
W20260202 02:52:29.041441 999 leader_election.cc:343] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 8774c6bc0fe9414ea2f17ed859d9392a
W20260202 02:52:29.041827 999 leader_election.cc:343] T d30557fbe9084cd3860820f69dda8752 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: d30557fbe9084cd3860820f69dda8752
I20260202 02:52:29.041935 999 leader_election.cc:304] T d30557fbe9084cd3860820f69dda8752 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:29.042202 2127 raft_consensus.cc:2749] T d30557fbe9084cd3860820f69dda8752 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:29.042335 999 leader_election.cc:343] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 8774c6bc0fe9414ea2f17ed859d9392a
I20260202 02:52:29.042608 1882 ts_tablet_manager.cc:1929] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:29.042781 1882 log.cc:1199] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/1f54c1dfe0a942b3bafd85522953a032
I20260202 02:52:29.043212 1882 ts_tablet_manager.cc:1950] T 1f54c1dfe0a942b3bafd85522953a032 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:29.044000 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 445) took 4355 ms. Trace:
I20260202 02:52:29.044152 1882 rpcz_store.cc:276] 0202 02:52:24.688907 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.708083 (+ 19176us) service_pool.cc:225] Handling call
0202 02:52:29.018059 (+4309976us) ts_tablet_manager.cc:1151] Deleting tablet 1f54c1dfe0a942b3bafd85522953a032
0202 02:52:29.018069 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.039294 (+ 21225us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.042508 (+ 3214us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.043937 (+ 1429us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4303742}
I20260202 02:52:29.044461 1180 ts_tablet_manager.cc:1929] T d21fca9a5b234c26879717f51c8ea0f0 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:29.044762 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 1f54c1dfe0a942b3bafd85522953a032 (table default.Table_a22f252ee8d44be5a9d7202afb726cfc [id=891d9355855640a28939b84b2dd30537]) successfully deleted
I20260202 02:52:29.044783 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_abc8ed4a399d4fdb9479666c6368f587"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:29.045260 1882 tablet_replica.cc:333] T 3711910e050c4515bed4294d1271d5a9 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
W20260202 02:52:29.045399 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_abc8ed4a399d4fdb9479666c6368f587 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:29.045588 1882 raft_consensus.cc:2243] T 3711910e050c4515bed4294d1271d5a9 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:29.045816 1882 raft_consensus.cc:2272] T 3711910e050c4515bed4294d1271d5a9 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:29.046518 1882 ts_tablet_manager.cc:1916] T 3711910e050c4515bed4294d1271d5a9 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:29.046823 1180 log.cc:1199] T d21fca9a5b234c26879717f51c8ea0f0 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/d21fca9a5b234c26879717f51c8ea0f0
I20260202 02:52:29.047533 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 072bb7f0e4ab4935be49461bc8a3c163 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:29.047844 1182 tablet_replica.cc:333] T 072bb7f0e4ab4935be49461bc8a3c163 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:29.048031 1182 raft_consensus.cc:2243] T 072bb7f0e4ab4935be49461bc8a3c163 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:29.048174 1182 raft_consensus.cc:2272] T 072bb7f0e4ab4935be49461bc8a3c163 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:29.048182 1180 ts_tablet_manager.cc:1950] T d21fca9a5b234c26879717f51c8ea0f0 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:29.048695 1182 ts_tablet_manager.cc:1916] T 072bb7f0e4ab4935be49461bc8a3c163 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:29.049870 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 440) took 4970 ms. Trace:
I20260202 02:52:29.051390 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 550895b4010f4088b2cc34acff50e03f with delete_type TABLET_DATA_DELETED (replaced by tablet 5416eab7a0914ecb855fbccde4894285) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:29.050621 714 leader_election.cc:343] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 9c1271ce0f194021b588b82ac9d3db79
I20260202 02:52:29.051656 1180 rpcz_store.cc:276] 0202 02:52:24.079512 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.079593 (+ 81us) service_pool.cc:225] Handling call
0202 02:52:29.032537 (+4952944us) ts_tablet_manager.cc:1151] Deleting tablet d21fca9a5b234c26879717f51c8ea0f0
0202 02:52:29.032545 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.036244 (+ 3699us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.044319 (+ 8075us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.049825 (+ 5506us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4952701}
I20260202 02:52:29.052083 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 441) took 4953 ms. Trace:
I20260202 02:52:29.052207 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet d21fca9a5b234c26879717f51c8ea0f0 (table default.Table_b5dcfcce852a48a0b1fbd3deed34f4d2 [id=a77fa0c30d27405294adaef676bc76f7]) successfully deleted
I20260202 02:52:29.052179 1180 rpcz_store.cc:276] 0202 02:52:24.098737 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.098799 (+ 62us) service_pool.cc:225] Handling call
0202 02:52:29.051927 (+4953128us) ts_tablet_manager.cc:1151] Deleting tablet dc4c3f52f4f44a84b0d0e3c9a4a49f09
0202 02:52:29.051937 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.052046 (+ 109us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4952878}
W20260202 02:52:29.052546 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet dc4c3f52f4f44a84b0d0e3c9a4a49f09 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dc4c3f52f4f44a84b0d0e3c9a4a49f09 already in progress: creating tablet
I20260202 02:52:29.053426 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 442) took 4935 ms. Trace:
I20260202 02:52:29.053869 1180 rpcz_store.cc:276] 0202 02:52:24.118164 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.118222 (+ 58us) service_pool.cc:225] Handling call
0202 02:52:29.053297 (+4935075us) ts_tablet_manager.cc:1151] Deleting tablet da5435f969074364bc8f8a6ddc5e4f8a
0202 02:52:29.053307 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.053391 (+ 84us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4934849}
I20260202 02:52:29.054095 1882 ts_tablet_manager.cc:1929] T 3711910e050c4515bed4294d1271d5a9 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:29.054244 1882 log.cc:1199] T 3711910e050c4515bed4294d1271d5a9 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/3711910e050c4515bed4294d1271d5a9
I20260202 02:52:29.054330 1906 fs_manager.cc:730] Time spent opening block manager: real 0.107s user 0.012s sys 0.006s
I20260202 02:52:29.054695 1882 ts_tablet_manager.cc:1950] T 3711910e050c4515bed4294d1271d5a9 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
W20260202 02:52:29.054733 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet da5435f969074364bc8f8a6ddc5e4f8a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet da5435f969074364bc8f8a6ddc5e4f8a already in progress: creating tablet
I20260202 02:52:29.054553 1906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-0/data,/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-0/wal
uuid: "7dab2abef14445d688ce1335b1f9b2f8"
format_stamp: "Formatted at 2026-02-02 02:52:10 on dist-test-slave-llj2"
I20260202 02:52:29.055576 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 443) took 4891 ms. Trace:
I20260202 02:52:29.055621 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 448) took 4337 ms. Trace:
I20260202 02:52:29.055558 1906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260202 02:52:29.055697 1180 rpcz_store.cc:276] 0202 02:52:24.164035 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.164095 (+ 60us) service_pool.cc:225] Handling call
0202 02:52:29.055448 (+4891353us) ts_tablet_manager.cc:1151] Deleting tablet c9ef08b5830147a3a55049285a40535b
0202 02:52:29.055458 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.055544 (+ 86us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4891118}
I20260202 02:52:29.055986 1882 rpcz_store.cc:276] 0202 02:52:24.717977 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.722406 (+ 4429us) service_pool.cc:225] Handling call
0202 02:52:29.045172 (+4322766us) ts_tablet_manager.cc:1151] Deleting tablet 3711910e050c4515bed4294d1271d5a9
0202 02:52:29.045183 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.050090 (+ 4907us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.053959 (+ 3869us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.055584 (+ 1625us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4322502}
W20260202 02:52:29.056129 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet c9ef08b5830147a3a55049285a40535b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c9ef08b5830147a3a55049285a40535b already in progress: creating tablet
I20260202 02:52:29.056542 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 3711910e050c4515bed4294d1271d5a9 (table default.Table_ab29f752f1e140da9038d9e11ee61f39 [id=9ef504467a544f299b48499e49adf27f]) successfully deleted
I20260202 02:52:29.056879 1882 tablet_replica.cc:333] T 6154460838eb4e508f3646d3e3e4cf21 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:29.057129 1882 raft_consensus.cc:2243] T 6154460838eb4e508f3646d3e3e4cf21 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:29.057349 1882 raft_consensus.cc:2272] T 6154460838eb4e508f3646d3e3e4cf21 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:29.056922 1180 tablet_replica.cc:333] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:29.057889 1180 raft_consensus.cc:2243] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:29.058068 1882 ts_tablet_manager.cc:1916] T 6154460838eb4e508f3646d3e3e4cf21 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:29.058074 1180 raft_consensus.cc:2272] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:29.058559 1180 ts_tablet_manager.cc:1916] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:29.064760 1882 ts_tablet_manager.cc:1929] T 6154460838eb4e508f3646d3e3e4cf21 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:29.065275 1882 log.cc:1199] T 6154460838eb4e508f3646d3e3e4cf21 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/6154460838eb4e508f3646d3e3e4cf21
I20260202 02:52:29.065670 1882 ts_tablet_manager.cc:1950] T 6154460838eb4e508f3646d3e3e4cf21 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:29.066491 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 447) took 4348 ms. Trace:
I20260202 02:52:29.066646 1882 rpcz_store.cc:276] 0202 02:52:24.717837 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.722877 (+ 5040us) service_pool.cc:225] Handling call
0202 02:52:29.056815 (+4333938us) ts_tablet_manager.cc:1151] Deleting tablet 6154460838eb4e508f3646d3e3e4cf21
0202 02:52:29.056826 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.061318 (+ 4492us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.064634 (+ 3316us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.066434 (+ 1800us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4333693}
I20260202 02:52:29.067255 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 6154460838eb4e508f3646d3e3e4cf21 (table default.Table_6e6ab095d9a6497d8a89bcc865b6f42f [id=7d8f7ac4c2ee4b72b2ff25aff060010f]) successfully deleted
I20260202 02:52:29.070236 1173 tablet_bootstrap.cc:492] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:29.070410 1173 ts_tablet_manager.cc:1403] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.105s user 0.000s sys 0.003s
I20260202 02:52:29.070881 1882 tablet_replica.cc:333] T c25e329023c04f609d8a743b4f7a95a4 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:29.071141 1882 raft_consensus.cc:2243] T c25e329023c04f609d8a743b4f7a95a4 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:29.071223 1173 raft_consensus.cc:359] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.071902 1173 raft_consensus.cc:385] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:29.072050 1173 raft_consensus.cc:740] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:29.072309 1173 consensus_queue.cc:260] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.072965 1173 ts_tablet_manager.cc:1434] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260202 02:52:29.073302 1180 ts_tablet_manager.cc:1929] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:29.073714 1182 ts_tablet_manager.cc:1929] T 072bb7f0e4ab4935be49461bc8a3c163 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:29.073881 1182 log.cc:1199] T 072bb7f0e4ab4935be49461bc8a3c163 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/072bb7f0e4ab4935be49461bc8a3c163
I20260202 02:52:29.074198 1182 ts_tablet_manager.cc:1950] T 072bb7f0e4ab4935be49461bc8a3c163 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:29.074699 1180 log.cc:1199] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/eb0ac55fbb534b29a86d6b3e8aa72ebc
I20260202 02:52:29.075186 1180 ts_tablet_manager.cc:1950] T eb0ac55fbb534b29a86d6b3e8aa72ebc P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:29.080052 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 445) took 4835 ms. Trace:
I20260202 02:52:29.080880 1180 rpcz_store.cc:276] 0202 02:52:24.244695 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.244927 (+ 232us) service_pool.cc:225] Handling call
0202 02:52:29.056858 (+4811931us) ts_tablet_manager.cc:1151] Deleting tablet eb0ac55fbb534b29a86d6b3e8aa72ebc
0202 02:52:29.056867 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.062628 (+ 5761us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.073168 (+ 10540us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.079989 (+ 6821us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4811696}
I20260202 02:52:29.081151 1173 tablet_bootstrap.cc:492] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:29.081714 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 446) took 4836 ms. Trace:
I20260202 02:52:29.081821 1180 rpcz_store.cc:276] 0202 02:52:24.244865 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.245379 (+ 514us) service_pool.cc:225] Handling call
0202 02:52:29.081577 (+4836198us) ts_tablet_manager.cc:1151] Deleting tablet d1e0e74f32a24729b250c5751677cfb0
0202 02:52:29.081590 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.081678 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4835979}
I20260202 02:52:29.082191 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 457) took 4672 ms. Trace:
I20260202 02:52:29.082285 1180 rpcz_store.cc:276] 0202 02:52:24.409391 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.409448 (+ 57us) service_pool.cc:225] Handling call
0202 02:52:29.082066 (+4672618us) ts_tablet_manager.cc:1151] Deleting tablet 587cfd0ae62f4cfd937faf6890e78f4a
0202 02:52:29.082076 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.082162 (+ 86us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4672349}
I20260202 02:52:29.082605 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 459) took 4410 ms. Trace:
I20260202 02:52:29.082693 1180 rpcz_store.cc:276] 0202 02:52:24.672174 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.672238 (+ 64us) service_pool.cc:225] Handling call
0202 02:52:29.082484 (+4410246us) ts_tablet_manager.cc:1151] Deleting tablet 75569511179b41038e521fa3346a92ae
0202 02:52:29.082493 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.082573 (+ 80us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4403021}
I20260202 02:52:29.082989 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 461) took 4397 ms. Trace:
I20260202 02:52:29.083086 1180 rpcz_store.cc:276] 0202 02:52:24.685496 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.685558 (+ 62us) service_pool.cc:225] Handling call
0202 02:52:29.082885 (+4397327us) ts_tablet_manager.cc:1151] Deleting tablet fe9ea4c0502f4976b440465992f2ea2c
0202 02:52:29.082894 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.082961 (+ 67us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4393619}
I20260202 02:52:29.083338 1180 tablet_replica.cc:333] T ffd80badce1048f3ac95366cad05ae53 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:29.083554 1180 raft_consensus.cc:2243] T ffd80badce1048f3ac95366cad05ae53 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:29.083730 1180 raft_consensus.cc:2272] T ffd80badce1048f3ac95366cad05ae53 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:29.084918 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet eb0ac55fbb534b29a86d6b3e8aa72ebc (table default.Table_8f23e3356aa14793a1aad274d2b845eb [id=21c284c2be45496ca760633f24f090d2]) successfully deleted
W20260202 02:52:29.085302 513 catalog_manager.cc:4952] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet d1e0e74f32a24729b250c5751677cfb0 because the tablet was not found. No further retry: Not found: Tablet not found: d1e0e74f32a24729b250c5751677cfb0
W20260202 02:52:29.085537 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 587cfd0ae62f4cfd937faf6890e78f4a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 587cfd0ae62f4cfd937faf6890e78f4a already in progress: creating tablet
I20260202 02:52:29.085672 1173 tablet_bootstrap.cc:654] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:29.085762 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 75569511179b41038e521fa3346a92ae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 75569511179b41038e521fa3346a92ae already in progress: creating tablet
W20260202 02:52:29.085994 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet fe9ea4c0502f4976b440465992f2ea2c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fe9ea4c0502f4976b440465992f2ea2c already in progress: creating tablet
I20260202 02:52:29.086755 1882 raft_consensus.cc:2272] T c25e329023c04f609d8a743b4f7a95a4 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:29.087392 1882 ts_tablet_manager.cc:1916] T c25e329023c04f609d8a743b4f7a95a4 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:29.087579 1180 ts_tablet_manager.cc:1916] T ffd80badce1048f3ac95366cad05ae53 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:29.104704 1173 tablet_bootstrap.cc:492] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:29.104997 1173 ts_tablet_manager.cc:1403] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.007s
I20260202 02:52:29.105809 1173 raft_consensus.cc:359] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.106182 1173 raft_consensus.cc:385] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:29.106313 1173 raft_consensus.cc:740] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:29.106630 1173 consensus_queue.cc:260] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.110250 1173 ts_tablet_manager.cc:1434] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260202 02:52:29.110682 1173 tablet_bootstrap.cc:492] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:29.111608 1180 ts_tablet_manager.cc:1929] T ffd80badce1048f3ac95366cad05ae53 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:29.112694 1180 log.cc:1199] T ffd80badce1048f3ac95366cad05ae53 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/ffd80badce1048f3ac95366cad05ae53
I20260202 02:52:29.113266 1180 ts_tablet_manager.cc:1950] T ffd80badce1048f3ac95366cad05ae53 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:29.114238 1882 ts_tablet_manager.cc:1929] T c25e329023c04f609d8a743b4f7a95a4 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:29.114513 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 460) took 4429 ms. Trace:
I20260202 02:52:29.115607 1180 rpcz_store.cc:276] 0202 02:52:24.685218 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.685325 (+ 107us) service_pool.cc:225] Handling call
0202 02:52:29.083273 (+4397948us) ts_tablet_manager.cc:1151] Deleting tablet ffd80badce1048f3ac95366cad05ae53
0202 02:52:29.083281 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.101847 (+ 18566us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.111459 (+ 9612us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.114473 (+ 3014us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4393887}
I20260202 02:52:29.076766 1175 ts_tablet_manager.cc:1434] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.175s user 0.023s sys 0.006s
I20260202 02:52:29.116696 1175 tablet_bootstrap.cc:492] T 474df41e4fe24a818fa45a4ce84d5474 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:29.117446 1182 tablet_replica.cc:333] T 550895b4010f4088b2cc34acff50e03f P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:29.117630 1182 raft_consensus.cc:2243] T 550895b4010f4088b2cc34acff50e03f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:29.117784 1182 raft_consensus.cc:2272] T 550895b4010f4088b2cc34acff50e03f P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:29.118250 1175 tablet_bootstrap.cc:654] T 474df41e4fe24a818fa45a4ce84d5474 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:29.118602 1180 tablet_replica.cc:333] T a1f39e5b02f747819e636cddb4292eb6 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:29.119071 1180 raft_consensus.cc:2243] T a1f39e5b02f747819e636cddb4292eb6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:29.119560 1180 raft_consensus.cc:2272] T a1f39e5b02f747819e636cddb4292eb6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:29.120810 1180 ts_tablet_manager.cc:1916] T a1f39e5b02f747819e636cddb4292eb6 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:29.095391 2236 raft_consensus.cc:493] T 6df3d11cf578404ca41807f0777b6c53 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:29.122685 1882 log.cc:1199] T c25e329023c04f609d8a743b4f7a95a4 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/c25e329023c04f609d8a743b4f7a95a4
I20260202 02:52:29.122624 2236 raft_consensus.cc:515] T 6df3d11cf578404ca41807f0777b6c53 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:29.123214 1882 ts_tablet_manager.cc:1950] T c25e329023c04f609d8a743b4f7a95a4 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:29.123847 1058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6df3d11cf578404ca41807f0777b6c53" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:29.124076 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 450) took 4329 ms. Trace:
I20260202 02:52:29.124197 1882 rpcz_store.cc:276] 0202 02:52:24.794467 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.794897 (+ 430us) service_pool.cc:225] Handling call
0202 02:52:29.070788 (+4275891us) ts_tablet_manager.cc:1151] Deleting tablet c25e329023c04f609d8a743b4f7a95a4
0202 02:52:29.070798 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.097190 (+ 26392us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.114087 (+ 16897us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.124033 (+ 9946us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4275672}
W20260202 02:52:29.124518 859 leader_election.cc:343] T 6df3d11cf578404ca41807f0777b6c53 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:29.122473 1173 tablet_bootstrap.cc:654] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:29.124866 2236 leader_election.cc:290] T 6df3d11cf578404ca41807f0777b6c53 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:29.125515 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6df3d11cf578404ca41807f0777b6c53" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:29.125772 1882 tablet_replica.cc:333] T 042d1ebf357b4abe8814a407891ce265 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:29.125909 885 tablet_service.cc:1505] Processing CreateTablet for tablet c03a12f911084222be15ea6f482ebe32 (DEFAULT_TABLE table=default.Table_d2f92c0892494da18afb7e3336faec7a [id=a945d15ff17e430699e9c605c1b731e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.126003 1882 raft_consensus.cc:2243] T 042d1ebf357b4abe8814a407891ce265 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:29.126173 1882 raft_consensus.cc:2272] T 042d1ebf357b4abe8814a407891ce265 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:29.126400 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c03a12f911084222be15ea6f482ebe32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.126863 1175 tablet_bootstrap.cc:492] T 474df41e4fe24a818fa45a4ce84d5474 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:29.126849 1882 ts_tablet_manager.cc:1916] T 042d1ebf357b4abe8814a407891ce265 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:29.126994 1175 ts_tablet_manager.cc:1403] T 474df41e4fe24a818fa45a4ce84d5474 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.001s
I20260202 02:52:29.127660 1175 raft_consensus.cc:359] T 474df41e4fe24a818fa45a4ce84d5474 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:29.127815 1175 raft_consensus.cc:385] T 474df41e4fe24a818fa45a4ce84d5474 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:29.127867 1175 raft_consensus.cc:740] T 474df41e4fe24a818fa45a4ce84d5474 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:29.128050 1175 consensus_queue.cc:260] T 474df41e4fe24a818fa45a4ce84d5474 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:29.128428 1039 tablet_service.cc:1505] Processing CreateTablet for tablet c03a12f911084222be15ea6f482ebe32 (DEFAULT_TABLE table=default.Table_d2f92c0892494da18afb7e3336faec7a [id=a945d15ff17e430699e9c605c1b731e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.128569 1175 ts_tablet_manager.cc:1434] T 474df41e4fe24a818fa45a4ce84d5474 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:29.128824 1175 tablet_bootstrap.cc:492] T 842e54adae2a452a98521de24326eaff P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:29.129083 1041 tablet_service.cc:1505] Processing CreateTablet for tablet 37ca0cc961e447cba3f26d516c65f8e1 (DEFAULT_TABLE table=default.Table_b08cae12b7694236b6451641de70c882 [id=b2c3806bd75c43d9ae284898e6aea9ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.129334 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c03a12f911084222be15ea6f482ebe32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.129601 1042 tablet_service.cc:1505] Processing CreateTablet for tablet 7d716a4bc00047cbb7568d7ec6353750 (DEFAULT_TABLE table=default.Table_3295106ca36c404cb1acc75bd288dd40 [id=7d01d382fc944f75a0987031ea2610b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.129927 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d716a4bc00047cbb7568d7ec6353750. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.130434 1175 tablet_bootstrap.cc:654] T 842e54adae2a452a98521de24326eaff P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:29.132202 887 tablet_service.cc:1505] Processing CreateTablet for tablet 37ca0cc961e447cba3f26d516c65f8e1 (DEFAULT_TABLE table=default.Table_b08cae12b7694236b6451641de70c882 [id=b2c3806bd75c43d9ae284898e6aea9ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.132565 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37ca0cc961e447cba3f26d516c65f8e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.132562 1044 tablet_service.cc:1505] Processing CreateTablet for tablet e6e47f0a987441028616fa3410be2cd7 (DEFAULT_TABLE table=default.Table_65aa136d4433457aa4321915254761dc [id=c88e31afaf914f5aa881c9ea94096258]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.132929 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6e47f0a987441028616fa3410be2cd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.136677 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 81abd5ef66cc4f0bb37844c6ff5c256d (DEFAULT_TABLE table=default.Table_edaccffa45e248b587c41ba28dda9dbc [id=8d681f90242c4179bd40265cd815742a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.137106 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81abd5ef66cc4f0bb37844c6ff5c256d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.139040 896 tablet_service.cc:1505] Processing CreateTablet for tablet e6e47f0a987441028616fa3410be2cd7 (DEFAULT_TABLE table=default.Table_65aa136d4433457aa4321915254761dc [id=c88e31afaf914f5aa881c9ea94096258]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.139422 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6e47f0a987441028616fa3410be2cd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.140664 1033 tablet_service.cc:1505] Processing CreateTablet for tablet de76ca9a98c840d0967a97154b940b60 (DEFAULT_TABLE table=default.Table_f4a5e710b7a444bbbddef402779ebc8d [id=277a1bdd805e49a794c46be1f9a03316]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.141045 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de76ca9a98c840d0967a97154b940b60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.144060 1180 ts_tablet_manager.cc:1929] T a1f39e5b02f747819e636cddb4292eb6 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:29.144186 1180 log.cc:1199] T a1f39e5b02f747819e636cddb4292eb6 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/a1f39e5b02f747819e636cddb4292eb6
I20260202 02:52:29.144506 1180 ts_tablet_manager.cc:1950] T a1f39e5b02f747819e636cddb4292eb6 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:29.144649 1028 tablet_service.cc:1505] Processing CreateTablet for tablet 4453a3e7ef784a528d8683e28d14e604 (DEFAULT_TABLE table=default.Table_e6e3448ca5cf4a07a1493a341487187a [id=244c9ad3b684452f8f20de3126d3e7e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.145150 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4453a3e7ef784a528d8683e28d14e604. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.145314 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 462) took 4455 ms. Trace:
I20260202 02:52:29.145409 1180 rpcz_store.cc:276] 0202 02:52:24.689504 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.690319 (+ 815us) service_pool.cc:225] Handling call
0202 02:52:29.118528 (+4428209us) ts_tablet_manager.cc:1151] Deleting tablet a1f39e5b02f747819e636cddb4292eb6
0202 02:52:29.118542 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.141161 (+ 22619us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.143940 (+ 2779us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.145269 (+ 1329us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4410040}
I20260202 02:52:29.147826 519 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet fbcb4a522df648ba9542250450c08f1b (table default.Table_0cd8f9947fbd430f8992cc1543aed9f7 [id=163eefc3d38a4b28bce88b948cc5df3c]). Expected version 2 got 0
I20260202 02:52:29.148212 891 tablet_service.cc:1505] Processing CreateTablet for tablet 81abd5ef66cc4f0bb37844c6ff5c256d (DEFAULT_TABLE table=default.Table_edaccffa45e248b587c41ba28dda9dbc [id=8d681f90242c4179bd40265cd815742a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.148696 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81abd5ef66cc4f0bb37844c6ff5c256d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.149520 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 072bb7f0e4ab4935be49461bc8a3c163 (table default.Table_959d3e3da2814b3a986d29864e47246c [id=bdf2f19a439d4be3aa2362cfde38ab7f]) successfully deleted
I20260202 02:52:29.150190 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37ca0cc961e447cba3f26d516c65f8e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.153147 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet ffd80badce1048f3ac95366cad05ae53 (table default.Table_38c5bca682e343c1be1ebdc253c9e848 [id=6f44bec7f8ea4d2abed8c0fd603c6569]) successfully deleted
I20260202 02:52:29.145150 1882 ts_tablet_manager.cc:1929] T 042d1ebf357b4abe8814a407891ce265 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:29.153390 1882 log.cc:1199] T 042d1ebf357b4abe8814a407891ce265 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/042d1ebf357b4abe8814a407891ce265
I20260202 02:52:29.154153 1882 ts_tablet_manager.cc:1950] T 042d1ebf357b4abe8814a407891ce265 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:29.157156 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 449) took 4375 ms. Trace:
I20260202 02:52:29.157522 1882 rpcz_store.cc:276] 0202 02:52:24.781233 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.789708 (+ 8475us) service_pool.cc:225] Handling call
0202 02:52:29.125713 (+4336005us) ts_tablet_manager.cc:1151] Deleting tablet 042d1ebf357b4abe8814a407891ce265
0202 02:52:29.125721 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.138406 (+ 12685us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.145049 (+ 6643us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.157131 (+ 12082us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4330538}
I20260202 02:52:29.157624 887 tablet_service.cc:1505] Processing CreateTablet for tablet 7d716a4bc00047cbb7568d7ec6353750 (DEFAULT_TABLE table=default.Table_3295106ca36c404cb1acc75bd288dd40 [id=7d01d382fc944f75a0987031ea2610b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.158047 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d716a4bc00047cbb7568d7ec6353750. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.158692 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 451) took 4332 ms. Trace:
I20260202 02:52:29.159111 1882 rpcz_store.cc:276] 0202 02:52:24.826380 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.828852 (+ 2472us) service_pool.cc:225] Handling call
0202 02:52:29.158545 (+4329693us) ts_tablet_manager.cc:1151] Deleting tablet cd1b6662da254889ae4bbe8d0b1ef1bb
0202 02:52:29.158556 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.158677 (+ 121us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4329315}
W20260202 02:52:29.159855 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet cd1b6662da254889ae4bbe8d0b1ef1bb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cd1b6662da254889ae4bbe8d0b1ef1bb already in progress: creating tablet
I20260202 02:52:29.160379 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 452) took 4243 ms. Trace:
I20260202 02:52:29.160851 1882 rpcz_store.cc:276] 0202 02:52:24.916800 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.916856 (+ 56us) service_pool.cc:225] Handling call
0202 02:52:29.160229 (+4243373us) ts_tablet_manager.cc:1151] Deleting tablet c977c146bac34638874b7a990604a0d5
0202 02:52:29.160239 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.160358 (+ 119us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4243167}
W20260202 02:52:29.162367 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet c977c146bac34638874b7a990604a0d5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c977c146bac34638874b7a990604a0d5 already in progress: creating tablet
I20260202 02:52:29.162791 1882 tablet_replica.cc:333] T 2f0e8c5e2fd44370ae150a206f4e6cbc P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:29.162989 902 tablet_service.cc:1505] Processing CreateTablet for tablet de76ca9a98c840d0967a97154b940b60 (DEFAULT_TABLE table=default.Table_f4a5e710b7a444bbbddef402779ebc8d [id=277a1bdd805e49a794c46be1f9a03316]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.163378 1882 raft_consensus.cc:2243] T 2f0e8c5e2fd44370ae150a206f4e6cbc P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:29.163455 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de76ca9a98c840d0967a97154b940b60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.163780 1882 raft_consensus.cc:2272] T 2f0e8c5e2fd44370ae150a206f4e6cbc P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:29.164958 1882 ts_tablet_manager.cc:1916] T 2f0e8c5e2fd44370ae150a206f4e6cbc P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:29.172572 857 leader_election.cc:343] T 6df3d11cf578404ca41807f0777b6c53 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:29.172825 857 leader_election.cc:304] T 6df3d11cf578404ca41807f0777b6c53 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:29.174103 902 tablet_service.cc:1505] Processing CreateTablet for tablet 4453a3e7ef784a528d8683e28d14e604 (DEFAULT_TABLE table=default.Table_e6e3448ca5cf4a07a1493a341487187a [id=244c9ad3b684452f8f20de3126d3e7e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.174396 2236 raft_consensus.cc:2749] T 6df3d11cf578404ca41807f0777b6c53 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:29.177825 1882 ts_tablet_manager.cc:1929] T 2f0e8c5e2fd44370ae150a206f4e6cbc P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:29.179782 1882 log.cc:1199] T 2f0e8c5e2fd44370ae150a206f4e6cbc P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/2f0e8c5e2fd44370ae150a206f4e6cbc
I20260202 02:52:29.180434 1882 ts_tablet_manager.cc:1950] T 2f0e8c5e2fd44370ae150a206f4e6cbc P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:29.181634 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 453) took 4215 ms. Trace:
I20260202 02:52:29.182574 1882 rpcz_store.cc:276] 0202 02:52:24.966162 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.966466 (+ 304us) service_pool.cc:225] Handling call
0202 02:52:29.162741 (+4196275us) ts_tablet_manager.cc:1151] Deleting tablet 2f0e8c5e2fd44370ae150a206f4e6cbc
0202 02:52:29.162751 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.170279 (+ 7528us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.177710 (+ 7431us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.181614 (+ 3904us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4195985}
I20260202 02:52:29.183743 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 27158aa96e2742d5b0b2c3fa9531178d (DEFAULT_TABLE table=default.Table_cba18d80d56e482f84815b720dc2a83f [id=dfed6a73db7b4872a7e2915d85c317cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.183988 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 2f0e8c5e2fd44370ae150a206f4e6cbc (table default.Table_04337834a3ce43ae8f6012b818d70d9f [id=3d1bb64023a145bb9fdcb2378a844a37]) successfully deleted
I20260202 02:52:29.184130 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27158aa96e2742d5b0b2c3fa9531178d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.184484 1882 tablet_replica.cc:333] T 59990791123c4d4eb620090a1e8e31e1 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:29.185106 1882 raft_consensus.cc:2243] T 59990791123c4d4eb620090a1e8e31e1 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:29.185395 1882 raft_consensus.cc:2272] T 59990791123c4d4eb620090a1e8e31e1 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:29.186759 1882 ts_tablet_manager.cc:1916] T 59990791123c4d4eb620090a1e8e31e1 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:29.190024 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4453a3e7ef784a528d8683e28d14e604. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.118254 1182 ts_tablet_manager.cc:1916] T 550895b4010f4088b2cc34acff50e03f P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:29.199154 525 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 16b47a2831f54d809ed0b16deaec7e3d (table default.Table_e4018d1c57714a24be5d3a20c1ccaef4 [id=1e291b35a6764638acfb781bb4e6adb3]). Expected version 2 got 0
I20260202 02:52:29.200824 525 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet be993c1512b04df4b92cab3195241d84 (table default.Table_dd517ae94bf54267bc3a03dee4896708 [id=22f1ff6e13a84a72820c887dd3107f49]). Expected version 1 got 0
I20260202 02:52:29.201069 885 tablet_service.cc:1505] Processing CreateTablet for tablet 27158aa96e2742d5b0b2c3fa9531178d (DEFAULT_TABLE table=default.Table_cba18d80d56e482f84815b720dc2a83f [id=dfed6a73db7b4872a7e2915d85c317cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.201498 1882 ts_tablet_manager.cc:1929] T 59990791123c4d4eb620090a1e8e31e1 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:29.201592 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27158aa96e2742d5b0b2c3fa9531178d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.202430 1882 log.cc:1199] T 59990791123c4d4eb620090a1e8e31e1 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/59990791123c4d4eb620090a1e8e31e1
I20260202 02:52:29.202975 1882 ts_tablet_manager.cc:1950] T 59990791123c4d4eb620090a1e8e31e1 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:29.204820 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 454) took 4238 ms. Trace:
I20260202 02:52:29.205302 1882 rpcz_store.cc:276] 0202 02:52:24.966299 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.966796 (+ 497us) service_pool.cc:225] Handling call
0202 02:52:29.184443 (+4217647us) ts_tablet_manager.cc:1151] Deleting tablet 59990791123c4d4eb620090a1e8e31e1
0202 02:52:29.184452 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.192787 (+ 8335us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.201384 (+ 8597us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.204794 (+ 3410us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4217491}
I20260202 02:52:29.206221 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 59990791123c4d4eb620090a1e8e31e1 (table default.Table_26642341d3d54c0fa9af19fef3a5b06d [id=9ff2eb6ee89943918019f5e899ac362b]) successfully deleted
I20260202 02:52:29.206808 1882 tablet_replica.cc:333] T d0671586e45e45ffb7fd9e1be6e2bbf8 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:29.207317 1882 raft_consensus.cc:2243] T d0671586e45e45ffb7fd9e1be6e2bbf8 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:29.207633 2127 raft_consensus.cc:493] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:29.207788 2127 raft_consensus.cc:515] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.208458 2127 leader_election.cc:290] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:29.209125 1882 raft_consensus.cc:2272] T d0671586e45e45ffb7fd9e1be6e2bbf8 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:29.209895 999 leader_election.cc:343] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 02b434fa058b45529ff3938b053263c6
I20260202 02:52:29.211050 1882 ts_tablet_manager.cc:1916] T d0671586e45e45ffb7fd9e1be6e2bbf8 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:29.216991 1882 ts_tablet_manager.cc:1929] T d0671586e45e45ffb7fd9e1be6e2bbf8 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:29.220981 1175 tablet_bootstrap.cc:492] T 842e54adae2a452a98521de24326eaff P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:29.221144 1175 ts_tablet_manager.cc:1403] T 842e54adae2a452a98521de24326eaff P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.092s user 0.010s sys 0.032s
I20260202 02:52:29.222007 1175 raft_consensus.cc:359] T 842e54adae2a452a98521de24326eaff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:29.222182 1175 raft_consensus.cc:385] T 842e54adae2a452a98521de24326eaff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:29.222229 1175 raft_consensus.cc:740] T 842e54adae2a452a98521de24326eaff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:29.222429 1175 consensus_queue.cc:260] T 842e54adae2a452a98521de24326eaff P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:29.223624 1175 ts_tablet_manager.cc:1434] T 842e54adae2a452a98521de24326eaff P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:29.223815 1882 log.cc:1199] T d0671586e45e45ffb7fd9e1be6e2bbf8 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/d0671586e45e45ffb7fd9e1be6e2bbf8
I20260202 02:52:29.224201 1175 tablet_bootstrap.cc:492] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:29.224634 1882 ts_tablet_manager.cc:1950] T d0671586e45e45ffb7fd9e1be6e2bbf8 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:29.225871 1175 tablet_bootstrap.cc:654] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:29.225903 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 455) took 4259 ms. Trace:
I20260202 02:52:29.226738 1882 rpcz_store.cc:276] 0202 02:52:24.966415 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.966925 (+ 510us) service_pool.cc:225] Handling call
0202 02:52:29.206737 (+4239812us) ts_tablet_manager.cc:1151] Deleting tablet d0671586e45e45ffb7fd9e1be6e2bbf8
0202 02:52:29.206755 (+ 18us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.214123 (+ 7368us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.216880 (+ 2757us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.225883 (+ 9003us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4239606}
I20260202 02:52:29.227478 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet d0671586e45e45ffb7fd9e1be6e2bbf8 (table default.Table_53cc5731c3e24222a97702bdde84a954 [id=b2fe4f8ae50642e9a6ea05759313b31b]) successfully deleted
I20260202 02:52:29.228009 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 465) took 4127 ms. Trace:
I20260202 02:52:29.228415 1882 rpcz_store.cc:276] 0202 02:52:25.101020 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:25.101089 (+ 69us) service_pool.cc:225] Handling call
0202 02:52:29.227874 (+4126785us) ts_tablet_manager.cc:1151] Deleting tablet a5f864fd8a3d4c4a83e7f68abb8a0a83
0202 02:52:29.227885 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.227992 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4126534}
W20260202 02:52:29.229115 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet a5f864fd8a3d4c4a83e7f68abb8a0a83 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a5f864fd8a3d4c4a83e7f68abb8a0a83 already in progress: creating tablet
I20260202 02:52:29.229571 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 474) took 3993 ms. Trace:
I20260202 02:52:29.229936 1882 rpcz_store.cc:276] 0202 02:52:25.236486 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:25.242106 (+ 5620us) service_pool.cc:225] Handling call
0202 02:52:29.229463 (+3987357us) ts_tablet_manager.cc:1151] Deleting tablet ccd6bc8bd0504bf18d108c73685ef9ff
0202 02:52:29.229473 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.229556 (+ 83us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3987123}
I20260202 02:52:29.230819 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 475) took 3960 ms. Trace:
I20260202 02:52:29.231204 1882 rpcz_store.cc:276] 0202 02:52:25.270608 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:25.273213 (+ 2605us) service_pool.cc:225] Handling call
0202 02:52:29.230716 (+3957503us) ts_tablet_manager.cc:1151] Deleting tablet 361c0c3832614ec78695c28720b841e0
0202 02:52:29.230726 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.230806 (+ 80us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3925205}
W20260202 02:52:29.231844 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 361c0c3832614ec78695c28720b841e0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 361c0c3832614ec78695c28720b841e0 already in progress: creating tablet
I20260202 02:52:29.232261 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 476) took 3934 ms. Trace:
I20260202 02:52:29.232640 1882 rpcz_store.cc:276] 0202 02:52:25.302576 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:25.302797 (+ 221us) service_pool.cc:225] Handling call
0202 02:52:29.232162 (+3929365us) ts_tablet_manager.cc:1151] Deleting tablet 7f9dfac19c04475aadb45e95a6474eb4
0202 02:52:29.232171 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.232247 (+ 76us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3925693}
W20260202 02:52:29.233306 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 7f9dfac19c04475aadb45e95a6474eb4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7f9dfac19c04475aadb45e95a6474eb4 already in progress: creating tablet
I20260202 02:52:29.233723 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 477) took 3928 ms. Trace:
I20260202 02:52:29.234062 1882 rpcz_store.cc:276] 0202 02:52:25.305724 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:25.308065 (+ 2341us) service_pool.cc:225] Handling call
0202 02:52:29.233612 (+3925547us) ts_tablet_manager.cc:1151] Deleting tablet a0b5d82e5fb94b60b4aa324b978a5378
0202 02:52:29.233621 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.233708 (+ 87us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3920025}
W20260202 02:52:29.234671 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet a0b5d82e5fb94b60b4aa324b978a5378 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a0b5d82e5fb94b60b4aa324b978a5378 already in progress: creating tablet
I20260202 02:52:29.235092 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 478) took 3923 ms. Trace:
I20260202 02:52:29.235417 1882 rpcz_store.cc:276] 0202 02:52:25.311333 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:25.311388 (+ 55us) service_pool.cc:225] Handling call
0202 02:52:29.234996 (+3923608us) ts_tablet_manager.cc:1151] Deleting tablet ac2ab12f5040488aa306cb69b107b485
0202 02:52:29.235004 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.235078 (+ 74us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3882332}
I20260202 02:52:29.235736 2127 raft_consensus.cc:493] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:29.236016 2127 raft_consensus.cc:515] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.236619 2127 leader_election.cc:290] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:29.237794 999 leader_election.cc:343] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 75569511179b41038e521fa3346a92ae
I20260202 02:52:29.154235 750 tablet_service.cc:1505] Processing CreateTablet for tablet 7d716a4bc00047cbb7568d7ec6353750 (DEFAULT_TABLE table=default.Table_3295106ca36c404cb1acc75bd288dd40 [id=7d01d382fc944f75a0987031ea2610b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.241823 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d716a4bc00047cbb7568d7ec6353750. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.154264 744 tablet_service.cc:1505] Processing CreateTablet for tablet 81abd5ef66cc4f0bb37844c6ff5c256d (DEFAULT_TABLE table=default.Table_edaccffa45e248b587c41ba28dda9dbc [id=8d681f90242c4179bd40265cd815742a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.244647 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81abd5ef66cc4f0bb37844c6ff5c256d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.154596 742 tablet_service.cc:1505] Processing CreateTablet for tablet c03a12f911084222be15ea6f482ebe32 (DEFAULT_TABLE table=default.Table_d2f92c0892494da18afb7e3336faec7a [id=a945d15ff17e430699e9c605c1b731e3]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:29.247288 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet ac2ab12f5040488aa306cb69b107b485 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ac2ab12f5040488aa306cb69b107b485 already in progress: creating tablet
I20260202 02:52:29.247596 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c03a12f911084222be15ea6f482ebe32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.247787 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 480) took 3900 ms. Trace:
I20260202 02:52:29.247875 1882 rpcz_store.cc:276] 0202 02:52:25.347464 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:25.349977 (+ 2513us) service_pool.cc:225] Handling call
0202 02:52:29.247642 (+3897665us) ts_tablet_manager.cc:1151] Deleting tablet 3706562a22634eb6b72e36bb6a2d1ae5
0202 02:52:29.247654 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.247766 (+ 112us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3887128}
W20260202 02:52:29.248358 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 3706562a22634eb6b72e36bb6a2d1ae5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3706562a22634eb6b72e36bb6a2d1ae5 already in progress: creating tablet
I20260202 02:52:29.249073 1175 tablet_bootstrap.cc:492] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:29.249199 1175 ts_tablet_manager.cc:1403] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.025s user 0.013s sys 0.005s
I20260202 02:52:29.249243 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 482) took 3901 ms. Trace:
I20260202 02:52:29.249316 1882 rpcz_store.cc:276] 0202 02:52:25.347698 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:25.350014 (+ 2316us) service_pool.cc:225] Handling call
0202 02:52:29.249129 (+3899115us) ts_tablet_manager.cc:1151] Deleting tablet 474df41e4fe24a818fa45a4ce84d5474
0202 02:52:29.249140 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.249226 (+ 86us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3888601}
W20260202 02:52:29.249701 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 474df41e4fe24a818fa45a4ce84d5474 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 474df41e4fe24a818fa45a4ce84d5474 already in progress: creating tablet
I20260202 02:52:29.249877 1175 raft_consensus.cc:359] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.250041 1175 raft_consensus.cc:385] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:29.250087 1175 raft_consensus.cc:740] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:29.154632 752 tablet_service.cc:1505] Processing CreateTablet for tablet e6e47f0a987441028616fa3410be2cd7 (DEFAULT_TABLE table=default.Table_65aa136d4433457aa4321915254761dc [id=c88e31afaf914f5aa881c9ea94096258]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.250275 1175 consensus_queue.cc:260] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.250231 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_da4a1a30b6e34996a02ab80a072959d4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:29.250463 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 481) took 3902 ms. Trace:
I20260202 02:52:29.250510 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6e47f0a987441028616fa3410be2cd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.250528 1882 rpcz_store.cc:276] 0202 02:52:25.347602 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:25.350036 (+ 2434us) service_pool.cc:225] Handling call
0202 02:52:29.250357 (+3900321us) ts_tablet_manager.cc:1151] Deleting tablet 7921daa000f54c8dbed04473e21cf0a5
0202 02:52:29.250367 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.250446 (+ 79us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3877201}
I20260202 02:52:29.250838 1175 ts_tablet_manager.cc:1434] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
W20260202 02:52:29.250867 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_da4a1a30b6e34996a02ab80a072959d4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:29.250947 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 7921daa000f54c8dbed04473e21cf0a5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7921daa000f54c8dbed04473e21cf0a5 already in progress: creating tablet
I20260202 02:52:29.251111 1175 tablet_bootstrap.cc:492] T 1f379a29e2454f689d68f5951837512b P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:29.252000 1882 tablet_replica.cc:333] T 79f694b740954d429145fba9157338ca P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:29.252187 1882 raft_consensus.cc:2243] T 79f694b740954d429145fba9157338ca P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:29.252123 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_5201d08d823b4303b0be4f42a443dd2d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:29.252346 1882 raft_consensus.cc:2272] T 79f694b740954d429145fba9157338ca P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:29.252511 1175 tablet_bootstrap.cc:654] T 1f379a29e2454f689d68f5951837512b P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:29.252696 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5201d08d823b4303b0be4f42a443dd2d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:29.252866 1882 ts_tablet_manager.cc:1916] T 79f694b740954d429145fba9157338ca P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:29.254078 520 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_026b5574b3af4ff08f92f7ef3b3025ff" } new_table_name: "default.Table_23afb316ffac47aea4db07bda15a956b" modify_external_catalogs: true
I20260202 02:52:29.154963 759 tablet_service.cc:1505] Processing CreateTablet for tablet 37ca0cc961e447cba3f26d516c65f8e1 (DEFAULT_TABLE table=default.Table_b08cae12b7694236b6451641de70c882 [id=b2c3806bd75c43d9ae284898e6aea9ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.254617 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37ca0cc961e447cba3f26d516c65f8e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.258416 522 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_12c352712c374a45b7de7a0e634a1327 [id=feb8b9f926434cec8b119843413f8c9b] alter complete (version 1)
I20260202 02:52:29.260157 1175 tablet_bootstrap.cc:492] T 1f379a29e2454f689d68f5951837512b P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:29.260291 1175 ts_tablet_manager.cc:1403] T 1f379a29e2454f689d68f5951837512b P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260202 02:52:29.155241 755 tablet_service.cc:1505] Processing CreateTablet for tablet de76ca9a98c840d0967a97154b940b60 (DEFAULT_TABLE table=default.Table_f4a5e710b7a444bbbddef402779ebc8d [id=277a1bdd805e49a794c46be1f9a03316]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.155579 747 tablet_service.cc:1505] Processing CreateTablet for tablet 4453a3e7ef784a528d8683e28d14e604 (DEFAULT_TABLE table=default.Table_e6e3448ca5cf4a07a1493a341487187a [id=244c9ad3b684452f8f20de3126d3e7e2]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:29.170972 715 consensus_peers.cc:597] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d67e91a022f1401ea724dff898c3b81b. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:29.182765 756 tablet_service.cc:1505] Processing CreateTablet for tablet 27158aa96e2742d5b0b2c3fa9531178d (DEFAULT_TABLE table=default.Table_cba18d80d56e482f84815b720dc2a83f [id=dfed6a73db7b4872a7e2915d85c317cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.209381 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "02b434fa058b45529ff3938b053263c6" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:29.260618 715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 89695.7us
W20260202 02:52:29.261222 998 leader_election.cc:343] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:29.261375 998 leader_election.cc:304] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:29.261327 1175 raft_consensus.cc:359] T 1f379a29e2454f689d68f5951837512b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.261489 1175 raft_consensus.cc:385] T 1f379a29e2454f689d68f5951837512b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:29.261538 1175 raft_consensus.cc:740] T 1f379a29e2454f689d68f5951837512b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:29.261765 1721 raft_consensus.cc:2749] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:29.261719 1175 consensus_queue.cc:260] T 1f379a29e2454f689d68f5951837512b P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.262399 1175 ts_tablet_manager.cc:1434] T 1f379a29e2454f689d68f5951837512b P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:29.262454 519 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ed8a8cefa5de4d90a2d57043595329ae" } new_table_name: "default.Table_7655d1ccf43246cfbeaf0c7b30fc5d32" modify_external_catalogs: true
I20260202 02:52:29.262713 1175 tablet_bootstrap.cc:492] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:29.264094 1175 tablet_bootstrap.cc:654] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:29.265189 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 2b8ba9dd450948ada395207ae1c5b325 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:29.265254 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 13cd1383348b458894fdecc22bf24b74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:29.265458 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 2c31b45dc8004eae993e972e9faa3ff8 with delete_type TABLET_DATA_DELETED (replaced by tablet aed1081136f7431b9b14784c21112d1c) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:29.265746 522 master_service.cc:954] ReplaceTablet: received request to replace tablet 3d1a7578e83447669e3c3972a426ad44 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:29.217480 2150 raft_consensus.cc:493] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:29.266290 2150 raft_consensus.cc:515] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:29.267180 2150 leader_election.cc:290] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:29.237308 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "75569511179b41038e521fa3346a92ae" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:29.238397 1174 tablet_bootstrap.cc:492] T 16b47a2831f54d809ed0b16deaec7e3d P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:29.267457 1174 ts_tablet_manager.cc:1403] T 16b47a2831f54d809ed0b16deaec7e3d P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.244s user 0.038s sys 0.041s
I20260202 02:52:29.241291 2189 raft_consensus.cc:493] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:29.268342 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 38fdbc6df7ec40b9afe4e0dfe0119e3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:29.268435 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 26eeef6eebbf4f2cb44f69255fb37594 with delete_type TABLET_DATA_DELETED (replaced by tablet dc4c3f52f4f44a84b0d0e3c9a4a49f09) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:29.268448 2189 raft_consensus.cc:515] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:29.268602 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 38301e13e17c41fcae912c4b38c26a45 with delete_type TABLET_DATA_DELETED (replaced by tablet 79badde1001b432b95f58b5161ace15c) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:29.269024 2189 leader_election.cc:290] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:29.269179 998 leader_election.cc:343] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:29.268230 1174 raft_consensus.cc:359] T 16b47a2831f54d809ed0b16deaec7e3d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.269296 998 leader_election.cc:304] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:29.269393 1174 raft_consensus.cc:385] T 16b47a2831f54d809ed0b16deaec7e3d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:29.269479 1174 raft_consensus.cc:740] T 16b47a2831f54d809ed0b16deaec7e3d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:29.269654 1721 raft_consensus.cc:2749] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:29.269676 1174 consensus_queue.cc:260] T 16b47a2831f54d809ed0b16deaec7e3d P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.269937 922 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cb634d4bae554ef3b39eeaa9958adc6c" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:29.270246 1174 ts_tablet_manager.cc:1434] T 16b47a2831f54d809ed0b16deaec7e3d P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
W20260202 02:52:29.270479 714 leader_election.cc:343] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 38301e13e17c41fcae912c4b38c26a45
I20260202 02:52:29.271556 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "38301e13e17c41fcae912c4b38c26a45" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:29.271723 1033 tablet_service.cc:1552] Processing DeleteTablet for tablet 042d1ebf357b4abe8814a407891ce265 with delete_type TABLET_DATA_DELETED (replaced by tablet 2c2a535de2294cc697d75561d0463642) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:29.271777 1062 raft_consensus.cc:2468] T 38301e13e17c41fcae912c4b38c26a45 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:29.271919 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 1f54c1dfe0a942b3bafd85522953a032 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:29.272079 1058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cb634d4bae554ef3b39eeaa9958adc6c" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:29.272614 896 tablet_service.cc:1552] Processing DeleteTablet for tablet 1116872cc25a4ab5b1a3a6bbba1f0da7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:25 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:29.273994 715 leader_election.cc:304] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:29.274518 715 leader_election.cc:304] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:29.274930 2189 raft_consensus.cc:2804] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:29.275023 2189 raft_consensus.cc:493] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:29.275102 2189 raft_consensus.cc:3060] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:29.275243 1882 ts_tablet_manager.cc:1929] T 79f694b740954d429145fba9157338ca P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:29.275345 1882 log.cc:1199] T 79f694b740954d429145fba9157338ca P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/79f694b740954d429145fba9157338ca
I20260202 02:52:29.275600 1882 ts_tablet_manager.cc:1950] T 79f694b740954d429145fba9157338ca P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:29.276428 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 479) took 3964 ms. Trace:
I20260202 02:52:29.276510 1882 rpcz_store.cc:276] 0202 02:52:25.312274 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:25.313972 (+ 1698us) service_pool.cc:225] Handling call
0202 02:52:29.251952 (+3937980us) ts_tablet_manager.cc:1151] Deleting tablet 79f694b740954d429145fba9157338ca
0202 02:52:29.251962 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.257060 (+ 5098us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.275121 (+ 18061us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.276403 (+ 1282us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3829905}
I20260202 02:52:29.276993 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 79f694b740954d429145fba9157338ca (table default.Table_662111850b514df5ad7c69aa21783397 [id=d595db36d702401ab81d5401ad3a76e2]) successfully deleted
I20260202 02:52:29.278064 2189 raft_consensus.cc:515] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:29.278812 2189 leader_election.cc:290] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:29.279029 2189 raft_consensus.cc:2749] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:29.279412 2236 raft_consensus.cc:493] T d8418a9dd0944456b4d4820dde0cde02 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:29.279539 2236 raft_consensus.cc:515] T d8418a9dd0944456b4d4820dde0cde02 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:29.280061 1882 tablet_replica.cc:333] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:29.280241 1882 raft_consensus.cc:2243] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:29.280294 2236 leader_election.cc:290] T d8418a9dd0944456b4d4820dde0cde02 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:29.281710 1882 pending_rounds.cc:70] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d: Trying to abort 1 pending ops.
I20260202 02:52:29.281780 1882 pending_rounds.cc:77] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923053020717056 op_type: NO_OP noop_request { }
I20260202 02:52:29.281858 1882 raft_consensus.cc:2889] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:29.281945 1882 raft_consensus.cc:2272] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:29.283150 1882 ts_tablet_manager.cc:1916] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:29.283923 714 leader_election.cc:343] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 38301e13e17c41fcae912c4b38c26a45
I20260202 02:52:29.284236 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 6df3d11cf578404ca41807f0777b6c53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:29.286494 1058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d8418a9dd0944456b4d4820dde0cde02" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:29.295049 1882 ts_tablet_manager.cc:1929] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260202 02:52:29.297770 859 leader_election.cc:343] T d8418a9dd0944456b4d4820dde0cde02 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:29.298842 527 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 7665dbea16d14f619af57ea056639eef
I20260202 02:52:29.299232 1882 log.cc:1199] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/a5622ef925bc46cca76f481ddfd12afd
I20260202 02:52:29.296985 1058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "38301e13e17c41fcae912c4b38c26a45" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:29.301961 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 7665dbea16d14f619af57ea056639eef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:28 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:29.302090 902 tablet_service.cc:1552] Processing DeleteTablet for tablet 7665dbea16d14f619af57ea056639eef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:28 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:29.301352 1882 ts_tablet_manager.cc:1950] T a5622ef925bc46cca76f481ddfd12afd P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:29.302934 528 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 842197c689064660b791b40e521d346b
I20260202 02:52:29.301373 1058 raft_consensus.cc:3060] T 38301e13e17c41fcae912c4b38c26a45 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:29.303761 528 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_94b42fda242b4e46a0aadf9d900c1395" } new_table_name: "default.Table_616a8ade4b5e481b8d4906b7a2a026d0" modify_external_catalogs: true
I20260202 02:52:29.305254 902 tablet_service.cc:1552] Processing DeleteTablet for tablet 842197c689064660b791b40e521d346b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:28 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:29.305814 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 842197c689064660b791b40e521d346b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:28 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:29.306129 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 7665dbea16d14f619af57ea056639eef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:28 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:29.306285 1058 raft_consensus.cc:2468] T 38301e13e17c41fcae912c4b38c26a45 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:29.306169 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 486) took 3885 ms. Trace:
I20260202 02:52:29.306547 1882 rpcz_store.cc:276] 0202 02:52:25.421056 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:25.422531 (+ 1475us) service_pool.cc:225] Handling call
0202 02:52:29.280010 (+3857479us) ts_tablet_manager.cc:1151] Deleting tablet a5622ef925bc46cca76f481ddfd12afd
0202 02:52:29.280019 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.287359 (+ 7340us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.294935 (+ 7576us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.306151 (+ 11216us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3832266}
I20260202 02:52:29.306878 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 842197c689064660b791b40e521d346b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:28 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:29.307488 715 leader_election.cc:304] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:29.307523 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet a5622ef925bc46cca76f481ddfd12afd (table default.Table_3d26f7a34d0c48379e3cebda66e1ef13 [id=45e6d83589fa493195b06f633e295b00]) successfully deleted
I20260202 02:52:29.308013 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 487) took 3695 ms. Trace:
I20260202 02:52:29.308064 2150 raft_consensus.cc:2804] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:29.308214 2150 raft_consensus.cc:697] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:29.308492 1882 rpcz_store.cc:276] 0202 02:52:25.612540 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:25.612595 (+ 55us) service_pool.cc:225] Handling call
0202 02:52:29.307855 (+3695260us) ts_tablet_manager.cc:1151] Deleting tablet 93f958924885457ab15f2aac5509e0aa
0202 02:52:29.307871 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.307993 (+ 122us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3694981}
I20260202 02:52:29.308557 2150 consensus_queue.cc:237] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:29.309015 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 93f958924885457ab15f2aac5509e0aa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 93f958924885457ab15f2aac5509e0aa already in progress: creating tablet
I20260202 02:52:29.310304 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 488) took 3409 ms. Trace:
I20260202 02:52:29.310380 1882 rpcz_store.cc:276] 0202 02:52:25.900503 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:25.900565 (+ 62us) service_pool.cc:225] Handling call
0202 02:52:29.310180 (+3409615us) ts_tablet_manager.cc:1151] Deleting tablet 47e50559a7c04ae080fcc67342c631c9
0202 02:52:29.310192 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.310287 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3409347}
W20260202 02:52:29.310823 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 47e50559a7c04ae080fcc67342c631c9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 47e50559a7c04ae080fcc67342c631c9 already in progress: creating tablet
I20260202 02:52:29.311615 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 500) took 3139 ms. Trace:
I20260202 02:52:29.312029 1882 rpcz_store.cc:276] 0202 02:52:26.171909 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.176376 (+ 4467us) service_pool.cc:225] Handling call
0202 02:52:29.311498 (+3135122us) ts_tablet_manager.cc:1151] Deleting tablet f6c2bc531d194a4b8f3062c9b40e3da6
0202 02:52:29.311509 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.311598 (+ 89us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3134619}
W20260202 02:52:29.312708 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet f6c2bc531d194a4b8f3062c9b40e3da6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f6c2bc531d194a4b8f3062c9b40e3da6 already in progress: creating tablet
I20260202 02:52:29.313148 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 501) took 3141 ms. Trace:
I20260202 02:52:29.313493 1882 rpcz_store.cc:276] 0202 02:52:26.172049 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.176930 (+ 4881us) service_pool.cc:225] Handling call
0202 02:52:29.313041 (+3136111us) ts_tablet_manager.cc:1151] Deleting tablet 8ab5c6fbec394b9bab322a197181a618
0202 02:52:29.313050 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.313132 (+ 82us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3135901}
I20260202 02:52:29.314348 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 496) took 3193 ms. Trace:
I20260202 02:52:29.314713 1882 rpcz_store.cc:276] 0202 02:52:26.120874 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.120949 (+ 75us) service_pool.cc:225] Handling call
0202 02:52:29.314246 (+3193297us) ts_tablet_manager.cc:1151] Deleting tablet 5e20842de8cb4355b732685605c88834
0202 02:52:29.314256 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.314333 (+ 77us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3135996}
W20260202 02:52:29.315459 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 5e20842de8cb4355b732685605c88834 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5e20842de8cb4355b732685605c88834 already in progress: creating tablet
I20260202 02:52:29.316218 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 497) took 3190 ms. Trace:
I20260202 02:52:29.316542 1882 rpcz_store.cc:276] 0202 02:52:26.126095 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.126174 (+ 79us) service_pool.cc:225] Handling call
0202 02:52:29.315784 (+3189610us) ts_tablet_manager.cc:1151] Deleting tablet 0ddc726efd3f49489092919c9c6e01fb
0202 02:52:29.315793 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.316189 (+ 396us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3137269}
W20260202 02:52:29.317232 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 0ddc726efd3f49489092919c9c6e01fb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0ddc726efd3f49489092919c9c6e01fb already in progress: creating tablet
I20260202 02:52:29.317963 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 498) took 3184 ms. Trace:
I20260202 02:52:29.318316 1882 rpcz_store.cc:276] 0202 02:52:26.133687 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.133750 (+ 63us) service_pool.cc:225] Handling call
0202 02:52:29.317857 (+3184107us) ts_tablet_manager.cc:1151] Deleting tablet 1116872cc25a4ab5b1a3a6bbba1f0da7
0202 02:52:29.317866 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.317947 (+ 81us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3138160}
I20260202 02:52:29.318905 772 raft_consensus.cc:3060] T 16b47a2831f54d809ed0b16deaec7e3d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:29.321586 524 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet e7df9a78735547cbbaa006c604b95fec
I20260202 02:52:29.323146 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d8418a9dd0944456b4d4820dde0cde02" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:29.323362 524 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_06273452ab3e4d2e8bdb1e3a41721dcc" } new_table_name: "default.Table_66adee49a1b045918360d13af6bca9a2" modify_external_catalogs: true
I20260202 02:52:29.323429 902 tablet_service.cc:1552] Processing DeleteTablet for tablet e7df9a78735547cbbaa006c604b95fec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:28 UTC) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:29.323701 857 leader_election.cc:343] T d8418a9dd0944456b4d4820dde0cde02 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:29.323813 857 leader_election.cc:304] T d8418a9dd0944456b4d4820dde0cde02 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:29.323664 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet e7df9a78735547cbbaa006c604b95fec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:28 UTC) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:29.324267 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 1116872cc25a4ab5b1a3a6bbba1f0da7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1116872cc25a4ab5b1a3a6bbba1f0da7 already in progress: creating tablet
I20260202 02:52:29.324700 2170 raft_consensus.cc:2749] T d8418a9dd0944456b4d4820dde0cde02 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:29.324723 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 504) took 3112 ms. Trace:
I20260202 02:52:29.324795 1882 rpcz_store.cc:276] 0202 02:52:26.211895 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.211962 (+ 67us) service_pool.cc:225] Handling call
0202 02:52:29.324580 (+3112618us) ts_tablet_manager.cc:1151] Deleting tablet 2616e1bbce4049dbb584550ccc150288
0202 02:52:29.324593 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.324705 (+ 112us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3111837}
W20260202 02:52:29.325158 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 2616e1bbce4049dbb584550ccc150288 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2616e1bbce4049dbb584550ccc150288 already in progress: creating tablet
I20260202 02:52:29.325374 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de76ca9a98c840d0967a97154b940b60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.326007 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 505) took 3113 ms. Trace:
I20260202 02:52:29.326072 1882 rpcz_store.cc:276] 0202 02:52:26.212263 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.212310 (+ 47us) service_pool.cc:225] Handling call
0202 02:52:29.325898 (+3113588us) ts_tablet_manager.cc:1151] Deleting tablet 3191a80374724effa8389adc842e3448
0202 02:52:29.325908 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.325992 (+ 84us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":21,"tablet-delete.queue_time_us":3113153}
W20260202 02:52:29.326452 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 3191a80374724effa8389adc842e3448 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3191a80374724effa8389adc842e3448 already in progress: creating tablet
I20260202 02:52:29.327113 1882 tablet_replica.cc:333] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:29.327286 1882 raft_consensus.cc:2243] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:29.327586 1882 raft_consensus.cc:2272] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:29.328174 1882 ts_tablet_manager.cc:1916] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:29.329506 749 tablet_service.cc:1552] Processing DeleteTablet for tablet e7df9a78735547cbbaa006c604b95fec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:28 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:29.329614 1174 tablet_bootstrap.cc:492] T 75569511179b41038e521fa3346a92ae P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:29.330152 1721 consensus_queue.cc:1048] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:29.331753 1174 tablet_bootstrap.cc:654] T 75569511179b41038e521fa3346a92ae P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:29.332515 2170 raft_consensus.cc:493] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:29.332662 2170 raft_consensus.cc:515] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:29.333312 2170 leader_election.cc:290] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:29.334599 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8bb14bf551f441dcbaaff7057b51b2be" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:29.335176 857 leader_election.cc:343] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:29.342368 1882 ts_tablet_manager.cc:1929] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:29.344898 1882 log.cc:1199] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/ef073880898049e2be93d2cb6dc0c0bb
I20260202 02:52:29.345199 521 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet fc3bfd542b8e41d98586645358061261
I20260202 02:52:29.345654 1882 ts_tablet_manager.cc:1950] T ef073880898049e2be93d2cb6dc0c0bb P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:29.346843 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 506) took 3132 ms. Trace:
I20260202 02:52:29.345158 1058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8bb14bf551f441dcbaaff7057b51b2be" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:29.348410 902 tablet_service.cc:1552] Processing DeleteTablet for tablet fc3bfd542b8e41d98586645358061261 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:28 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:29.348212 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet fc3bfd542b8e41d98586645358061261 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:28 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:29.348866 749 tablet_service.cc:1552] Processing DeleteTablet for tablet fc3bfd542b8e41d98586645358061261 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:28 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:29.347841 1882 rpcz_store.cc:276] 0202 02:52:26.214299 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.214354 (+ 55us) service_pool.cc:225] Handling call
0202 02:52:29.327067 (+3112713us) ts_tablet_manager.cc:1151] Deleting tablet ef073880898049e2be93d2cb6dc0c0bb
0202 02:52:29.327077 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.335145 (+ 8068us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.342251 (+ 7106us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.346821 (+ 4570us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3112497}
I20260202 02:52:29.349278 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet ef073880898049e2be93d2cb6dc0c0bb (table default.Table_e8e9b4cdaf434b6cbbfed1a65bff1fc0 [id=78a7fb3a2eed4565b55d5dacb8a3d7df]) successfully deleted
W20260202 02:52:29.349299 859 leader_election.cc:343] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:29.349421 859 leader_election.cc:304] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:29.349620 2126 raft_consensus.cc:493] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:29.349762 2126 raft_consensus.cc:515] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.349828 2170 raft_consensus.cc:2749] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:29.350124 526 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_d1dc608853d84069a5d8fdfcc68cc10a" } modify_external_catalogs: true
I20260202 02:52:29.350308 526 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_d1dc608853d84069a5d8fdfcc68cc10a" } modify_external_catalogs: true
I20260202 02:52:29.350363 2126 leader_election.cc:290] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:29.350754 922 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "be993c1512b04df4b92cab3195241d84" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:29.350951 922 raft_consensus.cc:2468] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:29.351421 999 leader_election.cc:304] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:29.351521 1882 tablet_replica.cc:333] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:29.351712 1882 raft_consensus.cc:2243] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:29.352221 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "be993c1512b04df4b92cab3195241d84" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:29.352353 1882 raft_consensus.cc:2272] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:29.351779 2126 raft_consensus.cc:2804] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:29.352639 2126 raft_consensus.cc:493] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:29.352737 2126 raft_consensus.cc:3060] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:29.352919 1882 ts_tablet_manager.cc:1916] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:29.355351 2126 raft_consensus.cc:515] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.357240 922 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "be993c1512b04df4b92cab3195241d84" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:29.357457 922 raft_consensus.cc:3060] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:29.360153 922 raft_consensus.cc:2468] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:29.360502 2126 leader_election.cc:290] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:29.361248 998 leader_election.cc:343] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:29.361948 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "be993c1512b04df4b92cab3195241d84" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
W20260202 02:52:29.362612 998 leader_election.cc:343] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:29.363409 999 leader_election.cc:304] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:29.364617 1882 ts_tablet_manager.cc:1929] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:29.364859 2126 raft_consensus.cc:2804] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:29.366706 2126 raft_consensus.cc:697] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:29.367015 2126 consensus_queue.cc:237] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.371344 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_ae634747dc6e4a789c36df9b29d861c0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:29.372078 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ae634747dc6e4a789c36df9b29d861c0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:29.372201 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27158aa96e2742d5b0b2c3fa9531178d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.375358 1882 log.cc:1199] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/38fdbc6df7ec40b9afe4e0dfe0119e3e
I20260202 02:52:29.375746 1882 ts_tablet_manager.cc:1950] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:29.376535 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 514) took 3067 ms. Trace:
I20260202 02:52:29.376567 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4453a3e7ef784a528d8683e28d14e604. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.379460 1882 rpcz_store.cc:276] 0202 02:52:26.309022 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.309446 (+ 424us) service_pool.cc:225] Handling call
0202 02:52:29.351465 (+3042019us) ts_tablet_manager.cc:1151] Deleting tablet 38fdbc6df7ec40b9afe4e0dfe0119e3e
0202 02:52:29.351477 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.358646 (+ 7169us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.364508 (+ 5862us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.376512 (+ 12004us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3041769}
I20260202 02:52:29.380941 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 38fdbc6df7ec40b9afe4e0dfe0119e3e (table default.Table_87277c5d80184738b96b773f026b32a5 [id=52687dbd0e554cfb81d88e8fcdd4dd5d]) successfully deleted
I20260202 02:52:29.381791 1882 tablet_replica.cc:333] T 38301e13e17c41fcae912c4b38c26a45 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:29.382516 1882 raft_consensus.cc:2243] T 38301e13e17c41fcae912c4b38c26a45 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:29.383261 1882 raft_consensus.cc:2272] T 38301e13e17c41fcae912c4b38c26a45 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:29.386145 1882 ts_tablet_manager.cc:1916] T 38301e13e17c41fcae912c4b38c26a45 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:29.388041 2189 raft_consensus.cc:493] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:29.388227 2189 raft_consensus.cc:515] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
W20260202 02:52:29.391999 714 leader_election.cc:343] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: d30557fbe9084cd3860820f69dda8752
I20260202 02:52:29.396220 1882 ts_tablet_manager.cc:1929] T 38301e13e17c41fcae912c4b38c26a45 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:29.396616 2189 leader_election.cc:290] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:29.397779 520 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_38e8a46dd2dd498baa45935fa4670b09" } new_table_name: "default.Table_b30dfc448e1646a7b90534a8fa5fa24a" modify_external_catalogs: true
I20260202 02:52:29.401177 2189 raft_consensus.cc:493] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:29.401408 2189 raft_consensus.cc:515] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:29.403201 714 leader_election.cc:343] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 13cd1383348b458894fdecc22bf24b74
I20260202 02:52:29.405028 2189 leader_election.cc:290] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:29.405694 2170 raft_consensus.cc:493] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:29.405896 2170 raft_consensus.cc:515] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.406612 2170 leader_election.cc:290] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:29.404016 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "13cd1383348b458894fdecc22bf24b74" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:29.406935 1062 raft_consensus.cc:2468] T 13cd1383348b458894fdecc22bf24b74 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:29.406966 1882 log.cc:1199] T 38301e13e17c41fcae912c4b38c26a45 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/38301e13e17c41fcae912c4b38c26a45
I20260202 02:52:29.407317 1882 ts_tablet_manager.cc:1950] T 38301e13e17c41fcae912c4b38c26a45 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:29.407930 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26b890ed87c3471b9fc0b172c90e0a11" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:29.408672 857 leader_election.cc:343] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:29.409121 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 515) took 3099 ms. Trace:
I20260202 02:52:29.409224 1882 rpcz_store.cc:276] 0202 02:52:26.309204 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.309733 (+ 529us) service_pool.cc:225] Handling call
0202 02:52:29.381724 (+3071991us) ts_tablet_manager.cc:1151] Deleting tablet 38301e13e17c41fcae912c4b38c26a45
0202 02:52:29.381739 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.393076 (+ 11337us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.396087 (+ 3011us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.409099 (+ 13012us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3071789}
I20260202 02:52:29.409359 715 leader_election.cc:304] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:29.409693 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 38301e13e17c41fcae912c4b38c26a45 (table default.Table_80912cc76f934f139da7f866546ae97a [id=2f3bf7e0cf4c41ac8a2554360b57214a]) successfully deleted
I20260202 02:52:29.410565 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26b890ed87c3471b9fc0b172c90e0a11" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:29.410768 1062 raft_consensus.cc:2468] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
I20260202 02:52:29.411278 859 leader_election.cc:304] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:29.411811 2170 raft_consensus.cc:2804] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:29.411942 2170 raft_consensus.cc:493] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:29.412073 2170 raft_consensus.cc:3060] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:29.412207 519 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 745fc215251e4ccb9a2abed02573dd84 consensus state has no leader
W20260202 02:52:29.412573 528 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b427c7e897c64977b924e7d586420e69 consensus state has no leader
I20260202 02:52:29.413166 519 master_service.cc:954] ReplaceTablet: received request to replace tablet 2616e1bbce4049dbb584550ccc150288 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:29.412509 1882 tablet_replica.cc:333] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:29.413924 1882 raft_consensus.cc:2243] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [term 2 LEADER]: Raft consensus shutting down.
I20260202 02:52:29.414163 1882 pending_rounds.cc:70] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d: Trying to abort 1 pending ops.
I20260202 02:52:29.414202 1882 pending_rounds.cc:77] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d: Aborting op as it isn't in flight: id { term: 2 index: 1 } timestamp: 7249923067841806336 op_type: NO_OP noop_request { }
I20260202 02:52:29.414286 1882 raft_consensus.cc:2889] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:29.414356 1882 raft_consensus.cc:2272] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:29.414875 2170 raft_consensus.cc:515] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.416198 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26b890ed87c3471b9fc0b172c90e0a11" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:29.416390 1062 raft_consensus.cc:3060] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:29.416556 1882 ts_tablet_manager.cc:1916] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:29.419147 1062 raft_consensus.cc:2468] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
I20260202 02:52:29.400296 1058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d30557fbe9084cd3860820f69dda8752" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:29.419767 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_f21cdfd093884683ab4ec1af5345e9d5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:29.420092 1058 raft_consensus.cc:2468] T d30557fbe9084cd3860820f69dda8752 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:29.420620 715 leader_election.cc:304] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:29.420643 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_c57661c84d424228a977c159e893db51"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:29.420941 2151 raft_consensus.cc:2804] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:29.421007 2151 raft_consensus.cc:493] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:29.421062 2151 raft_consensus.cc:3060] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:29.421172 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c57661c84d424228a977c159e893db51 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:29.422535 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f21cdfd093884683ab4ec1af5345e9d5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:29.423681 2151 raft_consensus.cc:515] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:29.428711 2151 leader_election.cc:290] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:29.428902 2151 raft_consensus.cc:2804] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:29.428962 2151 raft_consensus.cc:493] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:29.429015 2151 raft_consensus.cc:3060] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:29.429565 524 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 21fe651f45bf43609cf972bef5eca149 consensus state has no leader
I20260202 02:52:29.431555 2151 raft_consensus.cc:515] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
W20260202 02:52:29.435232 714 consensus_peers.cc:597] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 08d8d781bb4e4acf915f11afb4fb2e22. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:29.435426 714 leader_election.cc:343] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: d30557fbe9084cd3860820f69dda8752
I20260202 02:52:29.435787 1906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260202 02:52:29.436981 1906 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260202 02:52:29.437394 1906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260202 02:52:29.438041 1882 ts_tablet_manager.cc:1929] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260202 02:52:29.438196 1882 log.cc:1199] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/8774c6bc0fe9414ea2f17ed859d9392a
I20260202 02:52:29.438572 1882 ts_tablet_manager.cc:1950] T 8774c6bc0fe9414ea2f17ed859d9392a P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:29.439311 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 517) took 3129 ms. Trace:
I20260202 02:52:29.439407 1882 rpcz_store.cc:276] 0202 02:52:26.309391 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.310126 (+ 735us) service_pool.cc:225] Handling call
0202 02:52:29.412452 (+3102326us) ts_tablet_manager.cc:1151] Deleting tablet 8774c6bc0fe9414ea2f17ed859d9392a
0202 02:52:29.412463 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.426786 (+ 14323us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.429660 (+ 2874us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.439293 (+ 9633us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3102152}
I20260202 02:52:29.439707 1882 tablet_replica.cc:333] T 68823eff1bc04108ba83487605aec0ea P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:29.439895 1882 raft_consensus.cc:2243] T 68823eff1bc04108ba83487605aec0ea P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:29.439939 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 8774c6bc0fe9414ea2f17ed859d9392a (table default.Table_0eb450e3edbb4a4ba0906567f831830c [id=8918ec9e2cdf4f2995b2b4baf21297d7]) successfully deleted
I20260202 02:52:29.440061 1882 raft_consensus.cc:2272] T 68823eff1bc04108ba83487605aec0ea P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:29.440483 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_7ff322b57db648b3b480519c357a083b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:29.441253 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7ff322b57db648b3b480519c357a083b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:29.441952 714 leader_election.cc:343] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 13cd1383348b458894fdecc22bf24b74
I20260202 02:52:29.443360 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d30557fbe9084cd3860820f69dda8752" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:29.443563 1063 raft_consensus.cc:3060] T d30557fbe9084cd3860820f69dda8752 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:29.446472 1063 raft_consensus.cc:2468] T d30557fbe9084cd3860820f69dda8752 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:29.448295 715 leader_election.cc:304] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:29.449326 859 leader_election.cc:304] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:29.449972 1882 ts_tablet_manager.cc:1916] T 68823eff1bc04108ba83487605aec0ea P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:29.453377 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_1a44a09f9b4f49a9a5f1f9f2529b5a52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:29.454063 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1a44a09f9b4f49a9a5f1f9f2529b5a52 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:29.454173 1058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "13cd1383348b458894fdecc22bf24b74" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:29.454355 1058 raft_consensus.cc:3060] T 13cd1383348b458894fdecc22bf24b74 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:29.456226 2151 leader_election.cc:290] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:29.450875 2236 raft_consensus.cc:493] T 2c31b45dc8004eae993e972e9faa3ff8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:29.457163 2236 raft_consensus.cc:515] T 2c31b45dc8004eae993e972e9faa3ff8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:29.457844 2236 leader_election.cc:290] T 2c31b45dc8004eae993e972e9faa3ff8 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:29.458215 2236 raft_consensus.cc:2804] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:29.459510 1882 ts_tablet_manager.cc:1929] T 68823eff1bc04108ba83487605aec0ea P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:29.459628 1882 log.cc:1199] T 68823eff1bc04108ba83487605aec0ea P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/68823eff1bc04108ba83487605aec0ea
I20260202 02:52:29.459964 1882 ts_tablet_manager.cc:1950] T 68823eff1bc04108ba83487605aec0ea P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:29.460510 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2c31b45dc8004eae993e972e9faa3ff8" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:29.460793 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 516) took 3151 ms. Trace:
I20260202 02:52:29.460868 1882 rpcz_store.cc:276] 0202 02:52:26.309310 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.309964 (+ 654us) service_pool.cc:225] Handling call
0202 02:52:29.439659 (+3129695us) ts_tablet_manager.cc:1151] Deleting tablet 68823eff1bc04108ba83487605aec0ea
0202 02:52:29.439669 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.453822 (+ 14153us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.459395 (+ 5573us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.460771 (+ 1376us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3129313}
W20260202 02:52:29.461084 520 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1d98c7a46139413b8f0d3697b74b1e52 consensus state has no leader
I20260202 02:52:29.461297 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 68823eff1bc04108ba83487605aec0ea (table default.Table_d4ad0ff5676e4af2a187497eda81d7a2 [id=0769ae34ed3d479a9f7010b34c35a110]) successfully deleted
W20260202 02:52:29.461798 857 leader_election.cc:343] T 2c31b45dc8004eae993e972e9faa3ff8 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:29.465510 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 513) took 3156 ms. Trace:
I20260202 02:52:29.465425 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_eea651758bad47dfa286f8dcea6787b6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:29.465595 1882 rpcz_store.cc:276] 0202 02:52:26.308894 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.318387 (+ 9493us) service_pool.cc:225] Handling call
0202 02:52:29.465352 (+3146965us) ts_tablet_manager.cc:1151] Deleting tablet 8ae6390a856d4c838e0689cd0c6c5c92
0202 02:52:29.465366 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.465484 (+ 118us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3146744}
W20260202 02:52:29.466087 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 8ae6390a856d4c838e0689cd0c6c5c92 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8ae6390a856d4c838e0689cd0c6c5c92 already in progress: creating tablet
W20260202 02:52:29.466193 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eea651758bad47dfa286f8dcea6787b6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:29.467103 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 512) took 3158 ms. Trace:
I20260202 02:52:29.467573 1882 rpcz_store.cc:276] 0202 02:52:26.308784 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.341867 (+ 33083us) service_pool.cc:225] Handling call
0202 02:52:29.466967 (+3125100us) ts_tablet_manager.cc:1151] Deleting tablet 7ea7a5d8a92b4ac0903483627258ae66
0202 02:52:29.466979 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.467082 (+ 103us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3124763}
W20260202 02:52:29.468473 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 7ea7a5d8a92b4ac0903483627258ae66 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7ea7a5d8a92b4ac0903483627258ae66 already in progress: creating tablet
I20260202 02:52:29.468488 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2c31b45dc8004eae993e972e9faa3ff8" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:29.469096 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 523) took 3051 ms. Trace:
I20260202 02:52:29.469493 1882 rpcz_store.cc:276] 0202 02:52:26.417334 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.421817 (+ 4483us) service_pool.cc:225] Handling call
0202 02:52:29.468982 (+3047165us) ts_tablet_manager.cc:1151] Deleting tablet 7e94cacaa0714f92ad6a2b2ab6171f44
0202 02:52:29.468992 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.469079 (+ 87us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3046948}
W20260202 02:52:29.470181 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 7e94cacaa0714f92ad6a2b2ab6171f44 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7e94cacaa0714f92ad6a2b2ab6171f44 already in progress: creating tablet
I20260202 02:52:29.470613 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 524) took 3036 ms. Trace:
I20260202 02:52:29.470992 1882 rpcz_store.cc:276] 0202 02:52:26.434554 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.434611 (+ 57us) service_pool.cc:225] Handling call
0202 02:52:29.470501 (+3035890us) ts_tablet_manager.cc:1151] Deleting tablet 2f664f6119ba4d28b5ef1c212e611866
0202 02:52:29.470511 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.470595 (+ 84us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3035622}
W20260202 02:52:29.471642 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 2f664f6119ba4d28b5ef1c212e611866 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2f664f6119ba4d28b5ef1c212e611866 already in progress: creating tablet
I20260202 02:52:29.472106 1882 tablet_replica.cc:333] T 761f5a51335a4de5977f5f7fef0d4260 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:29.472450 2157 raft_consensus.cc:2804] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:29.472483 1882 raft_consensus.cc:2243] T 761f5a51335a4de5977f5f7fef0d4260 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:29.472604 2157 raft_consensus.cc:697] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:29.472656 1882 raft_consensus.cc:2272] T 761f5a51335a4de5977f5f7fef0d4260 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:29.472884 2157 consensus_queue.cc:237] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.473201 1882 ts_tablet_manager.cc:1916] T 761f5a51335a4de5977f5f7fef0d4260 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:29.476986 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_aef88b2493064de9b51f8eb3ee276ea2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:29.477640 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aef88b2493064de9b51f8eb3ee276ea2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:29.484408 1882 ts_tablet_manager.cc:1929] T 761f5a51335a4de5977f5f7fef0d4260 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:29.487836 2189 raft_consensus.cc:493] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:29.488031 2189 raft_consensus.cc:515] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.488652 2189 leader_election.cc:290] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:29.489234 1882 log.cc:1199] T 761f5a51335a4de5977f5f7fef0d4260 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/761f5a51335a4de5977f5f7fef0d4260
I20260202 02:52:29.491868 1882 ts_tablet_manager.cc:1950] T 761f5a51335a4de5977f5f7fef0d4260 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
W20260202 02:52:29.492084 859 leader_election.cc:343] T 2c31b45dc8004eae993e972e9faa3ff8 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:29.492237 859 leader_election.cc:304] T 2c31b45dc8004eae993e972e9faa3ff8 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:29.493515 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "06b80a1c9251420cbc16fc95ae98aca9" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:29.493747 1062 raft_consensus.cc:2468] T 06b80a1c9251420cbc16fc95ae98aca9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:29.493597 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 525) took 3058 ms. Trace:
I20260202 02:52:29.496141 887 tablet_service.cc:1552] Processing DeleteTablet for tablet ccd6bc8bd0504bf18d108c73685ef9ff with delete_type TABLET_DATA_DELETED (replaced by tablet 8ba28ab7d7ae44059006179067dff3c0) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:29.496253 1882 rpcz_store.cc:276] 0202 02:52:26.435567 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.435702 (+ 135us) service_pool.cc:225] Handling call
0202 02:52:29.472055 (+3036353us) ts_tablet_manager.cc:1151] Deleting tablet 761f5a51335a4de5977f5f7fef0d4260
0202 02:52:29.472065 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.481055 (+ 8990us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.484255 (+ 3200us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.493553 (+ 9298us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3035534}
I20260202 02:52:29.496475 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_4bfd0a9f04ae4defa2e130a44fe45b2f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:29.497131 2271 raft_consensus.cc:2749] T 2c31b45dc8004eae993e972e9faa3ff8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:29.497145 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4bfd0a9f04ae4defa2e130a44fe45b2f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:29.497224 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 761f5a51335a4de5977f5f7fef0d4260 (table default.Table_7913f8fb6eac487a971545a11cf257a9 [id=b40664207c364307ba6f5ade490a21d6]) successfully deleted
I20260202 02:52:29.497644 1882 tablet_replica.cc:333] T 9c1271ce0f194021b588b82ac9d3db79 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:29.497704 2170 leader_election.cc:290] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:29.498071 1882 raft_consensus.cc:2243] T 9c1271ce0f194021b588b82ac9d3db79 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:29.498503 1882 raft_consensus.cc:2272] T 9c1271ce0f194021b588b82ac9d3db79 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:29.498960 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26b890ed87c3471b9fc0b172c90e0a11" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:29.499289 1882 ts_tablet_manager.cc:1916] T 9c1271ce0f194021b588b82ac9d3db79 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:29.499653 857 leader_election.cc:343] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:29.504797 715 leader_election.cc:304] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:29.505100 1972 raft_consensus.cc:2804] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:29.505172 1972 raft_consensus.cc:493] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:29.505224 1972 raft_consensus.cc:3060] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:29.506633 527 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 26b890ed87c3471b9fc0b172c90e0a11 (table default.Table_044ef6a2b1af4e3cab3c8cf00505c6a6 [id=d2e4220a0cbf443e8380433da4f064f8]). Expected version 1 got 0
I20260202 02:52:29.507846 1972 raft_consensus.cc:515] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.508797 1972 leader_election.cc:290] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:29.510749 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 75569511179b41038e521fa3346a92ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:29.511857 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet ef073880898049e2be93d2cb6dc0c0bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:29.512073 525 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 842e54adae2a452a98521de24326eaff (table default.Table_c6e682a233904983978e7abb23565e41 [id=0d7624b55c1a4bf0a449f651ac419d77]). Expected version 1 got 0
I20260202 02:52:29.514930 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 474df41e4fe24a818fa45a4ce84d5474 with delete_type TABLET_DATA_DELETED (replaced by tablet f30b698c177c48ee9a4e577552bf7eee) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:29.515980 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 1f379a29e2454f689d68f5951837512b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:29.517192 1041 tablet_service.cc:1505] Processing CreateTablet for tablet bcc6cd3595364680b4c39c1ff740aebb (DEFAULT_TABLE table=default.Table_22af4f1f5edd4a0cba0252f162e49241 [id=2152128426644acaae4a2654b834064d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.517684 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcc6cd3595364680b4c39c1ff740aebb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.519084 1058 raft_consensus.cc:2468] T 13cd1383348b458894fdecc22bf24b74 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:29.520955 891 tablet_service.cc:1505] Processing CreateTablet for tablet 876868b3e7714328bf8cf480f0dabc67 (DEFAULT_TABLE table=default.Table_521f05f94e4b4b90a7304648efdd4919 [id=8bf8deaf2784486cb2f15e02fc3fde12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.521375 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 876868b3e7714328bf8cf480f0dabc67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.521291 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 876868b3e7714328bf8cf480f0dabc67 (DEFAULT_TABLE table=default.Table_521f05f94e4b4b90a7304648efdd4919 [id=8bf8deaf2784486cb2f15e02fc3fde12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.521656 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 876868b3e7714328bf8cf480f0dabc67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.522727 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 5ffafb371804467f8704d3c1c36af80d (DEFAULT_TABLE table=default.Table_d05a522f656546f9a12e61fe94ca4873 [id=7f8d853367174b48a302092b064ede7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.523092 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ffafb371804467f8704d3c1c36af80d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.525122 896 tablet_service.cc:1505] Processing CreateTablet for tablet 5ffafb371804467f8704d3c1c36af80d (DEFAULT_TABLE table=default.Table_d05a522f656546f9a12e61fe94ca4873 [id=7f8d853367174b48a302092b064ede7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.525522 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ffafb371804467f8704d3c1c36af80d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.531581 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 48863e2d60c84460858daabb9a08dcd2 (DEFAULT_TABLE table=default.Table_6b5c3a97133940d98d66d07f6be3d55a [id=9044e5eae72a4604913d50c032dc828a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.532083 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48863e2d60c84460858daabb9a08dcd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.532449 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 4482b24826884d0da0d34f43f69e1646 (DEFAULT_TABLE table=default.Table_7a09e5be1e824c82bfd2d252cbd1043e [id=b5a3bd05316f4e6ab2f80b3dcfd5c1f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.532579 884 tablet_service.cc:1505] Processing CreateTablet for tablet 4482b24826884d0da0d34f43f69e1646 (DEFAULT_TABLE table=default.Table_7a09e5be1e824c82bfd2d252cbd1043e [id=b5a3bd05316f4e6ab2f80b3dcfd5c1f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.532799 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4482b24826884d0da0d34f43f69e1646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.533016 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4482b24826884d0da0d34f43f69e1646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.538938 527 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_4fb44d5cbe71439ba073bd75711e77e0" } modify_external_catalogs: true
I20260202 02:52:29.539166 527 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_4fb44d5cbe71439ba073bd75711e77e0" } modify_external_catalogs: true
I20260202 02:52:29.541950 1882 ts_tablet_manager.cc:1929] T 9c1271ce0f194021b588b82ac9d3db79 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:29.542258 525 catalog_manager.cc:5654] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d reported cstate change: term changed from 0 to 1, leader changed from <none> to b1f05820b6034228bdad884cc348716d (127.0.104.131). New cstate: current_term: 1 leader_uuid: "b1f05820b6034228bdad884cc348716d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } }
I20260202 02:52:29.542721 1882 log.cc:1199] T 9c1271ce0f194021b588b82ac9d3db79 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/9c1271ce0f194021b588b82ac9d3db79
I20260202 02:52:29.542878 525 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet be993c1512b04df4b92cab3195241d84 (table default.Table_dd517ae94bf54267bc3a03dee4896708 [id=22f1ff6e13a84a72820c887dd3107f49]). Expected version 1 got 0
I20260202 02:52:29.543218 1882 ts_tablet_manager.cc:1950] T 9c1271ce0f194021b588b82ac9d3db79 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:29.543432 896 tablet_service.cc:1505] Processing CreateTablet for tablet 48863e2d60c84460858daabb9a08dcd2 (DEFAULT_TABLE table=default.Table_6b5c3a97133940d98d66d07f6be3d55a [id=9044e5eae72a4604913d50c032dc828a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.543856 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48863e2d60c84460858daabb9a08dcd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.544795 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 527) took 3091 ms. Trace:
I20260202 02:52:29.545673 1882 rpcz_store.cc:276] 0202 02:52:26.453197 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.453248 (+ 51us) service_pool.cc:225] Handling call
0202 02:52:29.497588 (+3044340us) ts_tablet_manager.cc:1151] Deleting tablet 9c1271ce0f194021b588b82ac9d3db79
0202 02:52:29.497601 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.504256 (+ 6655us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.541819 (+ 37563us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.544769 (+ 2950us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3044122}
I20260202 02:52:29.546458 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 9c1271ce0f194021b588b82ac9d3db79 (table default.Table_733044f900d44218a06c976c8701f122 [id=9f912bc02bec49a8b79e4e7e921595b9]) successfully deleted
I20260202 02:52:29.546880 1882 tablet_replica.cc:333] T 08d8d781bb4e4acf915f11afb4fb2e22 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:29.547327 1882 raft_consensus.cc:2243] T 08d8d781bb4e4acf915f11afb4fb2e22 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:29.547766 1882 raft_consensus.cc:2272] T 08d8d781bb4e4acf915f11afb4fb2e22 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:29.548184 887 tablet_service.cc:1505] Processing CreateTablet for tablet bcc6cd3595364680b4c39c1ff740aebb (DEFAULT_TABLE table=default.Table_22af4f1f5edd4a0cba0252f162e49241 [id=2152128426644acaae4a2654b834064d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.548658 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcc6cd3595364680b4c39c1ff740aebb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.549028 1882 ts_tablet_manager.cc:1916] T 08d8d781bb4e4acf915f11afb4fb2e22 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:29.551540 715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 46773.9us
I20260202 02:52:29.552587 715 leader_election.cc:304] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:29.552914 1972 raft_consensus.cc:2804] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:29.553033 1972 raft_consensus.cc:697] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:29.553295 1972 consensus_queue.cc:237] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:29.555902 524 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_b30dfc448e1646a7b90534a8fa5fa24a" } modify_external_catalogs: true
I20260202 02:52:29.556133 524 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_b30dfc448e1646a7b90534a8fa5fa24a" } modify_external_catalogs: true
I20260202 02:52:29.558807 1180 tablet_replica.cc:333] T 1f54c1dfe0a942b3bafd85522953a032 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:29.559064 1180 raft_consensus.cc:2243] T 1f54c1dfe0a942b3bafd85522953a032 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:29.559257 1180 raft_consensus.cc:2272] T 1f54c1dfe0a942b3bafd85522953a032 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:29.559583 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 8a17be71e0f444c9ac2b127f35420780 (DEFAULT_TABLE table=default.Table_af8ce4ca28bf4200b52f066fc4c30d63 [id=3fe28de6f2b44aa1b7312fd06115d80c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.559882 1180 ts_tablet_manager.cc:1916] T 1f54c1dfe0a942b3bafd85522953a032 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:29.560192 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a17be71e0f444c9ac2b127f35420780. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.567799 1182 ts_tablet_manager.cc:1929] T 550895b4010f4088b2cc34acff50e03f P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:29.567961 1182 log.cc:1199] T 550895b4010f4088b2cc34acff50e03f P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/550895b4010f4088b2cc34acff50e03f
I20260202 02:52:29.568317 1182 ts_tablet_manager.cc:1950] T 550895b4010f4088b2cc34acff50e03f P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:29.568888 1182 tablet_replica.cc:333] T 2c31b45dc8004eae993e972e9faa3ff8 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:29.569100 1182 raft_consensus.cc:2243] T 2c31b45dc8004eae993e972e9faa3ff8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:29.569271 1182 raft_consensus.cc:2272] T 2c31b45dc8004eae993e972e9faa3ff8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:29.569785 523 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_a115529cbd054b83bc0b16a843f8017d" } modify_external_catalogs: true
I20260202 02:52:29.570574 523 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_a115529cbd054b83bc0b16a843f8017d" } modify_external_catalogs: true
I20260202 02:52:29.571842 891 tablet_service.cc:1505] Processing CreateTablet for tablet 8a17be71e0f444c9ac2b127f35420780 (DEFAULT_TABLE table=default.Table_af8ce4ca28bf4200b52f066fc4c30d63 [id=3fe28de6f2b44aa1b7312fd06115d80c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.571964 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 550895b4010f4088b2cc34acff50e03f (table default.Table_c7654f22d0044539a2eb8a736163a22d [id=427f58cc70ef47a6afac8de0353975f5]) successfully deleted
I20260202 02:52:29.572314 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a17be71e0f444c9ac2b127f35420780. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.576205 884 tablet_service.cc:1505] Processing CreateTablet for tablet 365417e050304073b5de887082fc699b (DEFAULT_TABLE table=default.Table_25f0023dd22a4bb0a7c2863da66d0002 [id=e24ffad12f514d60907b6cadfd09dbe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.576798 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 365417e050304073b5de887082fc699b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.576902 1882 ts_tablet_manager.cc:1929] T 08d8d781bb4e4acf915f11afb4fb2e22 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:29.586035 520 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_dd517ae94bf54267bc3a03dee4896708" } modify_external_catalogs: true
I20260202 02:52:29.586246 520 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_dd517ae94bf54267bc3a03dee4896708" } modify_external_catalogs: true
W20260202 02:52:29.586334 715 consensus_peers.cc:597] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 59990791123c4d4eb620090a1e8e31e1. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:29.583560 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 365417e050304073b5de887082fc699b (DEFAULT_TABLE table=default.Table_25f0023dd22a4bb0a7c2863da66d0002 [id=e24ffad12f514d60907b6cadfd09dbe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.604905 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 365417e050304073b5de887082fc699b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.584574 2126 consensus_queue.cc:1048] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:29.595078 1173 tablet_bootstrap.cc:492] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:29.609154 1173 ts_tablet_manager.cc:1403] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.499s user 0.036s sys 0.102s
I20260202 02:52:29.609885 1173 raft_consensus.cc:359] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.610055 1173 raft_consensus.cc:385] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:29.610106 1173 raft_consensus.cc:740] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:29.610285 1173 consensus_queue.cc:260] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.610764 1173 ts_tablet_manager.cc:1434] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:29.611069 1173 tablet_bootstrap.cc:492] T 33150e1456144b8e9c64cc2a2ac6dcca P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:29.611768 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet a1f39e5b02f747819e636cddb4292eb6 (table default.Table_61c2ba441aa746edbb73f8b5de4204ba [id=abceac85d3524c96834546f67f41f933]) successfully deleted
I20260202 02:52:29.612854 744 tablet_service.cc:1505] Processing CreateTablet for tablet bcc6cd3595364680b4c39c1ff740aebb (DEFAULT_TABLE table=default.Table_22af4f1f5edd4a0cba0252f162e49241 [id=2152128426644acaae4a2654b834064d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.613296 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcc6cd3595364680b4c39c1ff740aebb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.620059 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_3c3eaedb107e485abb2fad9d4d958c62"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:29.620690 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3c3eaedb107e485abb2fad9d4d958c62 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:29.622368 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_00bfeb4d04d94815afb263182ee583f9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:29.622920 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_00bfeb4d04d94815afb263182ee583f9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:29.625075 2150 raft_consensus.cc:493] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:29.625241 2150 raft_consensus.cc:515] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:29.625847 1173 tablet_bootstrap.cc:654] T 33150e1456144b8e9c64cc2a2ac6dcca P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:29.626005 2150 leader_election.cc:290] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:29.628609 1721 raft_consensus.cc:493] T 3d1a7578e83447669e3c3972a426ad44 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:29.628753 1721 raft_consensus.cc:515] T 3d1a7578e83447669e3c3972a426ad44 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.629005 2236 raft_consensus.cc:697] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:29.629498 1721 leader_election.cc:290] T 3d1a7578e83447669e3c3972a426ad44 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:29.629392 2236 consensus_queue.cc:237] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.630275 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3d1a7578e83447669e3c3972a426ad44" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:29.630460 773 raft_consensus.cc:2468] T 3d1a7578e83447669e3c3972a426ad44 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:29.630939 998 leader_election.cc:304] T 3d1a7578e83447669e3c3972a426ad44 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:29.631237 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3d1a7578e83447669e3c3972a426ad44" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:29.631392 1721 raft_consensus.cc:2804] T 3d1a7578e83447669e3c3972a426ad44 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:29.631407 923 raft_consensus.cc:2468] T 3d1a7578e83447669e3c3972a426ad44 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:29.631460 1721 raft_consensus.cc:493] T 3d1a7578e83447669e3c3972a426ad44 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:29.631516 1721 raft_consensus.cc:3060] T 3d1a7578e83447669e3c3972a426ad44 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:29.633970 1721 raft_consensus.cc:515] T 3d1a7578e83447669e3c3972a426ad44 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.634253 522 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 3d1a7578e83447669e3c3972a426ad44
I20260202 02:52:29.634737 522 catalog_manager.cc:6724] ReplaceTablet: tablet 3d1a7578e83447669e3c3972a426ad44 deleted and replaced by tablet bdd72f995a704520a5fb1c24a53ed252
I20260202 02:52:29.635929 1721 leader_election.cc:290] T 3d1a7578e83447669e3c3972a426ad44 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:29.636155 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 3d1a7578e83447669e3c3972a426ad44 with delete_type TABLET_DATA_DELETED (replaced by tablet bdd72f995a704520a5fb1c24a53ed252) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:29.637020 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3d1a7578e83447669e3c3972a426ad44" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:29.637217 773 raft_consensus.cc:3060] T 3d1a7578e83447669e3c3972a426ad44 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:29.640046 773 raft_consensus.cc:2468] T 3d1a7578e83447669e3c3972a426ad44 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:29.640551 998 leader_election.cc:304] T 3d1a7578e83447669e3c3972a426ad44 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:29.640992 1721 raft_consensus.cc:2804] T 3d1a7578e83447669e3c3972a426ad44 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:29.641259 1721 raft_consensus.cc:697] T 3d1a7578e83447669e3c3972a426ad44 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:29.641902 1721 consensus_queue.cc:237] T 3d1a7578e83447669e3c3972a426ad44 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.598052 1058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "06b80a1c9251420cbc16fc95ae98aca9" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:29.642786 1058 raft_consensus.cc:3060] T 06b80a1c9251420cbc16fc95ae98aca9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:29.645404 1058 raft_consensus.cc:2468] T 06b80a1c9251420cbc16fc95ae98aca9 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:29.604929 1882 log.cc:1199] T 08d8d781bb4e4acf915f11afb4fb2e22 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/08d8d781bb4e4acf915f11afb4fb2e22
I20260202 02:52:29.647845 1882 ts_tablet_manager.cc:1950] T 08d8d781bb4e4acf915f11afb4fb2e22 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:29.648626 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 526) took 3195 ms. Trace:
I20260202 02:52:29.648737 1882 rpcz_store.cc:276] 0202 02:52:26.452865 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.455043 (+ 2178us) service_pool.cc:225] Handling call
0202 02:52:29.546825 (+3091782us) ts_tablet_manager.cc:1151] Deleting tablet 08d8d781bb4e4acf915f11afb4fb2e22
0202 02:52:29.546837 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.557671 (+ 10834us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.576777 (+ 19106us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.648609 (+ 71832us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3091553}
I20260202 02:52:29.649160 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 529) took 3125 ms. Trace:
I20260202 02:52:29.649232 1882 rpcz_store.cc:276] 0202 02:52:26.523546 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.523603 (+ 57us) service_pool.cc:225] Handling call
0202 02:52:29.649036 (+3125433us) ts_tablet_manager.cc:1151] Deleting tablet dc90eeb79ded4739883ad98928476680
0202 02:52:29.649046 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.649145 (+ 99us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3125190}
I20260202 02:52:29.649297 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 08d8d781bb4e4acf915f11afb4fb2e22 (table default.Table_2acd399bc33146fe9f1b6e2a15e0b101 [id=487a204c7845401ab7391ae8464fdf1d]) successfully deleted
I20260202 02:52:29.649546 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 530) took 3116 ms. Trace:
I20260202 02:52:29.649607 1882 rpcz_store.cc:276] 0202 02:52:26.533141 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.533201 (+ 60us) service_pool.cc:225] Handling call
0202 02:52:29.649449 (+3116248us) ts_tablet_manager.cc:1151] Deleting tablet 0a5bbb994cfa40f0add3163816b65269
0202 02:52:29.649458 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.649532 (+ 74us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3116028}
W20260202 02:52:29.649852 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet dc90eeb79ded4739883ad98928476680 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dc90eeb79ded4739883ad98928476680 already in progress: creating tablet
I20260202 02:52:29.649892 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 528) took 3129 ms. Trace:
I20260202 02:52:29.649948 1882 rpcz_store.cc:276] 0202 02:52:26.520561 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.537026 (+ 16465us) service_pool.cc:225] Handling call
0202 02:52:29.649803 (+3112777us) ts_tablet_manager.cc:1151] Deleting tablet 23671408fcbc454cbd918328015cf6e6
0202 02:52:29.649811 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.649879 (+ 68us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3112486}
I20260202 02:52:29.650290 885 tablet_service.cc:1552] Processing DeleteTablet for tablet a0b5d82e5fb94b60b4aa324b978a5378 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:29.650463 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 0a5bbb994cfa40f0add3163816b65269 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0a5bbb994cfa40f0add3163816b65269 already in progress: creating tablet
I20260202 02:52:29.650625 715 leader_election.cc:304] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
W20260202 02:52:29.650678 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 23671408fcbc454cbd918328015cf6e6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 23671408fcbc454cbd918328015cf6e6 already in progress: creating tablet
I20260202 02:52:29.651319 2157 raft_consensus.cc:2804] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:29.651443 2157 raft_consensus.cc:697] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:29.651707 2157 consensus_queue.cc:237] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.654502 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 3d1a7578e83447669e3c3972a426ad44 with delete_type TABLET_DATA_DELETED (replaced by tablet bdd72f995a704520a5fb1c24a53ed252) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:29.656689 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 531) took 3110 ms. Trace:
I20260202 02:52:29.656811 1882 rpcz_store.cc:276] 0202 02:52:26.546999 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.547701 (+ 702us) service_pool.cc:225] Handling call
0202 02:52:29.656517 (+3108816us) ts_tablet_manager.cc:1151] Deleting tablet 8ba28ab7d7ae44059006179067dff3c0
0202 02:52:29.656534 (+ 17us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.656661 (+ 127us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3101669}
I20260202 02:52:29.657192 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 538) took 2803 ms. Trace:
I20260202 02:52:29.657262 1882 rpcz_store.cc:276] 0202 02:52:26.854124 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.857358 (+ 3234us) service_pool.cc:225] Handling call
0202 02:52:29.657087 (+2799729us) ts_tablet_manager.cc:1151] Deleting tablet f1c8f7e1dc9644a6b6dd2bc264cce8ef
0202 02:52:29.657095 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.657178 (+ 83us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2799497}
W20260202 02:52:29.657374 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 8ba28ab7d7ae44059006179067dff3c0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8ba28ab7d7ae44059006179067dff3c0 already in progress: creating tablet
I20260202 02:52:29.657495 1882 tablet_replica.cc:333] T 757492288d7c44fb91b25b7e6e21e01b P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:29.657663 1882 raft_consensus.cc:2243] T 757492288d7c44fb91b25b7e6e21e01b P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:29.657855 1882 raft_consensus.cc:2272] T 757492288d7c44fb91b25b7e6e21e01b P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:29.657980 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet f1c8f7e1dc9644a6b6dd2bc264cce8ef because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f1c8f7e1dc9644a6b6dd2bc264cce8ef already in progress: creating tablet
I20260202 02:52:29.658445 1882 ts_tablet_manager.cc:1916] T 757492288d7c44fb91b25b7e6e21e01b P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:29.661731 521 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_d24b7b40906a4dfba59c9907d7b3cd1a" } new_table_name: "default.Table_3db4721b7f3d4b09aafe0f29ebb2039f" modify_external_catalogs: true
I20260202 02:52:29.662196 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3d1a7578e83447669e3c3972a426ad44" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:29.662410 923 raft_consensus.cc:3060] T 3d1a7578e83447669e3c3972a426ad44 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:29.663162 522 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ae54fc1a1cfc4475823c8c3b5cf86e5f" } new_table_name: "default.Table_1455877b4adc4c53a941d7fc145014b7" modify_external_catalogs: true
I20260202 02:52:29.665449 923 raft_consensus.cc:2468] T 3d1a7578e83447669e3c3972a426ad44 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
W20260202 02:52:29.666769 715 consensus_peers.cc:597] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 042d1ebf357b4abe8814a407891ce265. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:29.676453 2268 raft_consensus.cc:493] T 842e54adae2a452a98521de24326eaff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:29.676717 2268 raft_consensus.cc:515] T 842e54adae2a452a98521de24326eaff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:29.677454 2268 leader_election.cc:290] T 842e54adae2a452a98521de24326eaff P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:29.678452 526 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 67495e8e525542fb976f675b71512ed2
I20260202 02:52:29.679244 526 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_22af4f1f5edd4a0cba0252f162e49241" } new_table_name: "default.Table_8739c38e3fcd4069a2e7c623b11519b3" modify_external_catalogs: true
I20260202 02:52:29.681118 2127 raft_consensus.cc:493] T 26eeef6eebbf4f2cb44f69255fb37594 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:29.681213 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "842e54adae2a452a98521de24326eaff" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:29.681247 2127 raft_consensus.cc:515] T 26eeef6eebbf4f2cb44f69255fb37594 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.681802 2127 leader_election.cc:290] T 26eeef6eebbf4f2cb44f69255fb37594 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:29.682924 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 67495e8e525542fb976f675b71512ed2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:29 UTC) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:29.691069 859 leader_election.cc:343] T 842e54adae2a452a98521de24326eaff P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:29.692936 1882 ts_tablet_manager.cc:1929] T 757492288d7c44fb91b25b7e6e21e01b P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:29.695288 1882 log.cc:1199] T 757492288d7c44fb91b25b7e6e21e01b P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/757492288d7c44fb91b25b7e6e21e01b
I20260202 02:52:29.697357 1882 ts_tablet_manager.cc:1950] T 757492288d7c44fb91b25b7e6e21e01b P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:29.698371 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 539) took 2843 ms. Trace:
I20260202 02:52:29.679708 750 tablet_service.cc:1505] Processing CreateTablet for tablet 876868b3e7714328bf8cf480f0dabc67 (DEFAULT_TABLE table=default.Table_521f05f94e4b4b90a7304648efdd4919 [id=8bf8deaf2784486cb2f15e02fc3fde12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.698535 1882 rpcz_store.cc:276] 0202 02:52:26.854899 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.854945 (+ 46us) service_pool.cc:225] Handling call
0202 02:52:29.657454 (+2802509us) ts_tablet_manager.cc:1151] Deleting tablet 757492288d7c44fb91b25b7e6e21e01b
0202 02:52:29.657462 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.679428 (+ 21966us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.692814 (+ 13386us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.698346 (+ 5532us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2776483}
I20260202 02:52:29.682479 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26eeef6eebbf4f2cb44f69255fb37594" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:29.699095 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 757492288d7c44fb91b25b7e6e21e01b (table default.Table_7c01ea0d668a490099dbd42db17162ec [id=02e9e243f710428091de2f9497532c37]) successfully deleted
I20260202 02:52:29.688958 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "842e54adae2a452a98521de24326eaff" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:29.699967 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 876868b3e7714328bf8cf480f0dabc67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.689747 742 tablet_service.cc:1505] Processing CreateTablet for tablet 5ffafb371804467f8704d3c1c36af80d (DEFAULT_TABLE table=default.Table_d05a522f656546f9a12e61fe94ca4873 [id=7f8d853367174b48a302092b064ede7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.700541 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 537) took 2847 ms. Trace:
I20260202 02:52:29.700702 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ffafb371804467f8704d3c1c36af80d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:29.701820 998 leader_election.cc:343] T 26eeef6eebbf4f2cb44f69255fb37594 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:29.703341 776 raft_consensus.cc:3060] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:29.695988 756 tablet_service.cc:1505] Processing CreateTablet for tablet 48863e2d60c84460858daabb9a08dcd2 (DEFAULT_TABLE table=default.Table_6b5c3a97133940d98d66d07f6be3d55a [id=9044e5eae72a4604913d50c032dc828a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.707492 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48863e2d60c84460858daabb9a08dcd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.708674 1882 rpcz_store.cc:276] 0202 02:52:26.852862 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.854315 (+ 1453us) service_pool.cc:225] Handling call
0202 02:52:29.700396 (+2846081us) ts_tablet_manager.cc:1151] Deleting tablet 36db4ee7f5404cffaa3c14ad58e71701
0202 02:52:29.700407 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.700522 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2819332}
W20260202 02:52:29.711894 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 36db4ee7f5404cffaa3c14ad58e71701 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 36db4ee7f5404cffaa3c14ad58e71701 already in progress: creating tablet
I20260202 02:52:29.712553 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 540) took 2798 ms. Trace:
I20260202 02:52:29.715116 1882 rpcz_store.cc:276] 0202 02:52:26.914354 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.942046 (+ 27692us) service_pool.cc:225] Handling call
0202 02:52:29.712402 (+2770356us) ts_tablet_manager.cc:1151] Deleting tablet 1c8317adc47a4c399f2db650ed3d2e53
0202 02:52:29.712417 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.712531 (+ 114us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2770084}
W20260202 02:52:29.716020 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 1c8317adc47a4c399f2db650ed3d2e53 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1c8317adc47a4c399f2db650ed3d2e53 already in progress: creating tablet
I20260202 02:52:29.716512 1882 tablet_replica.cc:333] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:29.716943 1882 raft_consensus.cc:2243] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:29.719996 2170 raft_consensus.cc:493] T fbcb4a522df648ba9542250450c08f1b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:29.722730 2271 raft_consensus.cc:493] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:29.722995 2271 raft_consensus.cc:515] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.723084 2170 raft_consensus.cc:515] T fbcb4a522df648ba9542250450c08f1b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:29.723644 2271 leader_election.cc:290] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:29.724436 527 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 8dd62de8e8234d6c9ce380b1aadc097d
I20260202 02:52:29.724153 2170 leader_election.cc:290] T fbcb4a522df648ba9542250450c08f1b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:29.727774 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ec2d6011c0c6475d9cd5a11c6f94d5d6" } new_table_name: "default.Table_027e99b4f18f42948ac88f3658653753" modify_external_catalogs: true
I20260202 02:52:29.721516 1173 tablet_bootstrap.cc:492] T 33150e1456144b8e9c64cc2a2ac6dcca P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:29.724298 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1d8143f529b74e6ab779dd695920413c" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:29.724623 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fbcb4a522df648ba9542250450c08f1b" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:29.725281 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 8dd62de8e8234d6c9ce380b1aadc097d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:29 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:29.730343 1173 ts_tablet_manager.cc:1403] T 33150e1456144b8e9c64cc2a2ac6dcca P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.119s user 0.001s sys 0.009s
W20260202 02:52:29.730507 859 leader_election.cc:343] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:29.730741 859 leader_election.cc:343] T fbcb4a522df648ba9542250450c08f1b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:29.730923 2268 raft_consensus.cc:493] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260202 02:52:29.731117 715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 63926us
I20260202 02:52:29.731127 1173 raft_consensus.cc:359] T 33150e1456144b8e9c64cc2a2ac6dcca P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.733726 1173 raft_consensus.cc:385] T 33150e1456144b8e9c64cc2a2ac6dcca P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:29.733803 1173 raft_consensus.cc:740] T 33150e1456144b8e9c64cc2a2ac6dcca P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:29.733877 524 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 1d98c7a46139413b8f0d3697b74b1e52
I20260202 02:52:29.733999 1173 consensus_queue.cc:260] T 33150e1456144b8e9c64cc2a2ac6dcca P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.734714 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 1d98c7a46139413b8f0d3697b74b1e52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:29 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:29.736804 1173 ts_tablet_manager.cc:1434] T 33150e1456144b8e9c64cc2a2ac6dcca P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260202 02:52:29.737427 2268 raft_consensus.cc:515] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:29.738528 1173 tablet_bootstrap.cc:492] T dada209d982d48f3b74a49905f19c5b6 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
W20260202 02:52:29.736138 700 kernel_stack_watchdog.cc:198] Thread 1174 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:642 for 296ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f1e3b197980 (unknown) at ??:0
@ 0x93e827 __sanitizer::internal_sched_yield() at ??:?
@ 0x8a3edd __interceptor_pthread_create at ??:?
@ 0x7f1e37b7ec91 kudu::Thread::StartThread() at ??:0
@ 0xa76197 kudu::Thread::Create() at ??:?
@ 0x7f1e37b9ea4b kudu::ThreadPool::CreateThread() at ??:0
@ 0x7f1e37b9a46d kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7f1e37b9f79f kudu::ThreadPool::Submit() at ??:0
@ 0x7f1e432ddb0d kudu::log::SegmentAllocator::AsyncAllocateSegmentUnlocked() at ??:0
@ 0x7f1e432dcaff kudu::log::SegmentAllocator::AllocateSegmentAndRollOver() at ??:0
@ 0x7f1e432dc6be kudu::log::SegmentAllocator::Init() at ??:0
@ 0x7f1e432e4735 kudu::log::Log::Init() at ??:0
@ 0x7f1e432e3a91 kudu::log::Log::Open() at ??:0
@ 0x7f1e44d34b41 kudu::tablet::TabletBootstrap::OpenNewLog() at ??:0
@ 0x7f1e44d30eac kudu::tablet::TabletBootstrap::RunBootstrap() at ??:0
@ 0x7f1e44d2f62f kudu::tablet::TabletBootstrap::Bootstrap() at ??:0
I20260202 02:52:29.739228 747 tablet_service.cc:1505] Processing CreateTablet for tablet 4482b24826884d0da0d34f43f69e1646 (DEFAULT_TABLE table=default.Table_7a09e5be1e824c82bfd2d252cbd1043e [id=b5a3bd05316f4e6ab2f80b3dcfd5c1f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.739655 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4482b24826884d0da0d34f43f69e1646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.745193 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_e9aa59c2ab3343cdb4bfb8f0605bb417"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:29.745816 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e9aa59c2ab3343cdb4bfb8f0605bb417 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:29.747560 523 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 78a9a483b95c4bb5a5ab69abc899837b
I20260202 02:52:29.749307 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 78a9a483b95c4bb5a5ab69abc899837b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:29 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:29.753371 1173 tablet_bootstrap.cc:654] T dada209d982d48f3b74a49905f19c5b6 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:29.756946 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fbcb4a522df648ba9542250450c08f1b" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:29.757503 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1d8143f529b74e6ab779dd695920413c" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:29.758488 528 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_e0fefcc5497b47daa332f70f03a81639" } new_table_name: "default.Table_445635a75c87470c947b7d0731fc9307" modify_external_catalogs: true
I20260202 02:52:29.762980 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "29c996781cb947f3a1aee5dd0a9c4828" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:29.763577 859 leader_election.cc:343] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:29.692341 1182 ts_tablet_manager.cc:1916] T 2c31b45dc8004eae993e972e9faa3ff8 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:29.767189 752 tablet_service.cc:1505] Processing CreateTablet for tablet 8a17be71e0f444c9ac2b127f35420780 (DEFAULT_TABLE table=default.Table_af8ce4ca28bf4200b52f066fc4c30d63 [id=3fe28de6f2b44aa1b7312fd06115d80c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.767750 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a17be71e0f444c9ac2b127f35420780. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.772569 747 tablet_service.cc:1552] Processing DeleteTablet for tablet 3d1a7578e83447669e3c3972a426ad44 with delete_type TABLET_DATA_DELETED (replaced by tablet bdd72f995a704520a5fb1c24a53ed252) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:29.787212 1882 raft_consensus.cc:2272] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:29.787871 1882 ts_tablet_manager.cc:1916] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:29.793450 2170 raft_consensus.cc:493] T 474df41e4fe24a818fa45a4ce84d5474 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:29.793642 2170 raft_consensus.cc:515] T 474df41e4fe24a818fa45a4ce84d5474 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:29.794821 2170 leader_election.cc:290] T 474df41e4fe24a818fa45a4ce84d5474 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:29.799966 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "474df41e4fe24a818fa45a4ce84d5474" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:29.800670 859 leader_election.cc:343] T 474df41e4fe24a818fa45a4ce84d5474 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:29.803874 2296 tablet.cc:1722] T 91ec01d25f8f441cbe3783cacbee51db P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260202 02:52:29.805950 1882 ts_tablet_manager.cc:1929] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:29.806062 1882 log.cc:1199] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/4e79cd48fefb4ce29ef48beacfc4d26f
I20260202 02:52:29.810786 2297 raft_consensus.cc:493] T ccd6bc8bd0504bf18d108c73685ef9ff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:29.810940 2297 raft_consensus.cc:515] T ccd6bc8bd0504bf18d108c73685ef9ff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:29.812042 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ccd6bc8bd0504bf18d108c73685ef9ff" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:29.812372 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_19f091a0564b48e08eca8386c5222b02"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:29.812678 859 leader_election.cc:343] T ccd6bc8bd0504bf18d108c73685ef9ff P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:29.813042 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_19f091a0564b48e08eca8386c5222b02 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:29.813597 2297 leader_election.cc:290] T ccd6bc8bd0504bf18d108c73685ef9ff P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:29.816793 857 leader_election.cc:343] T 842e54adae2a452a98521de24326eaff P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:29.816937 857 leader_election.cc:304] T 842e54adae2a452a98521de24326eaff P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
W20260202 02:52:29.817365 857 leader_election.cc:343] T fbcb4a522df648ba9542250450c08f1b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:29.817469 857 leader_election.cc:304] T fbcb4a522df648ba9542250450c08f1b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:29.818179 2170 raft_consensus.cc:2749] T 842e54adae2a452a98521de24326eaff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:29.818423 1173 tablet_bootstrap.cc:492] T dada209d982d48f3b74a49905f19c5b6 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:29.818462 2170 raft_consensus.cc:2749] T fbcb4a522df648ba9542250450c08f1b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:29.818585 1173 ts_tablet_manager.cc:1403] T dada209d982d48f3b74a49905f19c5b6 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.080s user 0.005s sys 0.008s
I20260202 02:52:29.819372 1173 raft_consensus.cc:359] T dada209d982d48f3b74a49905f19c5b6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.819562 1173 raft_consensus.cc:385] T dada209d982d48f3b74a49905f19c5b6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:29.819657 1173 raft_consensus.cc:740] T dada209d982d48f3b74a49905f19c5b6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:29.819882 1173 consensus_queue.cc:260] T dada209d982d48f3b74a49905f19c5b6 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.820411 1173 ts_tablet_manager.cc:1434] T dada209d982d48f3b74a49905f19c5b6 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:29.821120 2295 tablet.cc:1722] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260202 02:52:29.822628 2268 leader_election.cc:290] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:29.822856 1173 tablet_bootstrap.cc:492] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:29.826826 759 tablet_service.cc:1552] Processing DeleteTablet for tablet d30557fbe9084cd3860820f69dda8752 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:29.827185 1174 tablet_bootstrap.cc:492] T 75569511179b41038e521fa3346a92ae P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:29.827224 759 tablet_service.cc:1552] Processing DeleteTablet for tablet fe53a8bdbd13417f9ce195c54a52a93c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:21 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:29.827309 1174 ts_tablet_manager.cc:1403] T 75569511179b41038e521fa3346a92ae P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.498s user 0.134s sys 0.160s
I20260202 02:52:29.827536 759 tablet_service.cc:1552] Processing DeleteTablet for tablet 06b80a1c9251420cbc16fc95ae98aca9 with delete_type TABLET_DATA_DELETED (replaced by tablet 2f5aae089fcc492f9ce55f84674a452c) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:29.827816 759 tablet_service.cc:1552] Processing DeleteTablet for tablet 67495e8e525542fb976f675b71512ed2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:29 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:29.828114 759 tablet_service.cc:1552] Processing DeleteTablet for tablet 8dd62de8e8234d6c9ce380b1aadc097d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:29 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:29.828033 1174 raft_consensus.cc:359] T 75569511179b41038e521fa3346a92ae P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.828186 1174 raft_consensus.cc:385] T 75569511179b41038e521fa3346a92ae P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:29.828231 1174 raft_consensus.cc:740] T 75569511179b41038e521fa3346a92ae P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:29.828429 1174 consensus_queue.cc:260] T 75569511179b41038e521fa3346a92ae P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.828905 1174 ts_tablet_manager.cc:1434] T 75569511179b41038e521fa3346a92ae P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:29.829170 1174 tablet_bootstrap.cc:492] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
W20260202 02:52:29.829156 521 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 41a4143247ed4eed98defbdfe2895372 consensus state has no leader
W20260202 02:52:29.829370 715 consensus_peers.cc:597] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 08d8d781bb4e4acf915f11afb4fb2e22. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:29.829460 521 master_service.cc:954] ReplaceTablet: received request to replace tablet 842197c689064660b791b40e521d346b from {username='slave'} at 127.0.0.1:47430
W20260202 02:52:29.829581 715 leader_election.cc:343] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Not found: Tablet not found: 38fdbc6df7ec40b9afe4e0dfe0119e3e
I20260202 02:52:29.830140 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_6f5ad3d161b6498ebccd807c6d76430f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:29.830614 1174 tablet_bootstrap.cc:654] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:29.830677 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6f5ad3d161b6498ebccd807c6d76430f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:29.832085 715 consensus_peers.cc:597] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 68823eff1bc04108ba83487605aec0ea. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:29.832245 715 consensus_peers.cc:597] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c25e329023c04f609d8a743b4f7a95a4. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:29.833324 1173 tablet_bootstrap.cc:654] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:29.833557 2301 tablet.cc:1722] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260202 02:52:29.837011 522 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e33f95e4f17a479c9acef5764c6c4c56 consensus state has no leader
I20260202 02:52:29.837453 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_45aac0057c8c4bae852d155bff2ed712"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:29.838088 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_45aac0057c8c4bae852d155bff2ed712 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:29.843133 756 tablet_service.cc:1505] Processing CreateTablet for tablet 365417e050304073b5de887082fc699b (DEFAULT_TABLE table=default.Table_25f0023dd22a4bb0a7c2863da66d0002 [id=e24ffad12f514d60907b6cadfd09dbe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:29.843547 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 365417e050304073b5de887082fc699b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:29.844067 1174 tablet_bootstrap.cc:492] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:29.844154 1882 ts_tablet_manager.cc:1950] T 4e79cd48fefb4ce29ef48beacfc4d26f P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:29.844183 1174 ts_tablet_manager.cc:1403] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.001s
I20260202 02:52:29.845078 1174 raft_consensus.cc:359] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.845250 1174 raft_consensus.cc:385] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:29.845295 1174 raft_consensus.cc:740] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:29.845242 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 541) took 2920 ms. Trace:
I20260202 02:52:29.845392 1882 rpcz_store.cc:276] 0202 02:52:26.925105 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.948725 (+ 23620us) service_pool.cc:225] Handling call
0202 02:52:29.716450 (+2767725us) ts_tablet_manager.cc:1151] Deleting tablet 4e79cd48fefb4ce29ef48beacfc4d26f
0202 02:52:29.716464 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.784717 (+ 68253us) spinlock_profiling.cc:243] Waited 54.7 ms on lock 0x617000057b70. stack: 00007fec53205a86 00007fec53205840 00007fec521e03ad 0000000000984f94 0000000000984f5d 000000000098492a 00007fec5efd282f 00007fec5f08301c 00007fec5f0f32c3 00007fec604410b3 00007fec63001757 00007fec63034744 00007fec63015240 00007fec63014fcd 0000000000a8970c 00007fec5324bdaf
0202 02:52:29.791036 (+ 6319us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.805818 (+ 14782us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.845190 (+ 39372us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":125866368,"tablet-delete.queue_time_us":2767421}
I20260202 02:52:29.845518 1174 consensus_queue.cc:260] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.845999 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 4e79cd48fefb4ce29ef48beacfc4d26f (table default.Table_bed5f0c218b24f529e48138901d1afa0 [id=2a625eeef55b43f19cd4bbeb61e85bca]) successfully deleted
I20260202 02:52:29.846068 1174 ts_tablet_manager.cc:1434] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:29.846338 1174 tablet_bootstrap.cc:492] T fe9ea4c0502f4976b440465992f2ea2c P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:29.847631 1174 tablet_bootstrap.cc:654] T fe9ea4c0502f4976b440465992f2ea2c P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:29.849490 1174 tablet_bootstrap.cc:492] T fe9ea4c0502f4976b440465992f2ea2c P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:29.849609 1174 ts_tablet_manager.cc:1403] T fe9ea4c0502f4976b440465992f2ea2c P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260202 02:52:29.850404 1174 raft_consensus.cc:359] T fe9ea4c0502f4976b440465992f2ea2c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.850550 1174 raft_consensus.cc:385] T fe9ea4c0502f4976b440465992f2ea2c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:29.850597 1174 raft_consensus.cc:740] T fe9ea4c0502f4976b440465992f2ea2c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:29.850843 1174 consensus_queue.cc:260] T fe9ea4c0502f4976b440465992f2ea2c P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.851257 1173 tablet_bootstrap.cc:492] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:29.851418 1173 ts_tablet_manager.cc:1403] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.029s user 0.006s sys 0.003s
I20260202 02:52:29.851648 1174 ts_tablet_manager.cc:1434] T fe9ea4c0502f4976b440465992f2ea2c P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:29.851913 1174 tablet_bootstrap.cc:492] T be993c1512b04df4b92cab3195241d84 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:29.852109 1173 raft_consensus.cc:359] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.852290 1173 raft_consensus.cc:385] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:29.852387 1173 raft_consensus.cc:740] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:29.852646 1173 consensus_queue.cc:260] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.853120 1173 ts_tablet_manager.cc:1434] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:29.854271 1174 tablet_bootstrap.cc:654] T be993c1512b04df4b92cab3195241d84 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:29.855198 759 tablet_service.cc:1552] Processing DeleteTablet for tablet 1d98c7a46139413b8f0d3697b74b1e52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:29 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:29.857318 759 tablet_service.cc:1552] Processing DeleteTablet for tablet 78a9a483b95c4bb5a5ab69abc899837b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:29 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:29.862025 1174 tablet_bootstrap.cc:492] T be993c1512b04df4b92cab3195241d84 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
W20260202 02:52:29.862002 714 leader_election.cc:343] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 06b80a1c9251420cbc16fc95ae98aca9
I20260202 02:52:29.862174 1174 ts_tablet_manager.cc:1403] T be993c1512b04df4b92cab3195241d84 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.007s
I20260202 02:52:29.862135 1721 raft_consensus.cc:493] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260202 02:52:29.862326 714 leader_election.cc:343] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 06b80a1c9251420cbc16fc95ae98aca9
I20260202 02:52:29.862293 1721 raft_consensus.cc:515] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.863068 1721 leader_election.cc:290] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:29.863304 1174 raft_consensus.cc:359] T be993c1512b04df4b92cab3195241d84 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.863415 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "874ca4ba4ebc4721bb4b60cba8e01841" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:29.863458 1174 raft_consensus.cc:385] T be993c1512b04df4b92cab3195241d84 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:29.863520 1173 tablet_bootstrap.cc:492] T 79c73f4a92044bafb5b8384229cb3a4d P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:29.863524 1174 raft_consensus.cc:740] T be993c1512b04df4b92cab3195241d84 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
W20260202 02:52:29.863821 998 leader_election.cc:343] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:29.863833 1174 consensus_queue.cc:260] T be993c1512b04df4b92cab3195241d84 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
W20260202 02:52:29.863876 857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 46506us
W20260202 02:52:29.864111 857 leader_election.cc:343] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:29.864202 857 leader_election.cc:304] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:29.864457 1174 ts_tablet_manager.cc:1434] T be993c1512b04df4b92cab3195241d84 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:29.864717 1174 tablet_bootstrap.cc:492] T 26b890ed87c3471b9fc0b172c90e0a11 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:29.865316 1173 tablet_bootstrap.cc:654] T 79c73f4a92044bafb5b8384229cb3a4d P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:29.865994 1174 tablet_bootstrap.cc:654] T 26b890ed87c3471b9fc0b172c90e0a11 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:29.865949 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 1d98c7a46139413b8f0d3697b74b1e52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:29 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:29.866358 2268 raft_consensus.cc:2749] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:29.870195 526 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bcc6cd3595364680b4c39c1ff740aebb consensus state has no leader
I20260202 02:52:29.870929 1174 tablet_bootstrap.cc:492] T 26b890ed87c3471b9fc0b172c90e0a11 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:29.871068 1174 ts_tablet_manager.cc:1403] T 26b890ed87c3471b9fc0b172c90e0a11 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.003s
I20260202 02:52:29.871912 1174 raft_consensus.cc:359] T 26b890ed87c3471b9fc0b172c90e0a11 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.872081 1174 raft_consensus.cc:385] T 26b890ed87c3471b9fc0b172c90e0a11 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:29.872121 1174 raft_consensus.cc:740] T 26b890ed87c3471b9fc0b172c90e0a11 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:29.872370 1174 consensus_queue.cc:260] T 26b890ed87c3471b9fc0b172c90e0a11 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.872989 1174 ts_tablet_manager.cc:1434] T 26b890ed87c3471b9fc0b172c90e0a11 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:29.873261 1174 tablet_bootstrap.cc:492] T 02b434fa058b45529ff3938b053263c6 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:29.874588 1174 tablet_bootstrap.cc:654] T 02b434fa058b45529ff3938b053263c6 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:29.875161 840 kernel_stack_watchdog.cc:198] Thread 1175 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:642 for 400ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fb3588d9980 (unknown) at ??:0
@ 0x93e827 __sanitizer::internal_sched_yield() at ??:?
@ 0x8a3edd __interceptor_pthread_create at ??:?
@ 0x7fb3552c0c91 kudu::Thread::StartThread() at ??:0
@ 0xa76197 kudu::Thread::Create() at ??:?
@ 0x7fb3552e0a4b kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fb3552dc46d kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fb3552e179f kudu::ThreadPool::Submit() at ??:0
@ 0x7fb360a1fb0d kudu::log::SegmentAllocator::AsyncAllocateSegmentUnlocked() at ??:0
@ 0x7fb360a1eaff kudu::log::SegmentAllocator::AllocateSegmentAndRollOver() at ??:0
@ 0x7fb360a1e6be kudu::log::SegmentAllocator::Init() at ??:0
@ 0x7fb360a26735 kudu::log::Log::Init() at ??:0
@ 0x7fb360a25a91 kudu::log::Log::Open() at ??:0
@ 0x7fb362476b41 kudu::tablet::TabletBootstrap::OpenNewLog() at ??:0
@ 0x7fb362472eac kudu::tablet::TabletBootstrap::RunBootstrap() at ??:0
@ 0x7fb36247162f kudu::tablet::TabletBootstrap::Bootstrap() at ??:0
I20260202 02:52:29.876817 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 67495e8e525542fb976f675b71512ed2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:29 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:29.878118 1174 tablet_bootstrap.cc:492] T 02b434fa058b45529ff3938b053263c6 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:29.878232 1174 ts_tablet_manager.cc:1403] T 02b434fa058b45529ff3938b053263c6 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260202 02:52:29.878913 1174 raft_consensus.cc:359] T 02b434fa058b45529ff3938b053263c6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.879063 1174 raft_consensus.cc:385] T 02b434fa058b45529ff3938b053263c6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:29.879102 1174 raft_consensus.cc:740] T 02b434fa058b45529ff3938b053263c6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:29.879508 1182 ts_tablet_manager.cc:1929] T 2c31b45dc8004eae993e972e9faa3ff8 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:29.879606 1182 log.cc:1199] T 2c31b45dc8004eae993e972e9faa3ff8 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/2c31b45dc8004eae993e972e9faa3ff8
I20260202 02:52:29.879949 1182 ts_tablet_manager.cc:1950] T 2c31b45dc8004eae993e972e9faa3ff8 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:29.880391 1182 tablet_replica.cc:333] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:29.880573 1182 raft_consensus.cc:2243] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:29.880733 1182 raft_consensus.cc:2272] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:29.881220 1182 ts_tablet_manager.cc:1916] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:29.882449 715 consensus_peers.cc:597] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9c1271ce0f194021b588b82ac9d3db79. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:29.882895 1059 raft_consensus.cc:1275] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Refusing update from remote peer 1f442ef85029461091de847118c34c0d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260202 02:52:29.883213 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 78a9a483b95c4bb5a5ab69abc899837b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:29 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:29.884145 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ccd6bc8bd0504bf18d108c73685ef9ff" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:29.884299 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 2c31b45dc8004eae993e972e9faa3ff8 (table default.Table_5db99844c1bb484a863a9e289a19da3e [id=f1f6c97fc88340da8bab0347ad6430ab]) successfully deleted
I20260202 02:52:29.884549 2301 tablet.cc:1722] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
W20260202 02:52:29.884728 999 leader_election.cc:343] T 26eeef6eebbf4f2cb44f69255fb37594 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 26eeef6eebbf4f2cb44f69255fb37594
I20260202 02:52:29.884760 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "29c996781cb947f3a1aee5dd0a9c4828" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:29.884886 999 leader_election.cc:304] T 26eeef6eebbf4f2cb44f69255fb37594 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:29.885326 1721 raft_consensus.cc:2749] T 26eeef6eebbf4f2cb44f69255fb37594 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:29.885406 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "474df41e4fe24a818fa45a4ce84d5474" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:29.890846 1175 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.626s user 0.073s sys 0.073s
W20260202 02:52:29.890954 1175 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.626s user 0.073s sys 0.073s
I20260202 02:52:29.891852 1182 ts_tablet_manager.cc:1929] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:29.891961 1182 log.cc:1199] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/1116872cc25a4ab5b1a3a6bbba1f0da7
I20260202 02:52:29.892272 1182 ts_tablet_manager.cc:1950] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:29.892524 2270 consensus_queue.cc:1048] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260202 02:52:29.892715 1182 tablet_replica.cc:333] T 6df3d11cf578404ca41807f0777b6c53 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:29.892848 2150 raft_consensus.cc:493] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:29.892982 2150 raft_consensus.cc:515] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:29.893422 1175 tablet_bootstrap.cc:492] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:29.893544 1175 ts_tablet_manager.cc:1403] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.631s user 0.075s sys 0.073s
I20260202 02:52:29.893553 2150 leader_election.cc:290] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:29.893931 1182 raft_consensus.cc:2243] T 6df3d11cf578404ca41807f0777b6c53 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:29.894269 1175 raft_consensus.cc:359] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:29.894393 1182 raft_consensus.cc:2272] T 6df3d11cf578404ca41807f0777b6c53 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:29.894917 1182 ts_tablet_manager.cc:1916] T 6df3d11cf578404ca41807f0777b6c53 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:29.895587 1174 consensus_queue.cc:260] T 02b434fa058b45529ff3938b053263c6 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:29.894408 1175 raft_consensus.cc:385] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:29.896122 1175 raft_consensus.cc:740] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:29.896253 1174 ts_tablet_manager.cc:1434] T 02b434fa058b45529ff3938b053263c6 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.018s user 0.001s sys 0.001s
I20260202 02:52:29.896716 1174 tablet_bootstrap.cc:492] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:29.897442 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 8dd62de8e8234d6c9ce380b1aadc097d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:29 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:29.898090 526 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet 16b47a2831f54d809ed0b16deaec7e3d (table default.Table_e4018d1c57714a24be5d3a20c1ccaef4 [id=1e291b35a6764638acfb781bb4e6adb3]). Expected version 2 got 0
I20260202 02:52:29.898046 1175 consensus_queue.cc:260] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:29.898168 1180 ts_tablet_manager.cc:1929] T 1f54c1dfe0a942b3bafd85522953a032 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:29.898258 1180 log.cc:1199] T 1f54c1dfe0a942b3bafd85522953a032 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/1f54c1dfe0a942b3bafd85522953a032
I20260202 02:52:29.899180 1180 ts_tablet_manager.cc:1950] T 1f54c1dfe0a942b3bafd85522953a032 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:29.899442 1175 ts_tablet_manager.cc:1434] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260202 02:52:29.900365 1175 tablet_bootstrap.cc:492] T d0ce2121d65b4a228596e0ca9018883d P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:29.901602 520 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet be993c1512b04df4b92cab3195241d84
I20260202 02:52:29.901691 1175 tablet_bootstrap.cc:654] T d0ce2121d65b4a228596e0ca9018883d P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:29.902381 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_5c8329977b724148a1a8d5c504a4a513"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:29.905571 920 raft_consensus.cc:3060] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:30.187927 1906 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.190:33981
W20260202 02:52:29.902923 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5c8329977b724148a1a8d5c504a4a513 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:29.907446 759 tablet_service.cc:1552] Processing DeleteTablet for tablet 26eeef6eebbf4f2cb44f69255fb37594 with delete_type TABLET_DATA_DELETED (replaced by tablet dc4c3f52f4f44a84b0d0e3c9a4a49f09) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:29.904997 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_9412ac9f6f6c495f917ebcf68650e5f1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:29.914011 857 leader_election.cc:343] T ccd6bc8bd0504bf18d108c73685ef9ff P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:29.914803 891 tablet_service.cc:1552] Processing DeleteTablet for tablet be993c1512b04df4b92cab3195241d84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:29 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:30.236770 857 leader_election.cc:304] T ccd6bc8bd0504bf18d108c73685ef9ff P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:30.237010 1906 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/master-0/data/info.pb
I20260202 02:52:29.907990 742 tablet_service.cc:1552] Processing DeleteTablet for tablet be993c1512b04df4b92cab3195241d84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:29 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:29.914517 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_a0621e0a3e8047fda1e5226ee396d4e8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:30.237146 857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 323342us
I20260202 02:52:29.915994 1174 tablet_bootstrap.cc:654] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:30.237540 857 leader_election.cc:343] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:30.237602 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a0621e0a3e8047fda1e5226ee396d4e8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:30.237634 857 leader_election.cc:304] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:29.903820 1058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fe53a8bdbd13417f9ce195c54a52a93c" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:30.237888 1058 raft_consensus.cc:2468] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
W20260202 02:52:30.237985 857 leader_election.cc:343] T 474df41e4fe24a818fa45a4ce84d5474 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:30.238063 857 leader_election.cc:304] T 474df41e4fe24a818fa45a4ce84d5474 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:30.188428 2363 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.190:33981 every 8 connection(s)
I20260202 02:52:29.934058 2301 tablet.cc:1722] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:29.919693 920 raft_consensus.cc:1275] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Refusing update from remote peer 1f442ef85029461091de847118c34c0d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260202 02:52:30.238564 2236 raft_consensus.cc:2749] T ccd6bc8bd0504bf18d108c73685ef9ff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:29.923607 527 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet eea6bb04ddf24276b46270d009ed78b4 consensus state has no leader
I20260202 02:52:30.238982 2236 raft_consensus.cc:2749] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:30.239125 2271 raft_consensus.cc:2749] T 474df41e4fe24a818fa45a4ce84d5474 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:30.239202 527 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_7ff322b57db648b3b480519c357a083b" } modify_external_catalogs: true
I20260202 02:52:29.946735 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 468) took 5229 ms. Trace:
I20260202 02:52:30.239413 527 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_7ff322b57db648b3b480519c357a083b" } modify_external_catalogs: true
I20260202 02:52:30.239488 1180 rpcz_store.cc:276] 0202 02:52:24.717050 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.718385 (+ 1335us) service_pool.cc:225] Handling call
0202 02:52:29.558728 (+4840343us) ts_tablet_manager.cc:1151] Deleting tablet 1f54c1dfe0a942b3bafd85522953a032
0202 02:52:29.558744 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:29.563361 (+ 4617us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.898057 (+334696us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:29.946710 (+ 48653us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4828313}
I20260202 02:52:30.239840 1180 tablet_replica.cc:333] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:30.240023 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskYXxklX/build/asan/bin/kudu as pid 1906
I20260202 02:52:30.240031 1180 raft_consensus.cc:2243] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:30.240279 1180 pending_rounds.cc:70] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d: Trying to abort 1 pending ops.
I20260202 02:52:30.240343 1180 pending_rounds.cc:77] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923066354032640 op_type: NO_OP noop_request { }
I20260202 02:52:30.240432 1180 raft_consensus.cc:2889] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:30.240509 1180 raft_consensus.cc:2272] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:29.916852 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet be993c1512b04df4b92cab3195241d84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:29 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:29.937757 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_df42f88a6807472088543ed874ac5e76"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:30.241150 1180 ts_tablet_manager.cc:1916] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:30.241382 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_df42f88a6807472088543ed874ac5e76 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:29.935202 2297 raft_consensus.cc:493] T 1f379a29e2454f689d68f5951837512b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:29.926309 1882 tablet_replica.cc:333] T 13cd1383348b458894fdecc22bf24b74 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
W20260202 02:52:29.946125 515 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet bb145df67e1a44569a3dbc91c024b584 (table default.Table_0eedfe3d0e4c4ba08086d0cf45e438ae [id=bf2ff444381f4d62a22258ffbfe1c040], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet bb145df67e1a44569a3dbc91c024b584 consensus state has no leader
I20260202 02:52:30.243371 2297 raft_consensus.cc:515] T 1f379a29e2454f689d68f5951837512b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:30.244066 2297 leader_election.cc:290] T 1f379a29e2454f689d68f5951837512b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:29.942662 1173 tablet_bootstrap.cc:492] T 79c73f4a92044bafb5b8384229cb3a4d P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:30.244509 1173 ts_tablet_manager.cc:1403] T 79c73f4a92044bafb5b8384229cb3a4d P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.381s user 0.006s sys 0.013s
I20260202 02:52:30.245031 777 raft_consensus.cc:3060] T be993c1512b04df4b92cab3195241d84 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:30.245293 1173 raft_consensus.cc:359] T 79c73f4a92044bafb5b8384229cb3a4d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:30.245472 1173 raft_consensus.cc:385] T 79c73f4a92044bafb5b8384229cb3a4d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:30.245558 1173 raft_consensus.cc:740] T 79c73f4a92044bafb5b8384229cb3a4d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:30.245777 1173 consensus_queue.cc:260] T 79c73f4a92044bafb5b8384229cb3a4d P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:30.246217 1173 ts_tablet_manager.cc:1434] T 79c73f4a92044bafb5b8384229cb3a4d P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
W20260202 02:52:29.947715 528 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6f81123900e648be8a117e1b925319e8 consensus state has no leader
I20260202 02:52:30.246594 528 master_service.cc:954] ReplaceTablet: received request to replace tablet d8418a9dd0944456b4d4820dde0cde02 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:29.957054 519 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_23afb316ffac47aea4db07bda15a956b" } modify_external_catalogs: true
I20260202 02:52:30.247769 519 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_23afb316ffac47aea4db07bda15a956b" } modify_external_catalogs: true
I20260202 02:52:29.978211 2157 raft_consensus.cc:493] T 26b890ed87c3471b9fc0b172c90e0a11 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:29.959903 521 master_service.cc:954] ReplaceTablet: received request to replace tablet fbcb4a522df648ba9542250450c08f1b from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:29.971752 522 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_a7d94a8b6efd406c845b24e72a7d4d25" } modify_external_catalogs: true
I20260202 02:52:29.977172 523 catalog_manager.cc:5654] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130). New cstate: current_term: 1 leader_uuid: "1521a6d262b7451bbef8debe970dd6b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } }
I20260202 02:52:30.249949 522 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_a7d94a8b6efd406c845b24e72a7d4d25" } modify_external_catalogs: true
I20260202 02:52:30.250109 523 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 26b890ed87c3471b9fc0b172c90e0a11 (table default.Table_044ef6a2b1af4e3cab3c8cf00505c6a6 [id=d2e4220a0cbf443e8380433da4f064f8]). Expected version 1 got 0
I20260202 02:52:30.250336 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f379a29e2454f689d68f5951837512b" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:29.912818 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 1116872cc25a4ab5b1a3a6bbba1f0da7 (table default.Table_d6ba351d68ec4a10939a09519b7d4b67 [id=52a6d31f993d4c74a520d504390cf12b]) successfully deleted
W20260202 02:52:30.250761 514 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 337935us
I20260202 02:52:30.099638 2189 raft_consensus.cc:493] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:30.251353 2189 raft_consensus.cc:515] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
W20260202 02:52:30.251729 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9412ac9f6f6c495f917ebcf68650e5f1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:30.251942 2189 leader_election.cc:290] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:29.945963 1182 ts_tablet_manager.cc:1929] T 6df3d11cf578404ca41807f0777b6c53 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:30.252225 1182 log.cc:1199] T 6df3d11cf578404ca41807f0777b6c53 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/6df3d11cf578404ca41807f0777b6c53
W20260202 02:52:29.913127 714 leader_election.cc:343] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 38fdbc6df7ec40b9afe4e0dfe0119e3e
W20260202 02:52:30.169394 715 consensus_peers.cc:597] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 757492288d7c44fb91b25b7e6e21e01b. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:30.252643 714 leader_election.cc:304] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
W20260202 02:52:30.252839 715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 83417.6us
I20260202 02:52:30.252945 2127 consensus_queue.cc:1048] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260202 02:52:30.253047 714 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 339902us
W20260202 02:52:30.253410 714 consensus_peers.cc:597] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9c1271ce0f194021b588b82ac9d3db79. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:30.253509 715 leader_election.cc:304] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
W20260202 02:52:30.253619 714 leader_election.cc:343] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: fe53a8bdbd13417f9ce195c54a52a93c
W20260202 02:52:30.207185 700 kernel_stack_watchdog.cc:198] Thread 714 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 227ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f1e3b197980 (unknown) at ??:0
@ 0x7f1e3b1919c6 __GI___pthread_rwlock_wrlock at ??:0
@ 0x7f1e34e0a904 google::LogMessage::Flush() at ??:0
@ 0x7f1e34e0aa11 google::LogMessage::~LogMessage() at ??:0
@ 0x7f1e43998503 kudu::consensus::LeaderElection::VoteResponseRpcCallback() at ??:0
@ 0x7f1e4399b07a kudu::consensus::LeaderElection::Run()::$_0::operator()() at ??:0
@ 0x7f1e4399ae3e std::_Function_handler<>::_M_invoke() at ??:0
@ 0xa8970d std::function<>::operator()() at ??:?
@ 0x7f1e3ea8c2df kudu::rpc::Proxy::AsyncRequest()::$_1::operator()() at ??:0
@ 0x7f1e3ea8bebe std::_Function_handler<>::_M_invoke() at ??:0
@ 0xa8970d std::function<>::operator()() at ??:?
@ 0x7f1e3ea77f10 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f1e3ea7845c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f1e3ea1381a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f1e3ea121d6 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f1e3ea246a5 ev::base<>::method_thunk<>() at ??:0
I20260202 02:52:30.253926 2150 raft_consensus.cc:2749] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:30.254271 2270 raft_consensus.cc:2804] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:30.254335 2270 raft_consensus.cc:493] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:30.254385 2270 raft_consensus.cc:3060] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:30.255193 999 leader_election.cc:343] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 874ca4ba4ebc4721bb4b60cba8e01841
I20260202 02:52:30.255337 999 leader_election.cc:304] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:30.255712 2157 raft_consensus.cc:515] T 26b890ed87c3471b9fc0b172c90e0a11 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:30.256281 2157 leader_election.cc:290] T 26b890ed87c3471b9fc0b172c90e0a11 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:30.256588 2157 consensus_queue.cc:1048] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:30.256922 2270 raft_consensus.cc:515] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:30.257427 2270 leader_election.cc:290] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:29.953503 1175 tablet_bootstrap.cc:492] T d0ce2121d65b4a228596e0ca9018883d P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:30.257853 1175 ts_tablet_manager.cc:1403] T d0ce2121d65b4a228596e0ca9018883d P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.358s user 0.004s sys 0.000s
I20260202 02:52:30.257890 1882 raft_consensus.cc:2243] T 13cd1383348b458894fdecc22bf24b74 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:30.258074 1882 raft_consensus.cc:2272] T 13cd1383348b458894fdecc22bf24b74 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:30.258620 1882 ts_tablet_manager.cc:1916] T 13cd1383348b458894fdecc22bf24b74 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:30.258611 1175 raft_consensus.cc:359] T d0ce2121d65b4a228596e0ca9018883d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:30.258764 1175 raft_consensus.cc:385] T d0ce2121d65b4a228596e0ca9018883d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:30.258811 1175 raft_consensus.cc:740] T d0ce2121d65b4a228596e0ca9018883d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:30.259011 1175 consensus_queue.cc:260] T d0ce2121d65b4a228596e0ca9018883d P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:30.259287 1182 ts_tablet_manager.cc:1950] T 6df3d11cf578404ca41807f0777b6c53 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:30.259470 1175 ts_tablet_manager.cc:1434] T d0ce2121d65b4a228596e0ca9018883d P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:30.259718 1175 tablet_bootstrap.cc:492] T ac2ab12f5040488aa306cb69b107b485 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:30.260171 1182 tablet_replica.cc:333] T ccd6bc8bd0504bf18d108c73685ef9ff P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
W20260202 02:52:30.138095 840 kernel_stack_watchdog.cc:198] Thread 857 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 161ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fb3588d9980 (unknown) at ??:0
@ 0x7fb3513ec255 __write at ??:0
@ 0x7fb35136728d _IO_file_write at ??:0
@ 0x7fb351367bff _IO_file_xsputn at ??:0
@ 0x7fb35135ba57 fwrite at ??:0
@ 0x8a8c97 __interceptor_fwrite.part.56 at ??:?
@ 0x7fb3525508a0 google::LogMessage::SendToLog() at ??:0
@ 0x7fb35254c7cc google::LogMessage::Flush() at ??:0
@ 0x7fb35254ca11 google::LogMessage::~LogMessage() at ??:0
@ 0x7fb3610da503 kudu::consensus::LeaderElection::VoteResponseRpcCallback() at ??:0
@ 0x7fb3610dd07a kudu::consensus::LeaderElection::Run()::$_0::operator()() at ??:0
@ 0x7fb3610dce3e std::_Function_handler<>::_M_invoke() at ??:0
@ 0xa8970d std::function<>::operator()() at ??:?
@ 0x7fb35c1ce2df kudu::rpc::Proxy::AsyncRequest()::$_1::operator()() at ??:0
@ 0x7fb35c1cdebe std::_Function_handler<>::_M_invoke() at ??:0
@ 0xa8970d std::function<>::operator()() at ??:?
I20260202 02:52:30.260362 1182 raft_consensus.cc:2243] T ccd6bc8bd0504bf18d108c73685ef9ff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:30.260499 1182 raft_consensus.cc:2272] T ccd6bc8bd0504bf18d108c73685ef9ff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:30.261011 1182 ts_tablet_manager.cc:1916] T ccd6bc8bd0504bf18d108c73685ef9ff P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:30.261108 2127 raft_consensus.cc:2749] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:30.265244 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 104b2c537c4f4e3da6711364cb5d2a2e (DEFAULT_TABLE table=default.Table_6da2058f0c8c44feb77c1106bda2bffe [id=2ae99f38f2464e099ac578796258f825]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.275810 1175 tablet_bootstrap.cc:654] T ac2ab12f5040488aa306cb69b107b485 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:30.295578 2297 raft_consensus.cc:493] T d0ce2121d65b4a228596e0ca9018883d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:30.301865 1182 ts_tablet_manager.cc:1929] T ccd6bc8bd0504bf18d108c73685ef9ff P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260202 02:52:30.273851 715 consensus_peers.cc:597] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d0671586e45e45ffb7fd9e1be6e2bbf8. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:30.536703 1182 log.cc:1199] T ccd6bc8bd0504bf18d108c73685ef9ff P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/ccd6bc8bd0504bf18d108c73685ef9ff
W20260202 02:52:30.536779 715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 262949us
I20260202 02:52:30.537072 1182 ts_tablet_manager.cc:1950] T ccd6bc8bd0504bf18d108c73685ef9ff P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:30.265396 1032 tablet_service.cc:1505] Processing CreateTablet for tablet 2654ace469dd42389a0d64bf27237d75 (DEFAULT_TABLE table=default.Table_da4a1a30b6e34996a02ab80a072959d4 [id=cb32193d435e488589469afb8653d656]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.331380 2374 raft_consensus.cc:493] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:30.295531 2301 tablet.cc:1722] T 16b47a2831f54d809ed0b16deaec7e3d P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:30.537875 2374 raft_consensus.cc:515] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:30.326778 1180 ts_tablet_manager.cc:1929] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:30.538331 1180 log.cc:1199] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/c25e329023c04f609d8a743b4f7a95a4
I20260202 02:52:30.538540 2374 leader_election.cc:290] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:30.381165 2372 sys_catalog.cc:263] Verifying existing consensus state
I20260202 02:52:30.538703 1180 ts_tablet_manager.cc:1950] T c25e329023c04f609d8a743b4f7a95a4 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:30.539503 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 471) took 5749 ms. Trace:
I20260202 02:52:30.539584 1180 rpcz_store.cc:276] 0202 02:52:24.790029 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.790091 (+ 62us) service_pool.cc:225] Handling call
0202 02:52:30.239782 (+5449691us) ts_tablet_manager.cc:1151] Deleting tablet c25e329023c04f609d8a743b4f7a95a4
0202 02:52:30.239795 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:30.281909 (+ 42114us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:30.326639 (+ 44730us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:30.539483 (+212844us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5441407}
I20260202 02:52:30.539891 1180 tablet_replica.cc:333] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:30.265523 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 91a604fb877a48479914a5b60144a22d (DEFAULT_TABLE table=default.Table_7ff322b57db648b3b480519c357a083b [id=e5b1821be6a74d9da2841d2c01c96e03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.265677 1041 tablet_service.cc:1505] Processing CreateTablet for tablet 532681264d6b4ac8940f5bc79eab3d1c (DEFAULT_TABLE table=default.Table_abc8ed4a399d4fdb9479666c6368f587 [id=732d42755bc84f72ae7e0960e2ae2749]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.540074 1180 raft_consensus.cc:2243] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:30.265990 1033 tablet_service.cc:1505] Processing CreateTablet for tablet 6466c63eb97a4a21b29f4f0871e8973c (DEFAULT_TABLE table=default.Table_c57661c84d424228a977c159e893db51 [id=9e3c1e6d5d7b44caa219cb744275d833]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.266273 1042 tablet_service.cc:1505] Processing CreateTablet for tablet c584f3163f1b45789492ccd27b7ad609 (DEFAULT_TABLE table=default.Table_1a44a09f9b4f49a9a5f1f9f2529b5a52 [id=183241be2ed84df9baa79bcc113db1b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.266551 1028 tablet_service.cc:1505] Processing CreateTablet for tablet c4649f7e97bb44f2b7dca47b26379791 (DEFAULT_TABLE table=default.Table_eea651758bad47dfa286f8dcea6787b6 [id=8ea252bb1b6148458be999c715bb331e]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:30.334211 857 leader_election.cc:343] T 1f379a29e2454f689d68f5951837512b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:30.267233 1039 tablet_service.cc:1505] Processing CreateTablet for tablet e04960479c92479b8bb864544dd0c8ac (DEFAULT_TABLE table=default.Table_f21cdfd093884683ab4ec1af5345e9d5 [id=ddab3068794747a89a4950dc1ba79b5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.299841 527 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 91a604fb877a48479914a5b60144a22d
I20260202 02:52:30.267546 1036 tablet_service.cc:1505] Processing CreateTablet for tablet c707180fc9c5457292a8f43bf676f4f9 (DEFAULT_TABLE table=default.Table_ae634747dc6e4a789c36df9b29d861c0 [id=150c028675db47809195ba67ba2ba008]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.540369 1180 raft_consensus.cc:2272] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:30.267876 1038 tablet_service.cc:1505] Processing CreateTablet for tablet ae08e323fb9a4011ac313faf4ca70090 (DEFAULT_TABLE table=default.Table_987beb2b868c44dd90fb8669aab44ca7 [id=787f9ee3032c4116b30df1c23382e358]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:30.540395 857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 206176us
I20260202 02:52:30.268169 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 95b95947aad94b849303eb344d18fd27 (DEFAULT_TABLE table=default.Table_5201d08d823b4303b0be4f42a443dd2d [id=128a2dfa4c03443fb7de8077b910376a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.268586 1173 tablet_bootstrap.cc:492] T 6b8f924aecf44e959e47d2f32e6b0bf5 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:30.268891 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26eeef6eebbf4f2cb44f69255fb37594" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:30.540758 1063 raft_consensus.cc:2468] T 26eeef6eebbf4f2cb44f69255fb37594 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:30.269393 1058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f379a29e2454f689d68f5951837512b" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:30.540932 1180 ts_tablet_manager.cc:1916] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:30.276961 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 104b2c537c4f4e3da6711364cb5d2a2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:30.541260 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_63e36ff851f849849172c3c8e473214e" } new_table_name: "default.Table_150ccd742b6f4b208f42e6d0876494e5" modify_external_catalogs: true
W20260202 02:52:30.337508 714 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 83210.5us
I20260202 02:52:30.330871 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 6df3d11cf578404ca41807f0777b6c53 (table default.Table_d1815e87de114d408122224276266f69 [id=4a43e638e8854fcda2702463684c08d7]) successfully deleted
W20260202 02:52:30.542496 514 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 211667us
W20260202 02:52:30.542737 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 7665dbea16d14f619af57ea056639eef because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7665dbea16d14f619af57ea056639eef already in progress: creating tablet
I20260202 02:52:30.338511 1175 tablet_bootstrap.cc:492] T ac2ab12f5040488aa306cb69b107b485 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
W20260202 02:52:30.542943 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 842197c689064660b791b40e521d346b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 842197c689064660b791b40e521d346b already in progress: creating tablet
I20260202 02:52:30.543064 1175 ts_tablet_manager.cc:1403] T ac2ab12f5040488aa306cb69b107b485 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.283s user 0.009s sys 0.001s
W20260202 02:52:30.543139 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet e7df9a78735547cbbaa006c604b95fec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e7df9a78735547cbbaa006c604b95fec already in progress: creating tablet
W20260202 02:52:30.543340 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet fc3bfd542b8e41d98586645358061261 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fc3bfd542b8e41d98586645358061261 already in progress: creating tablet
I20260202 02:52:30.543818 1175 raft_consensus.cc:359] T ac2ab12f5040488aa306cb69b107b485 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:30.543999 1175 raft_consensus.cc:385] T ac2ab12f5040488aa306cb69b107b485 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:30.544056 1175 raft_consensus.cc:740] T ac2ab12f5040488aa306cb69b107b485 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:30.277060 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fe53a8bdbd13417f9ce195c54a52a93c" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:30.544260 1175 consensus_queue.cc:260] T ac2ab12f5040488aa306cb69b107b485 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:30.544446 1062 raft_consensus.cc:3060] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:30.544695 891 tablet_service.cc:1505] Processing CreateTablet for tablet 95b95947aad94b849303eb344d18fd27 (DEFAULT_TABLE table=default.Table_5201d08d823b4303b0be4f42a443dd2d [id=128a2dfa4c03443fb7de8077b910376a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.544785 1175 ts_tablet_manager.cc:1434] T ac2ab12f5040488aa306cb69b107b485 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:30.433252 1174 tablet_bootstrap.cc:492] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:30.544992 1174 ts_tablet_manager.cc:1403] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.648s user 0.026s sys 0.023s
I20260202 02:52:30.545120 1175 tablet_bootstrap.cc:492] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:30.545708 1174 raft_consensus.cc:359] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:30.545871 1174 raft_consensus.cc:385] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:30.545926 1174 raft_consensus.cc:740] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:30.546120 1174 consensus_queue.cc:260] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:30.546571 1174 ts_tablet_manager.cc:1434] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:30.546742 885 tablet_service.cc:1505] Processing CreateTablet for tablet 104b2c537c4f4e3da6711364cb5d2a2e (DEFAULT_TABLE table=default.Table_6da2058f0c8c44feb77c1106bda2bffe [id=2ae99f38f2464e099ac578796258f825]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.546835 1174 tablet_bootstrap.cc:492] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:30.546804 887 tablet_service.cc:1505] Processing CreateTablet for tablet 532681264d6b4ac8940f5bc79eab3d1c (DEFAULT_TABLE table=default.Table_abc8ed4a399d4fdb9479666c6368f587 [id=732d42755bc84f72ae7e0960e2ae2749]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.547068 1062 raft_consensus.cc:2468] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:30.547124 884 tablet_service.cc:1505] Processing CreateTablet for tablet 6466c63eb97a4a21b29f4f0871e8973c (DEFAULT_TABLE table=default.Table_c57661c84d424228a977c159e893db51 [id=9e3c1e6d5d7b44caa219cb744275d833]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.547290 902 tablet_service.cc:1505] Processing CreateTablet for tablet c4649f7e97bb44f2b7dca47b26379791 (DEFAULT_TABLE table=default.Table_eea651758bad47dfa286f8dcea6787b6 [id=8ea252bb1b6148458be999c715bb331e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.547578 901 tablet_service.cc:1505] Processing CreateTablet for tablet e04960479c92479b8bb864544dd0c8ac (DEFAULT_TABLE table=default.Table_f21cdfd093884683ab4ec1af5345e9d5 [id=ddab3068794747a89a4950dc1ba79b5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.547883 900 tablet_service.cc:1505] Processing CreateTablet for tablet c707180fc9c5457292a8f43bf676f4f9 (DEFAULT_TABLE table=default.Table_ae634747dc6e4a789c36df9b29d861c0 [id=150c028675db47809195ba67ba2ba008]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.277184 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26b890ed87c3471b9fc0b172c90e0a11" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:30.332397 528 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet d8418a9dd0944456b4d4820dde0cde02
I20260202 02:52:30.548297 1059 raft_consensus.cc:2393] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f442ef85029461091de847118c34c0d in current term 1: Already voted for candidate 1521a6d262b7451bbef8debe970dd6b5 in this term.
I20260202 02:52:30.548533 742 tablet_service.cc:1552] Processing DeleteTablet for tablet 75569511179b41038e521fa3346a92ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:30.548837 528 catalog_manager.cc:6724] ReplaceTablet: tablet d8418a9dd0944456b4d4820dde0cde02 deleted and replaced by tablet 1b2b4f429b6a484ea6ef7c19124b236b
I20260202 02:52:30.548884 742 tablet_service.cc:1505] Processing CreateTablet for tablet 48c820d845e144249c94cd87a0f38ec3 (DEFAULT_TABLE table=default.Table_45aac0057c8c4bae852d155bff2ed712 [id=707ca538c3964adaa2d2b0324c888918]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.282706 2296 tablet.cc:1722] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:30.549238 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48c820d845e144249c94cd87a0f38ec3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:30.549238 528 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_971daa4f561047c5a129d2937ba0cd56" } new_table_name: "default.Table_0e7ceba4956f408c8a9fc0f61fa67c90" modify_external_catalogs: true
I20260202 02:52:30.333511 519 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 91ec01d25f8f441cbe3783cacbee51db
I20260202 02:52:30.550150 519 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_0eedfe3d0e4c4ba08086d0cf45e438ae" } new_table_name: "default.Table_8c1b93c2a1234d058590e574f89b770b" modify_external_catalogs: true
I20260202 02:52:30.320143 1721 raft_consensus.cc:493] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:30.551187 1721 raft_consensus.cc:515] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:30.551805 1721 leader_election.cc:290] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:30.334254 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_a2c0065f3593412fa5958d59d354f437"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:30.552099 752 tablet_service.cc:1505] Processing CreateTablet for tablet 104b2c537c4f4e3da6711364cb5d2a2e (DEFAULT_TABLE table=default.Table_6da2058f0c8c44feb77c1106bda2bffe [id=2ae99f38f2464e099ac578796258f825]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.552441 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 104b2c537c4f4e3da6711364cb5d2a2e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:30.552403 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a2c0065f3593412fa5958d59d354f437 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:30.552636 755 tablet_service.cc:1505] Processing CreateTablet for tablet 95b95947aad94b849303eb344d18fd27 (DEFAULT_TABLE table=default.Table_5201d08d823b4303b0be4f42a443dd2d [id=128a2dfa4c03443fb7de8077b910376a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.552924 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95b95947aad94b849303eb344d18fd27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:30.328361 2127 consensus_queue.cc:1048] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:30.343026 526 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_5ce52f099d05416f87636eaf90c9f958" } modify_external_catalogs: true
I20260202 02:52:30.554102 526 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_5ce52f099d05416f87636eaf90c9f958" } modify_external_catalogs: true
I20260202 02:52:30.554265 2372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8: Bootstrap starting.
I20260202 02:52:30.357062 1110 heartbeater.cc:344] Connected to a master server at 127.0.104.190:33981
I20260202 02:52:30.356173 520 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 16b47a2831f54d809ed0b16deaec7e3d (table default.Table_e4018d1c57714a24be5d3a20c1ccaef4 [id=1e291b35a6764638acfb781bb4e6adb3]). Expected version 2 got 0
I20260202 02:52:30.556660 756 tablet_service.cc:1505] Processing CreateTablet for tablet 532681264d6b4ac8940f5bc79eab3d1c (DEFAULT_TABLE table=default.Table_abc8ed4a399d4fdb9479666c6368f587 [id=732d42755bc84f72ae7e0960e2ae2749]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.556838 750 tablet_service.cc:1505] Processing CreateTablet for tablet c584f3163f1b45789492ccd27b7ad609 (DEFAULT_TABLE table=default.Table_1a44a09f9b4f49a9a5f1f9f2529b5a52 [id=183241be2ed84df9baa79bcc113db1b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.557047 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 532681264d6b4ac8940f5bc79eab3d1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:30.557247 1174 tablet_bootstrap.cc:654] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:30.557266 747 tablet_service.cc:1505] Processing CreateTablet for tablet c4649f7e97bb44f2b7dca47b26379791 (DEFAULT_TABLE table=default.Table_eea651758bad47dfa286f8dcea6787b6 [id=8ea252bb1b6148458be999c715bb331e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.557565 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4649f7e97bb44f2b7dca47b26379791. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:30.558091 520 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_bb45d0e2e2a14038aec12db59746877a" } modify_external_catalogs: true
I20260202 02:52:30.558281 520 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_bb45d0e2e2a14038aec12db59746877a" } modify_external_catalogs: true
I20260202 02:52:30.453444 1043 tablet_service.cc:1505] Processing CreateTablet for tablet efe78242e0654a1c9bdbaf9cfa88b034 (DEFAULT_TABLE table=default.Table_6f5ad3d161b6498ebccd807c6d76430f [id=bad8869dcc094cde92184879b15a76e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.372153 521 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet fbcb4a522df648ba9542250450c08f1b
I20260202 02:52:30.559957 521 catalog_manager.cc:6724] ReplaceTablet: tablet fbcb4a522df648ba9542250450c08f1b deleted and replaced by tablet 16b2d80274a54f1193a3f1746c278252
I20260202 02:52:30.560344 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_f468fb0750cf421e833dea1a00d62640"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:30.560653 749 tablet_service.cc:1505] Processing CreateTablet for tablet e04960479c92479b8bb864544dd0c8ac (DEFAULT_TABLE table=default.Table_f21cdfd093884683ab4ec1af5345e9d5 [id=ddab3068794747a89a4950dc1ba79b5b]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:30.560895 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f468fb0750cf421e833dea1a00d62640 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:30.560907 746 tablet_service.cc:1505] Processing CreateTablet for tablet 2654ace469dd42389a0d64bf27237d75 (DEFAULT_TABLE table=default.Table_da4a1a30b6e34996a02ab80a072959d4 [id=cb32193d435e488589469afb8653d656]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.561024 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e04960479c92479b8bb864544dd0c8ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:30.561172 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2654ace469dd42389a0d64bf27237d75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:30.455662 1035 tablet_service.cc:1505] Processing CreateTablet for tablet d06ebf3fe7fd4a59ad08d6d1ba3d85ec (DEFAULT_TABLE table=default.Table_00bfeb4d04d94815afb263182ee583f9 [id=cf0c93490088439dbf7f877ede0505da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.372947 523 master_service.cc:954] ReplaceTablet: received request to replace tablet 75a24a1bf54d4c799938a5723ac6fa99 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:30.457278 1040 tablet_service.cc:1505] Processing CreateTablet for tablet c8e489f59d1345549e29c4fa6e59c1c9 (DEFAULT_TABLE table=default.Table_3c3eaedb107e485abb2fad9d4d958c62 [id=307abaebfaab40b0bfc00dbfed03cde6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.458837 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 08fbd094d4df46cf82a694b4e1f30c6d (DEFAULT_TABLE table=default.Table_4bfd0a9f04ae4defa2e130a44fe45b2f [id=22f72717c1ce4885ab6891538a78cee4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.373234 522 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet f1fadf2cb5be4ec8980871fc96ee9730
I20260202 02:52:30.460001 1030 tablet_service.cc:1505] Processing CreateTablet for tablet 1dd3b956ae594e89b0599dbc9345f5c3 (DEFAULT_TABLE table=default.Table_aef88b2493064de9b51f8eb3ee276ea2 [id=dd6a1a813ed64da5934a43a5683b9e3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.461834 1031 tablet_service.cc:1505] Processing CreateTablet for tablet bdd72f995a704520a5fb1c24a53ed252 (DEFAULT_TABLE table=default.Table_7b5f2de229cb4b2f8bd174ccfd6b9553 [id=97a0437f66864a7291b966c9b72c0eb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.462805 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 7fed7eb10ec74f009b73fc5c66084851 (DEFAULT_TABLE table=default.Table_e9aa59c2ab3343cdb4bfb8f0605bb417 [id=5f38e75091c34b4fbc04ade774ec8718]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.464052 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 48c820d845e144249c94cd87a0f38ec3 (DEFAULT_TABLE table=default.Table_45aac0057c8c4bae852d155bff2ed712 [id=707ca538c3964adaa2d2b0324c888918]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.468019 1034 tablet_service.cc:1505] Processing CreateTablet for tablet 7ffadda20dd3431b84b2657485e7dec7 (DEFAULT_TABLE table=default.Table_19f091a0564b48e08eca8386c5222b02 [id=d41912d557814c5e9686a591e32b62f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.564127 522 master_service.cc:954] ReplaceTablet: received request to replace tablet b8f2b34b4f1e4ff2b2be4ff40aeb0c66 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:30.564301 745 tablet_service.cc:1505] Processing CreateTablet for tablet c707180fc9c5457292a8f43bf676f4f9 (DEFAULT_TABLE table=default.Table_ae634747dc6e4a789c36df9b29d861c0 [id=150c028675db47809195ba67ba2ba008]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.564646 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c707180fc9c5457292a8f43bf676f4f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:30.564913 740 tablet_service.cc:1505] Processing CreateTablet for tablet d06ebf3fe7fd4a59ad08d6d1ba3d85ec (DEFAULT_TABLE table=default.Table_00bfeb4d04d94815afb263182ee583f9 [id=cf0c93490088439dbf7f877ede0505da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.437934 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_31ed23ec1b894b32953e20125e57d70a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:30.565367 753 tablet_service.cc:1505] Processing CreateTablet for tablet c8e489f59d1345549e29c4fa6e59c1c9 (DEFAULT_TABLE table=default.Table_3c3eaedb107e485abb2fad9d4d958c62 [id=307abaebfaab40b0bfc00dbfed03cde6]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:30.565593 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_31ed23ec1b894b32953e20125e57d70a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:30.565809 748 tablet_service.cc:1505] Processing CreateTablet for tablet 08fbd094d4df46cf82a694b4e1f30c6d (DEFAULT_TABLE table=default.Table_4bfd0a9f04ae4defa2e130a44fe45b2f [id=22f72717c1ce4885ab6891538a78cee4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.556660 744 tablet_service.cc:1505] Processing CreateTablet for tablet 6466c63eb97a4a21b29f4f0871e8973c (DEFAULT_TABLE table=default.Table_c57661c84d424228a977c159e893db51 [id=9e3c1e6d5d7b44caa219cb744275d833]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.564587 743 tablet_service.cc:1505] Processing CreateTablet for tablet efe78242e0654a1c9bdbaf9cfa88b034 (DEFAULT_TABLE table=default.Table_6f5ad3d161b6498ebccd807c6d76430f [id=bad8869dcc094cde92184879b15a76e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.560680 754 tablet_service.cc:1505] Processing CreateTablet for tablet ae08e323fb9a4011ac313faf4ca70090 (DEFAULT_TABLE table=default.Table_987beb2b868c44dd90fb8669aab44ca7 [id=787f9ee3032c4116b30df1c23382e358]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.568581 757 tablet_service.cc:1505] Processing CreateTablet for tablet 91a604fb877a48479914a5b60144a22d (DEFAULT_TABLE table=default.Table_7ff322b57db648b3b480519c357a083b [id=e5b1821be6a74d9da2841d2c01c96e03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.569829 1173 tablet_bootstrap.cc:654] T 6b8f924aecf44e959e47d2f32e6b0bf5 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:30.569985 751 tablet_service.cc:1505] Processing CreateTablet for tablet 1dd3b956ae594e89b0599dbc9345f5c3 (DEFAULT_TABLE table=default.Table_aef88b2493064de9b51f8eb3ee276ea2 [id=dd6a1a813ed64da5934a43a5683b9e3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.570470 758 tablet_service.cc:1505] Processing CreateTablet for tablet bdd72f995a704520a5fb1c24a53ed252 (DEFAULT_TABLE table=default.Table_7b5f2de229cb4b2f8bd174ccfd6b9553 [id=97a0437f66864a7291b966c9b72c0eb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.570541 1174 tablet_bootstrap.cc:492] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:30.570659 1174 ts_tablet_manager.cc:1403] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.024s user 0.004s sys 0.004s
I20260202 02:52:30.570901 741 tablet_service.cc:1505] Processing CreateTablet for tablet 7fed7eb10ec74f009b73fc5c66084851 (DEFAULT_TABLE table=default.Table_e9aa59c2ab3343cdb4bfb8f0605bb417 [id=5f38e75091c34b4fbc04ade774ec8718]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.571125 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6466c63eb97a4a21b29f4f0871e8973c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:30.571414 1174 raft_consensus.cc:359] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:30.571571 1174 raft_consensus.cc:385] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:30.571619 1174 raft_consensus.cc:740] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:30.571722 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "026e6ee0b3ae4d938fdf1550044ed1fa" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:30.571779 1174 consensus_queue.cc:260] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:30.571892 776 raft_consensus.cc:2468] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:30.572211 1174 ts_tablet_manager.cc:1434] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260202 02:52:30.572496 1174 tablet_bootstrap.cc:492] T 33150e1456144b8e9c64cc2a2ac6dcca P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:30.552423 759 tablet_service.cc:1552] Processing DeleteTablet for tablet fe9ea4c0502f4976b440465992f2ea2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:30.573536 759 tablet_service.cc:1505] Processing CreateTablet for tablet 7ffadda20dd3431b84b2657485e7dec7 (DEFAULT_TABLE table=default.Table_19f091a0564b48e08eca8386c5222b02 [id=d41912d557814c5e9686a591e32b62f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.574355 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91a604fb877a48479914a5b60144a22d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:30.576551 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2654ace469dd42389a0d64bf27237d75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:30.578058 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4649f7e97bb44f2b7dca47b26379791. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:30.548233 899 tablet_service.cc:1505] Processing CreateTablet for tablet 91a604fb877a48479914a5b60144a22d (DEFAULT_TABLE table=default.Table_7ff322b57db648b3b480519c357a083b [id=e5b1821be6a74d9da2841d2c01c96e03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.434176 2236 consensus_queue.cc:1048] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:30.580595 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e04960479c92479b8bb864544dd0c8ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:30.548555 897 tablet_service.cc:1505] Processing CreateTablet for tablet ae08e323fb9a4011ac313faf4ca70090 (DEFAULT_TABLE table=default.Table_987beb2b868c44dd90fb8669aab44ca7 [id=787f9ee3032c4116b30df1c23382e358]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.356357 969 heartbeater.cc:344] Connected to a master server at 127.0.104.190:33981
I20260202 02:52:30.550638 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 532681264d6b4ac8940f5bc79eab3d1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:30.536695 2297 raft_consensus.cc:515] T d0ce2121d65b4a228596e0ca9018883d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:30.581562 2297 leader_election.cc:290] T d0ce2121d65b4a228596e0ca9018883d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:30.547124 896 tablet_service.cc:1505] Processing CreateTablet for tablet c584f3163f1b45789492ccd27b7ad609 (DEFAULT_TABLE table=default.Table_1a44a09f9b4f49a9a5f1f9f2529b5a52 [id=183241be2ed84df9baa79bcc113db1b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.582407 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91a604fb877a48479914a5b60144a22d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:30.586500 1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c707180fc9c5457292a8f43bf676f4f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:30.591843 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "44c8d5b74c8e4c499f7f68e8185efc17" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:30.592487 859 leader_election.cc:343] T 1f379a29e2454f689d68f5951837512b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:31.616303 2372 log.cc:826] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8: Log is configured to *not* fsync() on all Append() calls
I20260202 02:52:31.138350 2419 raft_consensus.cc:493] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92a8191cf1e14e4d99f1c378c2d3c85b)
I20260202 02:52:35.610800 2419 raft_consensus.cc:515] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } }
I20260202 02:52:35.613214 2419 leader_election.cc:290] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7dab2abef14445d688ce1335b1f9b2f8 (127.0.104.190:33981), 92a8191cf1e14e4d99f1c378c2d3c85b (127.0.104.189:46271)
W20260202 02:52:35.613214 1414 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.104.129:33337 (request call id 136) took 4349 ms (client timeout 1000 ms). Trace:
I20260202 02:52:30.594998 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 1f54c1dfe0a942b3bafd85522953a032 (table default.Table_a22f252ee8d44be5a9d7202afb726cfc [id=891d9355855640a28939b84b2dd30537]) successfully deleted
W20260202 02:52:35.613368 1413 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.104.130:38257 (request call id 126) took 2711 ms (client timeout 1000 ms). Trace:
W20260202 02:52:35.613543 513 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 5.01808e+06us
W20260202 02:52:31.003823 512 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 6a3dac5cd1ce47488a50b4c32eca67c7 (table default.Table_25f0023dd22a4bb0a7c2863da66d0002 [id=e24ffad12f514d60907b6cadfd09dbe6], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 6a3dac5cd1ce47488a50b4c32eca67c7 consensus state has no leader
W20260202 02:52:35.614161 512 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet c2cc2250701d46a7abe7be03b11d45c8 (table default.Table_0c83932cab13463f9cca28129f18b576 [id=7c76848b98614fbf8b2950e5d55cc609], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet c2cc2250701d46a7abe7be03b11d45c8 consensus state has no leader
W20260202 02:52:35.613453 1414 rpcz_store.cc:269] 0202 02:52:31.264109 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:31.264177 (+ 68us) service_pool.cc:225] Handling call
0202 02:52:35.612084 (+4347907us) spinlock_profiling.cc:243] Waited 4.35 s on lock 0x616000031354. stack: 00007f591057aa86 00007f591057a840 00007f590f5553ad 0000000000984f94 0000000000984f5d 00007f591f8a46bd 00007f591f8a47b3 00007f591c46b521 00007f59241366fd 00007f59242bd5d8 00007f591adbeca6 00007f591adbe9d5 00007f591753d55e 00007f591753c43e 00007f5917542675 00007f5917542f9f
0202 02:52:35.613146 (+ 1062us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":9997523200}
W20260202 02:52:35.613499 1413 rpcz_store.cc:269] 0202 02:52:32.902072 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:32.902150 (+ 78us) service_pool.cc:225] Handling call
0202 02:52:35.611695 (+2709545us) spinlock_profiling.cc:243] Waited 2.71 s on lock 0x616000031354. stack: 00007f591057aa86 00007f591057a840 00007f590f5553ad 0000000000984f94 0000000000984f5d 00007f591f8a46bd 00007f591f8a47b3 00007f591c46b521 00007f59241366fd 00007f59242bd5d8 00007f591adbeca6 00007f591adbe9d5 00007f591753d55e 00007f591753c43e 00007f5917542675 00007f5917542f9f
0202 02:52:35.613286 (+ 1591us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":6229780224}
W20260202 02:52:35.616987 1422 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.104.130:38257 (request call id 125) took 4218 ms (client timeout 1000 ms). Trace:
W20260202 02:52:35.617137 1422 rpcz_store.cc:269] 0202 02:52:31.398385 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:31.398498 (+ 113us) service_pool.cc:225] Handling call
0202 02:52:35.615947 (+4217449us) spinlock_profiling.cc:243] Waited 4.22 s on lock 0x616000031354. stack: 00007f591057aa86 00007f591057a840 00007f590f5553ad 0000000000984f94 0000000000984f5d 00007f591f8a46bd 00007f591f8a47b3 00007f591c46b521 00007f59241366fd 00007f59242bd5d8 00007f591adbeca6 00007f591adbe9d5 00007f591753d55e 00007f591753c43e 00007f5917542675 00007f5917542f9f
0202 02:52:35.616939 (+ 992us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":9697709952}
I20260202 02:52:30.592624 859 leader_election.cc:304] T 1f379a29e2454f689d68f5951837512b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:30.592563 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae08e323fb9a4011ac313faf4ca70090. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:30.593248 715 consensus_peers.cc:597] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38301e13e17c41fcae912c4b38c26a45. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:35.757207 715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 5.16349e+06us
W20260202 02:52:31.872982 501 kernel_stack_watchdog.cc:198] Thread 513 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 246ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc02756a980 (unknown) at ??:0
@ 0x7fc02007d255 __write at ??:0
@ 0x7fc01fff828d _IO_file_write at ??:0
@ 0x7fc01fff8bff _IO_file_xsputn at ??:0
@ 0x7fc01ffeca57 fwrite at ??:0
@ 0x8a8c97 __interceptor_fwrite.part.56 at ??:?
@ 0x7fc0211e18a0 google::LogMessage::SendToLog() at ??:0
@ 0x7fc0211dd7cc google::LogMessage::Flush() at ??:0
@ 0x7fc0211dda11 google::LogMessage::~LogMessage() at ??:0
@ 0x7fc037b60248 kudu::master::AsyncDeleteReplica::HandleResponse() at ??:0
@ 0x7fc037ab7b2d kudu::master::RetryingTSRpcTask::RpcCallback() at ??:0
@ 0x7fc037b63840 _ZZN4kudu6master18AsyncDeleteReplica11SendRequestEiENKUlvE_clEv at ??:0
@ 0x7fc037b6365e _ZNSt17_Function_handlerIFvvEZN4kudu6master18AsyncDeleteReplica11SendRequestEiEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xa8970d std::function<>::operator()() at ??:?
@ 0x7fc02ae5f2df kudu::rpc::Proxy::AsyncRequest()::$_1::operator()() at ??:0
@ 0x7fc02ae5eebe std::_Function_handler<>::_M_invoke() at ??:0
W20260202 02:52:32.299974 515 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 41a4143247ed4eed98defbdfe2895372 (table default.Table_3db4721b7f3d4b09aafe0f29ebb2039f [id=9ae42d61d911493caaa622f688e6c9c3], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 41a4143247ed4eed98defbdfe2895372 consensus state has no leader
I20260202 02:52:30.573992 893 tablet_service.cc:1505] Processing CreateTablet for tablet 2654ace469dd42389a0d64bf27237d75 (DEFAULT_TABLE table=default.Table_da4a1a30b6e34996a02ab80a072959d4 [id=cb32193d435e488589469afb8653d656]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.607550 2127 consensus_queue.cc:1048] T 3d1a7578e83447669e3c3972a426ad44 P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:30.602635 2377 raft_consensus.cc:493] T fe9ea4c0502f4976b440465992f2ea2c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:35.758533 2377 raft_consensus.cc:515] T fe9ea4c0502f4976b440465992f2ea2c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
W20260202 02:52:33.377024 514 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet d594295561ff4b01bf13451533a5743c (table default.Table_0c357b5eb1e14dbba252f4edfc630705 [id=64eea8b3e2d34698ad2dbe65d87d29f5], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet d594295561ff4b01bf13451533a5743c consensus state has no leader
I20260202 02:52:35.759166 2377 leader_election.cc:290] T fe9ea4c0502f4976b440465992f2ea2c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:35.659291 538 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 861 } } ignore_live_leader: false dest_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" is_pre_election: true
I20260202 02:52:30.593843 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4649f7e97bb44f2b7dca47b26379791. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:30.537886 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 672) took 1273 ms. Trace:
I20260202 02:52:35.759835 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6466c63eb97a4a21b29f4f0871e8973c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:35.759672 1182 rpcz_store.cc:276] 0202 02:52:29.264863 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.496060 (+231197us) service_pool.cc:225] Handling call
0202 02:52:30.260137 (+764077us) ts_tablet_manager.cc:1151] Deleting tablet ccd6bc8bd0504bf18d108c73685ef9ff
0202 02:52:30.260143 (+ 6us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:30.298124 (+ 37981us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:30.301737 (+ 3613us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:30.537868 (+236131us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":763746}
I20260202 02:52:35.760111 1182 tablet_replica.cc:333] T 474df41e4fe24a818fa45a4ce84d5474 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:30.606123 886 tablet_service.cc:1505] Processing CreateTablet for tablet d06ebf3fe7fd4a59ad08d6d1ba3d85ec (DEFAULT_TABLE table=default.Table_00bfeb4d04d94815afb263182ee583f9 [id=cf0c93490088439dbf7f877ede0505da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:35.760637 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d06ebf3fe7fd4a59ad08d6d1ba3d85ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:35.760636 494 master_runner.cc:186] Error getting master registration for 127.0.104.190:33981: OK, instance_id { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" instance_seqno: 1770000750063992 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260202 02:52:35.761628 494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:52:35.761850 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c707180fc9c5457292a8f43bf676f4f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:30.655155 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d0ce2121d65b4a228596e0ca9018883d" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:30.606381 894 tablet_service.cc:1505] Processing CreateTablet for tablet 1dd3b956ae594e89b0599dbc9345f5c3 (DEFAULT_TABLE table=default.Table_aef88b2493064de9b51f8eb3ee276ea2 [id=dd6a1a813ed64da5934a43a5683b9e3b]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:35.763679 1062 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.130:37027 (request call id 259) took 5171 ms (client timeout 3015 ms). Trace:
W20260202 02:52:35.763775 1062 rpcz_store.cc:269] 0202 02:52:30.591689 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.654523 (+ 62834us) service_pool.cc:225] Handling call
0202 02:52:35.763653 (+5109130us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:35.763955 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1dd3b956ae594e89b0599dbc9345f5c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:30.605178 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d0ce2121d65b4a228596e0ca9018883d" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:30.611078 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "44c8d5b74c8e4c499f7f68e8185efc17" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:35.764158 777 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.130:41545 (request call id 254) took 5159 ms (client timeout 3015 ms). Trace:
I20260202 02:52:35.673401 527 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:47430 (request call id 806) took 5429 ms. Trace:
W20260202 02:52:35.764251 776 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.130:41545 (request call id 253) took 5163 ms (client timeout 3206 ms). Trace:
W20260202 02:52:35.764389 776 rpcz_store.cc:269] 0202 02:52:30.600979 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.610962 (+ 9983us) service_pool.cc:225] Handling call
0202 02:52:35.764228 (+5153266us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:35.764431 527 rpcz_store.cc:276] 0202 02:52:30.244128 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.541050 (+296922us) service_pool.cc:225] Handling call
0202 02:52:30.541446 (+ 396us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0202 02:52:30.541658 (+ 212us) catalog_manager.cc:3708] Apply alter schema
0202 02:52:30.541841 (+ 183us) catalog_manager.cc:3740] Acquired catalog manager lock
0202 02:52:30.541942 (+ 101us) catalog_manager.cc:3898] Updating metadata on disk
0202 02:52:35.672790 (+5130848us) catalog_manager.cc:3929] Committing alterations to in-memory state
0202 02:52:35.673356 (+ 566us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:30.547356 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:30.547507 (+ 151us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:30.547515 (+ 8us) write_op.cc:435] Acquired schema lock
0202 02:52:30.547517 (+ 2us) tablet.cc:660] Decoding operations
0202 02:52:30.547637 (+ 120us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:30.547652 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:30.547657 (+ 5us) tablet.cc:683] Acquiring locks for 1 operations
0202 02:52:30.547732 (+ 75us) tablet.cc:699] Row locks acquired
0202 02:52:30.547734 (+ 2us) write_op.cc:260] PREPARE: finished
0202 02:52:30.547777 (+ 43us) write_op.cc:270] Start()
0202 02:52:30.547825 (+ 48us) write_op.cc:276] Timestamp: P: 1770000750547729 usec, L: 0
0202 02:52:30.547829 (+ 4us) op_driver.cc:348] REPLICATION: starting
0202 02:52:30.548052 (+ 223us) log.cc:844] Serialized 405 byte log entry
0202 02:52:35.663470 (+5115418us) op_driver.cc:464] REPLICATION: finished
0202 02:52:35.668773 (+ 5303us) write_op.cc:301] APPLY: starting
0202 02:52:35.668796 (+ 23us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:35.668848 (+ 52us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:35.668892 (+ 44us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:35.668975 (+ 83us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:35.669014 (+ 39us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0202 02:52:35.669021 (+ 7us) write_op.cc:312] APPLY: finished
0202 02:52:35.669144 (+ 123us) log.cc:844] Serialized 23 byte log entry
0202 02:52:35.672661 (+ 3517us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:35.672687 (+ 26us) write_op.cc:454] Released schema lock
0202 02:52:35.672696 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0202 02:52:35.673613 (+ 917us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:35.673620 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":5081,"apply.run_cpu_time_us":826,"apply.run_wall_time_us":4882,"num_ops":1,"prepare.queue_time_us":4926,"prepare.run_cpu_time_us":809,"prepare.run_wall_time_us":810,"raft.queue_time_us":760,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":226,"replication_time_us":5115575,"thread_start_us":9184,"threads_started":2,"wal-append.queue_time_us":1142}]]}
W20260202 02:52:35.765379 527 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.104.130:36691 (request call id 81) took 5166 ms (client timeout 1000 ms). Trace:
W20260202 02:52:35.765484 527 rpcz_store.cc:269] 0202 02:52:30.599152 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:35.765150 (+5165998us) service_pool.cc:212] Skipping call since client already timed out
0202 02:52:35.765349 (+ 199us) inbound_call.cc:173] Queueing failure response
Metrics: {}
W20260202 02:52:35.765820 527 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.104.129:48435 (request call id 71) took 5166 ms (client timeout 1000 ms). Trace:
W20260202 02:52:35.765918 527 rpcz_store.cc:269] 0202 02:52:30.599278 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:35.765756 (+5166478us) service_pool.cc:212] Skipping call since client already timed out
0202 02:52:35.765786 (+ 30us) inbound_call.cc:173] Queueing failure response
Metrics: {}
W20260202 02:52:35.766110 527 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.104.131:43527 (request call id 110) took 5098 ms (client timeout 1000 ms). Trace:
W20260202 02:52:35.766191 527 rpcz_store.cc:269] 0202 02:52:30.668234 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:35.766050 (+5097816us) service_pool.cc:212] Skipping call since client already timed out
0202 02:52:35.766090 (+ 40us) inbound_call.cc:173] Queueing failure response
Metrics: {}
W20260202 02:52:35.766359 527 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.104.130:36691 (request call id 82) took 4171 ms (client timeout 1000 ms). Trace:
W20260202 02:52:35.766435 527 rpcz_store.cc:269] 0202 02:52:31.594974 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:35.766315 (+4171341us) service_pool.cc:212] Skipping call since client already timed out
0202 02:52:35.766336 (+ 21us) inbound_call.cc:173] Queueing failure response
Metrics: {}
I20260202 02:52:30.666421 1721 raft_consensus.cc:493] T 79c73f4a92044bafb5b8384229cb3a4d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:30.684311 1032 tablet_service.cc:1505] Processing CreateTablet for tablet 39eaa2848cf4450dbe563ddd3b11df06 (DEFAULT_TABLE table=default.Table_a0621e0a3e8047fda1e5226ee396d4e8 [id=c6dfbf7e351a4f2a95d586c14399f15e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:35.767537 1721 raft_consensus.cc:515] T 79c73f4a92044bafb5b8384229cb3a4d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:30.686362 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 0da286b7f598457ca4d0254a7073cb74 (DEFAULT_TABLE table=default.Table_9412ac9f6f6c495f917ebcf68650e5f1 [id=d95e7272ae94469f92d4a482ea2c82a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.700385 1036 tablet_service.cc:1505] Processing CreateTablet for tablet 16b2d80274a54f1193a3f1746c278252 (DEFAULT_TABLE table=default.Table_0cd8f9947fbd430f8992cc1543aed9f7 [id=163eefc3d38a4b28bce88b948cc5df3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:35.767390 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae08e323fb9a4011ac313faf4ca70090. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:30.701854 1033 tablet_service.cc:1505] Processing CreateTablet for tablet 4f2627dcb9904589967407c3a8756a33 (DEFAULT_TABLE table=default.Table_5c8329977b724148a1a8d5c504a4a513 [id=893bd49b291d4748872a291b7afc6f77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.705407 1039 tablet_service.cc:1505] Processing CreateTablet for tablet 1b2b4f429b6a484ea6ef7c19124b236b (DEFAULT_TABLE table=default.Table_2d42dcf81d2d4b2c90fe09860b9b2318 [id=e620930eae8e4707b93e7ab16369e13d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:35.768115 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e04960479c92479b8bb864544dd0c8ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:35.768169 1721 leader_election.cc:290] T 79c73f4a92044bafb5b8384229cb3a4d P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:30.688807 1028 tablet_service.cc:1505] Processing CreateTablet for tablet fe97009dbcc64846844e381817148d7b (DEFAULT_TABLE table=default.Table_df42f88a6807472088543ed874ac5e76 [id=b2b4d069db034e46b4d9f3fa935b363e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.712159 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 91a604fb877a48479914a5b60144a22d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:30 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:30.606398 890 tablet_service.cc:1505] Processing CreateTablet for tablet c8e489f59d1345549e29c4fa6e59c1c9 (DEFAULT_TABLE table=default.Table_3c3eaedb107e485abb2fad9d4d958c62 [id=307abaebfaab40b0bfc00dbfed03cde6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.596946 998 leader_election.cc:304] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:30.865345 1173 tablet_bootstrap.cc:492] T 6b8f924aecf44e959e47d2f32e6b0bf5 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:30.621516 756 tablet_service.cc:1505] Processing CreateTablet for tablet 0da286b7f598457ca4d0254a7073cb74 (DEFAULT_TABLE table=default.Table_9412ac9f6f6c495f917ebcf68650e5f1 [id=d95e7272ae94469f92d4a482ea2c82a9]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:35.768473 998 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 5.17107e+06us
I20260202 02:52:35.768536 1173 ts_tablet_manager.cc:1403] T 6b8f924aecf44e959e47d2f32e6b0bf5 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 5.500s user 0.005s sys 0.012s
I20260202 02:52:30.606564 889 tablet_service.cc:1505] Processing CreateTablet for tablet 08fbd094d4df46cf82a694b4e1f30c6d (DEFAULT_TABLE table=default.Table_4bfd0a9f04ae4defa2e130a44fe45b2f [id=22f72717c1ce4885ab6891538a78cee4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.629252 747 tablet_service.cc:1505] Processing CreateTablet for tablet fe97009dbcc64846844e381817148d7b (DEFAULT_TABLE table=default.Table_df42f88a6807472088543ed874ac5e76 [id=b2b4d069db034e46b4d9f3fa935b363e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.606878 888 tablet_service.cc:1505] Processing CreateTablet for tablet bdd72f995a704520a5fb1c24a53ed252 (DEFAULT_TABLE table=default.Table_7b5f2de229cb4b2f8bd174ccfd6b9553 [id=97a0437f66864a7291b966c9b72c0eb7]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:35.769040 999 leader_election.cc:343] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 026e6ee0b3ae4d938fdf1550044ed1fa
I20260202 02:52:30.629503 2157 raft_consensus.cc:493] T 02b434fa058b45529ff3938b053263c6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:35.769270 1173 raft_consensus.cc:359] T 6b8f924aecf44e959e47d2f32e6b0bf5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:35.769377 999 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 5.09016e+06us
I20260202 02:52:35.769419 1173 raft_consensus.cc:385] T 6b8f924aecf44e959e47d2f32e6b0bf5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:35.769462 1173 raft_consensus.cc:740] T 6b8f924aecf44e959e47d2f32e6b0bf5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:35.769409 2157 raft_consensus.cc:515] T 02b434fa058b45529ff3938b053263c6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:35.769618 1173 consensus_queue.cc:260] T 6b8f924aecf44e959e47d2f32e6b0bf5 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:31.374557 986 kernel_stack_watchdog.cc:198] Thread 998 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 212ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fec56842980 (unknown) at ??:0
@ 0x7fec5683c9c6 __GI___pthread_rwlock_wrlock at ??:0
@ 0x7fec504b5904 google::LogMessage::Flush() at ??:0
@ 0x7fec504b5a11 google::LogMessage::~LogMessage() at ??:0
@ 0x7fec5f04230b kudu::consensus::LeaderElection::CheckForDecision() at ??:0
@ 0x7fec5f0436b2 kudu::consensus::LeaderElection::VoteResponseRpcCallback() at ??:0
@ 0x7fec5f04607a kudu::consensus::LeaderElection::Run()::$_0::operator()() at ??:0
@ 0x7fec5f045e3e std::_Function_handler<>::_M_invoke() at ??:0
@ 0xa8970d std::function<>::operator()() at ??:?
@ 0x7fec5a1372df kudu::rpc::Proxy::AsyncRequest()::$_1::operator()() at ??:0
@ 0x7fec5a136ebe std::_Function_handler<>::_M_invoke() at ??:0
@ 0xa8970d std::function<>::operator()() at ??:?
@ 0x7fec5a122f10 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7fec5a12345c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7fec5a0be81a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7fec5a0bd1d6 kudu::rpc::Connection::ReadHandler() at ??:0
I20260202 02:52:35.769977 2157 leader_election.cc:290] T 02b434fa058b45529ff3938b053263c6 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:30.607894 887 tablet_service.cc:1505] Processing CreateTablet for tablet efe78242e0654a1c9bdbaf9cfa88b034 (DEFAULT_TABLE table=default.Table_6f5ad3d161b6498ebccd807c6d76430f [id=bad8869dcc094cde92184879b15a76e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.625440 903 tablet_service.cc:1505] Processing CreateTablet for tablet 7fed7eb10ec74f009b73fc5c66084851 (DEFAULT_TABLE table=default.Table_e9aa59c2ab3343cdb4bfb8f0605bb417 [id=5f38e75091c34b4fbc04ade774ec8718]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:32.512163 2421 raft_consensus.cc:493] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
I20260202 02:52:30.631846 892 tablet_service.cc:1505] Processing CreateTablet for tablet 48c820d845e144249c94cd87a0f38ec3 (DEFAULT_TABLE table=default.Table_45aac0057c8c4bae852d155bff2ed712 [id=707ca538c3964adaa2d2b0324c888918]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.632526 895 tablet_service.cc:1505] Processing CreateTablet for tablet 7ffadda20dd3431b84b2657485e7dec7 (DEFAULT_TABLE table=default.Table_19f091a0564b48e08eca8386c5222b02 [id=d41912d557814c5e9686a591e32b62f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:35.770187 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet d8418a9dd0944456b4d4820dde0cde02 with delete_type TABLET_DATA_DELETED (replaced by tablet 1b2b4f429b6a484ea6ef7c19124b236b) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:35.770232 2421 raft_consensus.cc:515] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:35.770330 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 91ec01d25f8f441cbe3783cacbee51db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:30 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:35.770437 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 026e6ee0b3ae4d938fdf1550044ed1fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:30.648874 746 tablet_service.cc:1505] Processing CreateTablet for tablet 4f2627dcb9904589967407c3a8756a33 (DEFAULT_TABLE table=default.Table_5c8329977b724148a1a8d5c504a4a513 [id=893bd49b291d4748872a291b7afc6f77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:35.770541 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet dada209d982d48f3b74a49905f19c5b6 with delete_type TABLET_DATA_DELETED (replaced by tablet 67a8c9a1998b47b3b214a06ce62381e9) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:35.770659 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 874ca4ba4ebc4721bb4b60cba8e01841 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:35.770749 2421 leader_election.cc:290] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:35.770776 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 33150e1456144b8e9c64cc2a2ac6dcca with delete_type TABLET_DATA_DELETED (replaced by tablet 1d98c7a46139413b8f0d3697b74b1e52) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:35.770867 2421 raft_consensus.cc:2804] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:35.770886 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet fbcb4a522df648ba9542250450c08f1b with delete_type TABLET_DATA_DELETED (replaced by tablet 16b2d80274a54f1193a3f1746c278252) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:35.770932 2421 raft_consensus.cc:493] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:35.770992 2421 raft_consensus.cc:3060] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:30.632858 898 tablet_service.cc:1505] Processing CreateTablet for tablet 0da286b7f598457ca4d0254a7073cb74 (DEFAULT_TABLE table=default.Table_9412ac9f6f6c495f917ebcf68650e5f1 [id=d95e7272ae94469f92d4a482ea2c82a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.572940 1175 tablet_bootstrap.cc:654] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:30.652314 742 tablet_service.cc:1505] Processing CreateTablet for tablet 39eaa2848cf4450dbe563ddd3b11df06 (DEFAULT_TABLE table=default.Table_a0621e0a3e8047fda1e5226ee396d4e8 [id=c6dfbf7e351a4f2a95d586c14399f15e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.648000 755 tablet_service.cc:1505] Processing CreateTablet for tablet 16b2d80274a54f1193a3f1746c278252 (DEFAULT_TABLE table=default.Table_0cd8f9947fbd430f8992cc1543aed9f7 [id=163eefc3d38a4b28bce88b948cc5df3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.669536 921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26b890ed87c3471b9fc0b172c90e0a11" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
W20260202 02:52:35.772251 921 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.129:50505 (request call id 302) took 5105 ms (client timeout 3422 ms). Trace:
W20260202 02:52:35.772364 921 rpcz_store.cc:269] 0202 02:52:30.666945 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.669418 (+ 2473us) service_pool.cc:225] Handling call
0202 02:52:35.772226 (+5102808us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:35.772485 1721 raft_consensus.cc:493] T dada209d982d48f3b74a49905f19c5b6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:30.682348 749 tablet_service.cc:1505] Processing CreateTablet for tablet 1b2b4f429b6a484ea6ef7c19124b236b (DEFAULT_TABLE table=default.Table_2d42dcf81d2d4b2c90fe09860b9b2318 [id=e620930eae8e4707b93e7ab16369e13d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:35.772590 1721 raft_consensus.cc:515] T dada209d982d48f3b74a49905f19c5b6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:30.678018 899 tablet_service.cc:1505] Processing CreateTablet for tablet 39eaa2848cf4450dbe563ddd3b11df06 (DEFAULT_TABLE table=default.Table_a0621e0a3e8047fda1e5226ee396d4e8 [id=c6dfbf7e351a4f2a95d586c14399f15e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:30.748456 2416 tablet.cc:1722] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:30.755584 2374 raft_consensus.cc:493] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:30.683180 752 tablet_service.cc:1552] Processing DeleteTablet for tablet 91a604fb877a48479914a5b60144a22d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:30 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:30.694967 2150 raft_consensus.cc:493] T 75569511179b41038e521fa3346a92ae P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:35.773063 1721 leader_election.cc:290] T dada209d982d48f3b74a49905f19c5b6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:35.773052 2374 raft_consensus.cc:515] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:35.773144 2150 raft_consensus.cc:515] T 75569511179b41038e521fa3346a92ae P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:35.773412 1721 raft_consensus.cc:493] T 33150e1456144b8e9c64cc2a2ac6dcca P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:35.773505 1721 raft_consensus.cc:515] T 33150e1456144b8e9c64cc2a2ac6dcca P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:32.807260 2423 raft_consensus.cc:493] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:35.773584 2374 leader_election.cc:290] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:35.773651 2150 leader_election.cc:290] T 75569511179b41038e521fa3346a92ae P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:35.773648 2423 raft_consensus.cc:515] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:35.773764 2374 raft_consensus.cc:2749] T 1f379a29e2454f689d68f5951837512b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:30.776455 2417 tablet.cc:1722] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:35.773953 1721 leader_election.cc:290] T 33150e1456144b8e9c64cc2a2ac6dcca P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:33.123749 2427 raft_consensus.cc:493] T 91ec01d25f8f441cbe3783cacbee51db P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
I20260202 02:52:35.774040 2423 leader_election.cc:290] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:30.695950 714 leader_election.cc:343] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 26eeef6eebbf4f2cb44f69255fb37594
I20260202 02:52:35.774102 2427 raft_consensus.cc:515] T 91ec01d25f8f441cbe3783cacbee51db P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:35.774183 714 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 5.07779e+06us
I20260202 02:52:33.425150 2433 raft_consensus.cc:493] T 70306ed9dec54286885d965f2c012da4 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f442ef85029461091de847118c34c0d)
W20260202 02:52:35.774381 714 leader_election.cc:343] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: fe53a8bdbd13417f9ce195c54a52a93c
I20260202 02:52:35.774453 2433 raft_consensus.cc:515] T 70306ed9dec54286885d965f2c012da4 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:35.774571 714 consensus_peers.cc:597] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38301e13e17c41fcae912c4b38c26a45. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:35.774574 2427 leader_election.cc:290] T 91ec01d25f8f441cbe3783cacbee51db P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:35.770991 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet f1fadf2cb5be4ec8980871fc96ee9730 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:30 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:35.775010 2433 leader_election.cc:290] T 70306ed9dec54286885d965f2c012da4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:31.370249 2297 raft_consensus.cc:493] T f94f25f4059a43b8babedb6b45a08a19 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:35.775187 2297 raft_consensus.cc:515] T f94f25f4059a43b8babedb6b45a08a19 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:35.775728 2297 leader_election.cc:290] T f94f25f4059a43b8babedb6b45a08a19 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:35.775756 752 tablet_service.cc:1552] Processing DeleteTablet for tablet d8418a9dd0944456b4d4820dde0cde02 with delete_type TABLET_DATA_DELETED (replaced by tablet 1b2b4f429b6a484ea6ef7c19124b236b) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:33.668402 2434 raft_consensus.cc:493] T 73f8db8f30f14d369d2227c9be7c4647 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
I20260202 02:52:35.775928 752 tablet_service.cc:1552] Processing DeleteTablet for tablet 91ec01d25f8f441cbe3783cacbee51db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:30 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:35.775905 2434 raft_consensus.cc:515] T 73f8db8f30f14d369d2227c9be7c4647 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:35.776043 752 tablet_service.cc:1552] Processing DeleteTablet for tablet fbcb4a522df648ba9542250450c08f1b with delete_type TABLET_DATA_DELETED (replaced by tablet 16b2d80274a54f1193a3f1746c278252) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:35.776153 752 tablet_service.cc:1552] Processing DeleteTablet for tablet f1fadf2cb5be4ec8980871fc96ee9730 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:30 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:33.540741 2432 raft_consensus.cc:493] T b76d1837813c46e8a0925b49939d8884 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
I20260202 02:52:35.776315 2432 raft_consensus.cc:515] T b76d1837813c46e8a0925b49939d8884 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:35.776450 2434 leader_election.cc:290] T 73f8db8f30f14d369d2227c9be7c4647 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:33.752398 2440 raft_consensus.cc:493] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f442ef85029461091de847118c34c0d)
I20260202 02:52:35.776571 2157 raft_consensus.cc:493] T 25765fb687294e9db1289f76530dd877 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
I20260202 02:52:35.776638 2440 raft_consensus.cc:515] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:35.776676 2157 raft_consensus.cc:515] T 25765fb687294e9db1289f76530dd877 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:31.322044 713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260202 02:52:35.776861 2432 leader_election.cc:290] T b76d1837813c46e8a0925b49939d8884 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:35.777240 2157 leader_election.cc:290] T 25765fb687294e9db1289f76530dd877 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:31.900223 840 kernel_stack_watchdog.cc:198] Thread 859 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 206ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fb3588d9980 (unknown) at ??:0
@ 0x7fb3513ec255 __write at ??:0
@ 0x7fb35136728d _IO_file_write at ??:0
@ 0x7fb351367bff _IO_file_xsputn at ??:0
@ 0x7fb35135ba57 fwrite at ??:0
@ 0x8a8c97 __interceptor_fwrite.part.56 at ??:?
@ 0x7fb3525508a0 google::LogMessage::SendToLog() at ??:0
@ 0x7fb35254c7cc google::LogMessage::Flush() at ??:0
@ 0x7fb35254ca11 google::LogMessage::~LogMessage() at ??:0
@ 0x7fb3610d930b kudu::consensus::LeaderElection::CheckForDecision() at ??:0
@ 0x7fb3610da6b2 kudu::consensus::LeaderElection::VoteResponseRpcCallback() at ??:0
@ 0x7fb3610dd07a kudu::consensus::LeaderElection::Run()::$_0::operator()() at ??:0
@ 0x7fb3610dce3e std::_Function_handler<>::_M_invoke() at ??:0
@ 0xa8970d std::function<>::operator()() at ??:?
@ 0x7fb35c1ce2df kudu::rpc::Proxy::AsyncRequest()::$_1::operator()() at ??:0
@ 0x7fb35c1cdebe std::_Function_handler<>::_M_invoke() at ??:0
I20260202 02:52:31.520975 1180 ts_tablet_manager.cc:1929] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:35.789359 1180 log.cc:1199] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/042d1ebf357b4abe8814a407891ce265
I20260202 02:52:35.789570 2157 raft_consensus.cc:493] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:35.789772 1180 ts_tablet_manager.cc:1950] T 042d1ebf357b4abe8814a407891ce265 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:35.789683 2157 raft_consensus.cc:515] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:35.790259 2157 leader_election.cc:290] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:35.790519 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 470) took 11002 ms. Trace:
I20260202 02:52:35.790594 1180 rpcz_store.cc:276] 0202 02:52:24.788514 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.788749 (+ 235us) service_pool.cc:225] Handling call
0202 02:52:30.539839 (+5751090us) ts_tablet_manager.cc:1151] Deleting tablet 042d1ebf357b4abe8814a407891ce265
0202 02:52:30.539849 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:30.622949 (+ 83100us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:31.520839 (+897890us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:35.790489 (+4269650us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5734110}
I20260202 02:52:35.791038 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 472) took 10954 ms. Trace:
I20260202 02:52:35.791106 1180 rpcz_store.cc:276] 0202 02:52:24.842341 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.842703 (+ 362us) service_pool.cc:225] Handling call
0202 02:52:35.790909 (+10948206us) ts_tablet_manager.cc:1151] Deleting tablet cd1b6662da254889ae4bbe8d0b1ef1bb
0202 02:52:35.790921 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:35.791022 (+ 101us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10939990}
I20260202 02:52:35.791376 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 473) took 10877 ms. Trace:
I20260202 02:52:35.791432 1180 rpcz_store.cc:276] 0202 02:52:24.914437 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:24.914489 (+ 52us) service_pool.cc:225] Handling call
0202 02:52:35.791290 (+10876801us) ts_tablet_manager.cc:1151] Deleting tablet c977c146bac34638874b7a990604a0d5
0202 02:52:35.791297 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:35.791363 (+ 66us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10876435}
I20260202 02:52:35.791692 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 483) took 10691 ms. Trace:
I20260202 02:52:35.791749 1180 rpcz_store.cc:276] 0202 02:52:25.100428 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:25.100476 (+ 48us) service_pool.cc:225] Handling call
0202 02:52:35.791594 (+10691118us) ts_tablet_manager.cc:1151] Deleting tablet a5f864fd8a3d4c4a83e7f68abb8a0a83
0202 02:52:35.791602 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:35.791679 (+ 77us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10690868}
I20260202 02:52:35.792013 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 493) took 10566 ms. Trace:
I20260202 02:52:35.792074 1180 rpcz_store.cc:276] 0202 02:52:25.225459 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:25.225522 (+ 63us) service_pool.cc:225] Handling call
0202 02:52:35.791916 (+10566394us) ts_tablet_manager.cc:1151] Deleting tablet ccd6bc8bd0504bf18d108c73685ef9ff
0202 02:52:35.791923 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:35.791999 (+ 76us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10566144}
I20260202 02:52:32.678547 2420 raft_consensus.cc:493] T a0b5d82e5fb94b60b4aa324b978a5378 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:35.792464 2420 raft_consensus.cc:515] T a0b5d82e5fb94b60b4aa324b978a5378 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:35.773485 2421 raft_consensus.cc:515] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
W20260202 02:52:31.848789 700 kernel_stack_watchdog.cc:198] Thread 715 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 245ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f1e3b197980 (unknown) at ??:0
@ 0x7f1e33caa255 __write at ??:0
@ 0x7f1e33c2528d _IO_file_write at ??:0
@ 0x7f1e33c25bff _IO_file_xsputn at ??:0
@ 0x7f1e33c19a57 fwrite at ??:0
@ 0x8a8c97 __interceptor_fwrite.part.56 at ??:?
@ 0x7f1e34e0e8a0 google::LogMessage::SendToLog() at ??:0
@ 0x7f1e34e0a7cc google::LogMessage::Flush() at ??:0
@ 0x7f1e34e0aa11 google::LogMessage::~LogMessage() at ??:0
@ 0x7f1e439263de kudu::consensus::Peer::ProcessResponseErrorUnlocked() at ??:0
@ 0x7f1e4392549f kudu::consensus::Peer::ProcessResponse() at ??:0
@ 0x7f1e43925bd5 kudu::consensus::Peer::ProcessSingleResponse() at ??:0
@ 0x7f1e4392d761 kudu::consensus::Peer::SendNextRequest()::$_6::operator()() at ??:0
@ 0x7f1e4392d56e std::_Function_handler<>::_M_invoke() at ??:0
@ 0xa8970d std::function<>::operator()() at ??:?
@ 0x7f1e3ea8c2df kudu::rpc::Proxy::AsyncRequest()::$_1::operator()() at ??:0
I20260202 02:52:35.793468 2421 leader_election.cc:290] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:36.447750 2372 log.cc:927] Time spent T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8: Append to log took a long time: real 0.129s user 0.006s sys 0.000s
I20260202 02:52:33.753612 2437 raft_consensus.cc:493] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f442ef85029461091de847118c34c0d)
I20260202 02:52:38.864727 2437 raft_consensus.cc:515] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:36.248123 1389 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.104.190:33981 timed out after 0.630s (SENT)
I20260202 02:52:38.865540 2437 leader_election.cc:290] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:35.785480 715 leader_election.cc:304] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:35.848821 527 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 28a900bcaa8b41718a285e595886d805 (table default.Table_911bb209b4554ee8b9bc09edaf488d63 [id=cd07f9746ec94d9ebe956a7ca0d19b1c]). Expected version 1 got 0
I20260202 02:52:33.787957 2442 raft_consensus.cc:493] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
I20260202 02:52:38.866148 2442 raft_consensus.cc:515] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:38.866266 527 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 16b47a2831f54d809ed0b16deaec7e3d (table default.Table_e4018d1c57714a24be5d3a20c1ccaef4 [id=1e291b35a6764638acfb781bb4e6adb3]). Expected version 2 got 0
I20260202 02:52:38.866485 527 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 44c8d5b74c8e4c499f7f68e8185efc17 (table default.Table_f48ed1c7e02349adad7ce22c0e51ed81 [id=46b5d47377474ff9bac2acdb26b08c65]). Expected version 3 got 0
I20260202 02:52:38.866809 2442 leader_election.cc:290] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:37.072182 1313 kernel_stack_watchdog.cc:198] Thread 1389 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 216ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f5913bb7980 (unknown) at ??:0
@ 0x7f590c6ca255 __write at ??:0
@ 0x7f590c64528d _IO_file_write at ??:0
@ 0x7f590c645bff _IO_file_xsputn at ??:0
@ 0x7f590c639a57 fwrite at ??:0
@ 0x8a8c97 __interceptor_fwrite.part.56 at ??:?
@ 0x7f590d82e8a0 google::LogMessage::SendToLog() at ??:0
@ 0x7f590d82a7cc google::LogMessage::Flush() at ??:0
@ 0x7f590d82aa11 google::LogMessage::~LogMessage() at ??:0
@ 0x7f59174ac604 kudu::rpc::Proxy::AsyncRequest()::$_1::operator()() at ??:0
@ 0x7f59174abebe std::_Function_handler<>::_M_invoke() at ??:0
@ 0xa8970d std::function<>::operator()() at ??:?
@ 0x7f5917497f10 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f5917499602 kudu::rpc::OutboundCall::SetTimedOut() at ??:0
@ 0x7f591742f453 kudu::rpc::Connection::HandleOutboundCallTimeout() at ??:0
@ 0x7f591742efc6 kudu::rpc::Connection::CallAwaitingResponse::HandleTimeout() at ??:0
I20260202 02:52:35.777240 2440 leader_election.cc:290] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:35.793145 2420 leader_election.cc:290] T a0b5d82e5fb94b60b4aa324b978a5378 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:35.807940 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fed7eb10ec74f009b73fc5c66084851. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:33.267416 2426 raft_consensus.cc:493] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b1f05820b6034228bdad884cc348716d)
I20260202 02:52:38.868567 2426 raft_consensus.cc:515] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:38.869239 2426 leader_election.cc:290] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:38.869401 2426 raft_consensus.cc:2804] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:38.869457 2426 raft_consensus.cc:493] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:38.869510 2426 raft_consensus.cc:3060] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:33.492671 2435 raft_consensus.cc:493] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b1f05820b6034228bdad884cc348716d)
I20260202 02:52:35.834930 2127 raft_consensus.cc:493] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
I20260202 02:52:35.835319 2432 raft_consensus.cc:493] T d30557fbe9084cd3860820f69dda8752 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260202 02:52:35.973878 528 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8153510d5d154c7895b1ee794585612e consensus state has no leader
I20260202 02:52:38.872560 2127 raft_consensus.cc:515] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:38.872802 528 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:47430 (request call id 807) took 8628 ms. Trace:
I20260202 02:52:33.058341 2429 raft_consensus.cc:493] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260202 02:52:39.332623 2372 log.cc:927] Time spent T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8: Append to log took a long time: real 0.059s user 0.004s sys 0.000s
W20260202 02:52:38.910080 1389 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.190:33981, user_credentials={real_user=slave}} blocked reactor thread for 2.66121e+06us
I20260202 02:52:33.792121 2444 raft_consensus.cc:493] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:45.269172 2372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8: Bootstrap replaying log segment 1/1 (12.6K/149.8K this segment, stats: ops{read=42 overwritten=0 applied=42 ignored=0} inserts{seen=44 ignored=0} mutations{seen=26 ignored=0} orphaned_commits=0)
I20260202 02:52:45.269256 2444 raft_consensus.cc:515] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:38.873338 2429 raft_consensus.cc:515] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:45.270155 2444 leader_election.cc:290] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:38.872943 528 rpcz_store.cc:276] 0202 02:52:30.244199 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.549054 (+304855us) service_pool.cc:225] Handling call
0202 02:52:30.549415 (+ 361us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0202 02:52:30.549624 (+ 209us) catalog_manager.cc:3708] Apply alter schema
0202 02:52:30.549799 (+ 175us) catalog_manager.cc:3740] Acquired catalog manager lock
0202 02:52:30.549877 (+ 78us) catalog_manager.cc:3898] Updating metadata on disk
0202 02:52:35.973558 (+5423681us) catalog_manager.cc:3929] Committing alterations to in-memory state
0202 02:52:38.872745 (+2899187us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:30.611488 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:30.611644 (+ 156us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:30.611652 (+ 8us) write_op.cc:435] Acquired schema lock
0202 02:52:30.611655 (+ 3us) tablet.cc:660] Decoding operations
0202 02:52:30.611788 (+ 133us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:30.611801 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:30.611807 (+ 6us) tablet.cc:683] Acquiring locks for 1 operations
0202 02:52:30.611878 (+ 71us) tablet.cc:699] Row locks acquired
0202 02:52:30.611880 (+ 2us) write_op.cc:260] PREPARE: finished
0202 02:52:30.611928 (+ 48us) write_op.cc:270] Start()
0202 02:52:30.611968 (+ 40us) write_op.cc:276] Timestamp: P: 1770000750611871 usec, L: 0
0202 02:52:30.611971 (+ 3us) op_driver.cc:348] REPLICATION: starting
0202 02:52:30.612188 (+ 217us) log.cc:844] Serialized 405 byte log entry
0202 02:52:35.972722 (+5360534us) op_driver.cc:464] REPLICATION: finished
0202 02:52:35.973028 (+ 306us) write_op.cc:301] APPLY: starting
0202 02:52:35.973048 (+ 20us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:35.973095 (+ 47us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:35.973130 (+ 35us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:35.973196 (+ 66us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:35.973231 (+ 35us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0202 02:52:35.973238 (+ 7us) write_op.cc:312] APPLY: finished
0202 02:52:35.973328 (+ 90us) log.cc:844] Serialized 23 byte log entry
0202 02:52:35.973396 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:35.973418 (+ 22us) write_op.cc:454] Released schema lock
0202 02:52:35.973425 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0202 02:52:35.973512 (+ 87us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:35.973515 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":248,"apply.run_cpu_time_us":474,"apply.run_wall_time_us":474,"num_ops":1,"prepare.queue_time_us":61108,"prepare.run_cpu_time_us":794,"prepare.run_wall_time_us":793,"replication_time_us":5360712}]]}
I20260202 02:52:45.270638 528 master_service.cc:954] ReplaceTablet: received request to replace tablet f94f25f4059a43b8babedb6b45a08a19 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:45.270835 2429 leader_election.cc:290] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:33.945133 2447 raft_consensus.cc:493] T 622a113af04a41c6852c29fc8a8acd78 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
I20260202 02:52:45.271039 2447 raft_consensus.cc:515] T 622a113af04a41c6852c29fc8a8acd78 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:45.271664 2447 leader_election.cc:290] T 622a113af04a41c6852c29fc8a8acd78 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:33.455242 2431 raft_consensus.cc:493] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b1f05820b6034228bdad884cc348716d)
I20260202 02:52:45.271867 2431 raft_consensus.cc:515] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:38.872593 2432 raft_consensus.cc:515] T d30557fbe9084cd3860820f69dda8752 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:35.836760 2433 raft_consensus.cc:493] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:45.272481 2431 leader_election.cc:290] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:36.015074 524 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:47430 (request call id 803) took 5771 ms. Trace:
I20260202 02:52:45.272535 2433 raft_consensus.cc:515] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:45.273033 2432 leader_election.cc:290] T d30557fbe9084cd3860820f69dda8752 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:45.273064 2433 leader_election.cc:290] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:45.272639 524 rpcz_store.cc:276] 0202 02:52:30.243895 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.327795 (+ 83900us) service_pool.cc:225] Handling call
0202 02:52:30.334182 (+ 6387us) spinlock_profiling.cc:243] Waited 6.08 ms on lock 0x616000041254. stack: 00007fc023f2da86 00007fc023f2d840 00007fc022f083ad 0000000000984f94 0000000000984f5d 00007fc0332576bd 00007fc0332577b3 00007fc02fe1e521 00007fc037ae96fd 00007fc037c75d93 00007fc02e7747c6 00007fc02e7744f5 00007fc02aef055e 00007fc02aeef43e 00007fc02aef5675 00007fc02aef5f9f
0202 02:52:30.552527 (+218345us) catalog_manager.cc:2101] Acquired catalog manager lock
0202 02:52:30.552990 (+ 463us) catalog_manager.cc:2147] Created new table and tablet info
0202 02:52:36.014745 (+5461755us) catalog_manager.cc:2207] Wrote table and tablets to system table
0202 02:52:36.014873 (+ 128us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0202 02:52:36.014988 (+ 115us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:30.612905 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:30.612988 (+ 83us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:30.612992 (+ 4us) write_op.cc:435] Acquired schema lock
0202 02:52:30.612995 (+ 3us) tablet.cc:660] Decoding operations
0202 02:52:30.613061 (+ 66us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:30.613069 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:30.613073 (+ 4us) tablet.cc:683] Acquiring locks for 2 operations
0202 02:52:30.613129 (+ 56us) tablet.cc:699] Row locks acquired
0202 02:52:30.613132 (+ 3us) write_op.cc:260] PREPARE: finished
0202 02:52:30.613159 (+ 27us) write_op.cc:270] Start()
0202 02:52:30.613182 (+ 23us) write_op.cc:276] Timestamp: P: 1770000750613108 usec, L: 0
0202 02:52:30.613185 (+ 3us) op_driver.cc:348] REPLICATION: starting
0202 02:52:30.613313 (+ 128us) log.cc:844] Serialized 433 byte log entry
0202 02:52:36.013947 (+5400634us) op_driver.cc:464] REPLICATION: finished
0202 02:52:36.014122 (+ 175us) write_op.cc:301] APPLY: starting
0202 02:52:36.014141 (+ 19us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:36.014194 (+ 53us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:36.014235 (+ 41us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:36.014363 (+ 128us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:36.014400 (+ 37us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0202 02:52:36.014408 (+ 8us) write_op.cc:312] APPLY: finished
0202 02:52:36.014510 (+ 102us) log.cc:844] Serialized 29 byte log entry
0202 02:52:36.014586 (+ 76us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:36.014610 (+ 24us) write_op.cc:454] Released schema lock
0202 02:52:36.014617 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0202 02:52:36.014703 (+ 86us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:36.014707 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":13980928,"child_traces":[["op",{"apply.queue_time_us":113,"apply.run_cpu_time_us":572,"apply.run_wall_time_us":573,"num_ops":2,"prepare.queue_time_us":59460,"prepare.run_cpu_time_us":456,"prepare.run_wall_time_us":456,"replication_time_us":5400724}]]}
I20260202 02:52:35.836872 2434 raft_consensus.cc:493] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:33.945056 2446 raft_consensus.cc:493] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
I20260202 02:52:45.273309 2434 raft_consensus.cc:515] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:45.273561 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_48929b86815b49369aff936433cae314"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:45.330811 2434 leader_election.cc:290] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:35.837474 2423 raft_consensus.cc:493] T 78f122342dd144b48de89902faa0f94f P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f442ef85029461091de847118c34c0d)
I20260202 02:52:35.848944 2482 consensus_queue.cc:1048] T 3d1a7578e83447669e3c3972a426ad44 P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260202 02:52:35.964375 1173 ts_tablet_manager.cc:1434] T 6b8f924aecf44e959e47d2f32e6b0bf5 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.196s user 0.000s sys 0.011s
I20260202 02:52:45.331072 2423 raft_consensus.cc:515] T 78f122342dd144b48de89902faa0f94f P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
W20260202 02:52:45.331137 1173 ts_tablet_manager.cc:1467] T 6b8f924aecf44e959e47d2f32e6b0bf5 P b1f05820b6034228bdad884cc348716d: Tablet startup took 9562ms
W20260202 02:52:45.331310 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_48929b86815b49369aff936433cae314 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:45.331271 1173 ts_tablet_manager.cc:1469] T 6b8f924aecf44e959e47d2f32e6b0bf5 P b1f05820b6034228bdad884cc348716d: Trace:
0202 02:52:20.567654 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:20.567717 (+ 63us) service_pool.cc:225] Handling call
0202 02:52:20.568366 (+ 649us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:20.568396 (+ 30us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:20.572819 (+ 4423us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:20.576774 (+ 3955us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:20.576946 (+ 172us) inbound_call.cc:173] Queueing success response
0202 02:52:30.268464 (+9691518us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0202 02:52:35.768591 (+5500127us) ts_tablet_manager.cc:1435] Starting tablet replica
0202 02:52:35.768755 (+ 164us) tablet_replica.cc:261] Starting instrumentation
0202 02:52:35.768885 (+ 130us) tablet_replica.cc:280] Starting consensus
Metrics: {"raft.queue_time_us":2605,"raft.run_cpu_time_us":43,"raft.run_wall_time_us":43,"tablet-open.queue_time_us":9691572,"thread_start_us":194167,"threads_started":1}
I20260202 02:52:45.331645 2423 leader_election.cc:290] T 78f122342dd144b48de89902faa0f94f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:45.331892 1173 tablet_bootstrap.cc:492] T b7880cb051c24bc998003f0effed3cf8 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:33.605913 2436 raft_consensus.cc:493] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b1f05820b6034228bdad884cc348716d)
I20260202 02:52:33.702409 2439 raft_consensus.cc:493] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f442ef85029461091de847118c34c0d)
I20260202 02:52:45.273866 2446 raft_consensus.cc:515] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:45.332712 2439 raft_consensus.cc:515] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:45.333283 2446 leader_election.cc:290] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:45.333330 2439 leader_election.cc:290] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:36.016196 520 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 3dcf4f794ec14b2f9d181d015fc437fb
I20260202 02:52:33.711336 2441 raft_consensus.cc:493] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f442ef85029461091de847118c34c0d)
I20260202 02:52:36.567783 2501 tablet.cc:1722] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:45.333491 2441 raft_consensus.cc:515] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:34.127337 2449 raft_consensus.cc:493] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:31.518158 743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efe78242e0654a1c9bdbaf9cfa88b034. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:45.333858 2449 raft_consensus.cc:515] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:45.334002 2441 leader_election.cc:290] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:33.837446 2445 raft_consensus.cc:493] T af5aa68e47bb420eac300754d274fa73 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b1f05820b6034228bdad884cc348716d)
I20260202 02:52:33.917603 2443 raft_consensus.cc:493] T 3d1a7578e83447669e3c3972a426ad44 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:45.334161 2445 raft_consensus.cc:515] T af5aa68e47bb420eac300754d274fa73 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:45.334232 2443 raft_consensus.cc:515] T 3d1a7578e83447669e3c3972a426ad44 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:45.334515 2449 leader_election.cc:290] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:36.017712 521 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:47430 (request call id 810) took 5756 ms. Trace:
I20260202 02:52:45.334626 2445 leader_election.cc:290] T af5aa68e47bb420eac300754d274fa73 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:35.769413 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79c73f4a92044bafb5b8384229cb3a4d" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:45.334815 2443 leader_election.cc:290] T 3d1a7578e83447669e3c3972a426ad44 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:45.332712 2436 raft_consensus.cc:515] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:37.069291 2500 raft_consensus.cc:493] T 26eeef6eebbf4f2cb44f69255fb37594 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260202 02:52:37.331171 998 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 50891us
I20260202 02:52:45.525403 2436 leader_election.cc:290] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
W20260202 02:52:45.334878 772 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.131:34317 (request call id 235) took 9565 ms (client timeout 4118 ms). Trace:
I20260202 02:52:45.525396 2500 raft_consensus.cc:515] T 26eeef6eebbf4f2cb44f69255fb37594 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
W20260202 02:52:35.921876 713 connection.cc:232] RPC call timeout handler was delayed by -3.6133665020000194s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260202 02:52:45.525609 772 rpcz_store.cc:269] 0202 02:52:35.769280 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:35.769334 (+ 54us) service_pool.cc:225] Handling call
0202 02:52:45.334854 (+9565520us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:34.764526 2451 raft_consensus.cc:493] T 474df41e4fe24a818fa45a4ce84d5474 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:45.526089 2500 leader_election.cc:290] T 26eeef6eebbf4f2cb44f69255fb37594 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:45.526022 2451 raft_consensus.cc:515] T 474df41e4fe24a818fa45a4ce84d5474 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:45.334717 521 rpcz_store.cc:276] 0202 02:52:30.261564 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.560122 (+298558us) service_pool.cc:225] Handling call
0202 02:52:30.560992 (+ 870us) catalog_manager.cc:2101] Acquired catalog manager lock
0202 02:52:30.561438 (+ 446us) catalog_manager.cc:2147] Created new table and tablet info
0202 02:52:36.017069 (+5455631us) catalog_manager.cc:2207] Wrote table and tablets to system table
0202 02:52:36.017237 (+ 168us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0202 02:52:36.017658 (+ 421us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:30.614475 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:30.614557 (+ 82us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:30.614561 (+ 4us) write_op.cc:435] Acquired schema lock
0202 02:52:30.614563 (+ 2us) tablet.cc:660] Decoding operations
0202 02:52:30.614627 (+ 64us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:30.614634 (+ 7us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:30.614639 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
0202 02:52:30.614717 (+ 78us) tablet.cc:699] Row locks acquired
0202 02:52:30.614719 (+ 2us) write_op.cc:260] PREPARE: finished
0202 02:52:30.614746 (+ 27us) write_op.cc:270] Start()
0202 02:52:30.614768 (+ 22us) write_op.cc:276] Timestamp: P: 1770000750614695 usec, L: 0
0202 02:52:30.614772 (+ 4us) op_driver.cc:348] REPLICATION: starting
0202 02:52:30.614894 (+ 122us) log.cc:844] Serialized 433 byte log entry
0202 02:52:36.016547 (+5401653us) op_driver.cc:464] REPLICATION: finished
0202 02:52:36.016623 (+ 76us) write_op.cc:301] APPLY: starting
0202 02:52:36.016634 (+ 11us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:36.016664 (+ 30us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:36.016695 (+ 31us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:36.016796 (+ 101us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:36.016822 (+ 26us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0202 02:52:36.016830 (+ 8us) write_op.cc:312] APPLY: finished
0202 02:52:36.016884 (+ 54us) log.cc:844] Serialized 29 byte log entry
0202 02:52:36.016940 (+ 56us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:36.016955 (+ 15us) write_op.cc:454] Released schema lock
0202 02:52:36.016960 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":35,"apply.run_cpu_time_us":394,"apply.run_wall_time_us":394,"num_ops":2,"prepare.queue_time_us":52479,"prepare.run_cpu_time_us":481,"prepare.run_wall_time_us":463,"replication_time_us":5401750}]]}
W20260202 02:52:45.526417 998 connection.cc:232] RPC call timeout handler was delayed by -6.7773907780000116s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3108s for a response
W20260202 02:52:45.526523 998 connection.cc:232] RPC call timeout handler was delayed by -6.7323907780000241s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3158s for a response
W20260202 02:52:45.526592 998 connection.cc:232] RPC call timeout handler was delayed by -6.7206907780000051s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3171s for a response
I20260202 02:52:45.526594 2451 leader_election.cc:290] T 474df41e4fe24a818fa45a4ce84d5474 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:35.921901 2157 raft_consensus.cc:493] T ef073880898049e2be93d2cb6dc0c0bb P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b1f05820b6034228bdad884cc348716d)
W20260202 02:52:35.922142 714 connection.cc:232] RPC call timeout handler was delayed by -4.303350402999996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260202 02:52:45.655361 714 connection.cc:232] RPC call timeout handler was delayed by -2.1532275039999718s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.34220000000000006s for a response
I20260202 02:52:45.655272 2157 raft_consensus.cc:515] T ef073880898049e2be93d2cb6dc0c0bb P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:45.655962 2157 leader_election.cc:290] T ef073880898049e2be93d2cb6dc0c0bb P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:45.656111 2157 raft_consensus.cc:493] T f6746ccbf37e477196a9706396d1e3fd P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b1f05820b6034228bdad884cc348716d)
I20260202 02:52:45.656234 2157 raft_consensus.cc:515] T f6746ccbf37e477196a9706396d1e3fd P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:45.526597 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_33c051aaf8e1494fa559ae70367f89e9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:45.656764 2157 leader_election.cc:290] T f6746ccbf37e477196a9706396d1e3fd P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:45.656890 2157 raft_consensus.cc:493] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b1f05820b6034228bdad884cc348716d)
W20260202 02:52:45.656994 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_33c051aaf8e1494fa559ae70367f89e9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:45.656989 2157 raft_consensus.cc:515] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
W20260202 02:52:45.657083 714 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.104.130:38539 timed out after 3.422s (SENT)
I20260202 02:52:45.657419 2157 leader_election.cc:290] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:45.657537 2157 raft_consensus.cc:493] T be993c1512b04df4b92cab3195241d84 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b1f05820b6034228bdad884cc348716d)
I20260202 02:52:45.657635 2157 raft_consensus.cc:515] T be993c1512b04df4b92cab3195241d84 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:45.658241 2157 leader_election.cc:290] T be993c1512b04df4b92cab3195241d84 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:35.778218 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8e489f59d1345549e29c4fa6e59c1c9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:45.526655 998 connection.cc:232] RPC call timeout handler was delayed by -6.61809077800001s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3285s for a response
I20260202 02:52:32.950732 2428 raft_consensus.cc:493] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f442ef85029461091de847118c34c0d)
W20260202 02:52:45.658602 998 connection.cc:232] RPC call timeout handler was delayed by -6.05068830700003s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.41180000000000005s for a response
W20260202 02:52:45.658735 998 connection.cc:232] RPC call timeout handler was delayed by -5.9079907780000269s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.4074s for a response
W20260202 02:52:45.658802 998 connection.cc:232] RPC call timeout handler was delayed by -5.8953907780000065s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.40880000000000005s for a response
W20260202 02:52:45.658874 998 connection.cc:232] RPC call timeout handler was delayed by -5.8476907780000147s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.4141s for a response
W20260202 02:52:45.658934 998 connection.cc:232] RPC call timeout handler was delayed by -5.785590778000028s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.42100000000000004s for a response
I20260202 02:52:45.658663 2428 raft_consensus.cc:515] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:45.659001 998 connection.cc:232] RPC call timeout handler was delayed by -5.6082907780000255s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.44070000000000004s for a response
W20260202 02:52:45.659060 998 connection.cc:232] RPC call timeout handler was delayed by -3.6192907780000212s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.66170000000000007s for a response
I20260202 02:52:45.659534 2428 leader_election.cc:290] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:38.873149 2127 leader_election.cc:290] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:45.659631 998 leader_election.cc:343] T 79c73f4a92044bafb5b8384229cb3a4d P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:35.922515 2377 raft_consensus.cc:493] T 3d1a7578e83447669e3c3972a426ad44 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:45.659879 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 104b2c537c4f4e3da6711364cb5d2a2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:45.659888 2377 raft_consensus.cc:515] T 3d1a7578e83447669e3c3972a426ad44 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:35.922873 2488 raft_consensus.cc:493] T b76d1837813c46e8a0925b49939d8884 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
I20260202 02:52:45.661368 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48c820d845e144249c94cd87a0f38ec3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:45.661406 2488 raft_consensus.cc:515] T b76d1837813c46e8a0925b49939d8884 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:45.662014 2488 leader_election.cc:290] T b76d1837813c46e8a0925b49939d8884 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:35.943724 2486 tablet.cc:1717] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d: Skipping requested alter to schema version 1, tablet already version 1
I20260202 02:52:45.663168 760 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d30557fbe9084cd3860820f69dda8752" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:45.663569 760 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fe53a8bdbd13417f9ce195c54a52a93c" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:45.663700 760 raft_consensus.cc:2468] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:45.663892 760 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26eeef6eebbf4f2cb44f69255fb37594" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:35.956559 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d2d27ceea1bb4122b18f0fcf4d2d1bf4" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:45.664140 776 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.131:34317 (request call id 236) took 9707 ms (client timeout 4141 ms). Trace:
W20260202 02:52:45.664237 776 rpcz_store.cc:269] 0202 02:52:35.956368 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:35.956434 (+ 66us) service_pool.cc:225] Handling call
0202 02:52:45.664124 (+9707690us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:35.841288 886 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41736 (request call id 708) took 5235 ms. Trace:
I20260202 02:52:45.664441 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efe78242e0654a1c9bdbaf9cfa88b034. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:45.664400 886 rpcz_store.cc:276] 0202 02:52:30.605493 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.605833 (+ 340us) service_pool.cc:225] Handling call
0202 02:52:35.760458 (+5154625us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:35.760484 (+ 26us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:35.763496 (+ 3012us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:35.841058 (+ 77562us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:35.841257 (+ 199us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:45.664911 886 tablet_service.cc:1505] Processing CreateTablet for tablet fe97009dbcc64846844e381817148d7b (DEFAULT_TABLE table=default.Table_df42f88a6807472088543ed874ac5e76 [id=b2b4d069db034e46b4d9f3fa935b363e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:45.665256 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe97009dbcc64846844e381817148d7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:35.957095 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dada209d982d48f3b74a49905f19c5b6" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:36.019351 525 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:47430 (request call id 805) took 5775 ms. Trace:
W20260202 02:52:45.665407 773 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.131:34317 (request call id 238) took 9708 ms (client timeout 3171 ms). Trace:
W20260202 02:52:45.665481 773 rpcz_store.cc:269] 0202 02:52:35.956964 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:35.957038 (+ 74us) service_pool.cc:225] Handling call
0202 02:52:45.665391 (+9708353us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:45.665504 525 rpcz_store.cc:276] 0202 02:52:30.244037 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.437703 (+193666us) service_pool.cc:225] Handling call
0202 02:52:30.565703 (+128000us) catalog_manager.cc:2101] Acquired catalog manager lock
0202 02:52:30.566137 (+ 434us) catalog_manager.cc:2147] Created new table and tablet info
0202 02:52:36.019099 (+5452962us) catalog_manager.cc:2207] Wrote table and tablets to system table
0202 02:52:36.019187 (+ 88us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0202 02:52:36.019282 (+ 95us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:30.707625 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:30.708063 (+ 438us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:30.708120 (+ 57us) write_op.cc:435] Acquired schema lock
0202 02:52:30.708123 (+ 3us) tablet.cc:660] Decoding operations
0202 02:52:30.708692 (+ 569us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:30.708835 (+ 143us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:30.708841 (+ 6us) tablet.cc:683] Acquiring locks for 2 operations
0202 02:52:30.709801 (+ 960us) tablet.cc:699] Row locks acquired
0202 02:52:30.709804 (+ 3us) write_op.cc:260] PREPARE: finished
0202 02:52:30.710193 (+ 389us) write_op.cc:270] Start()
0202 02:52:30.710641 (+ 448us) write_op.cc:276] Timestamp: P: 1770000750709990 usec, L: 0
0202 02:52:30.710647 (+ 6us) op_driver.cc:348] REPLICATION: starting
0202 02:52:30.712766 (+ 2119us) log.cc:844] Serialized 433 byte log entry
0202 02:52:36.018643 (+5305877us) op_driver.cc:464] REPLICATION: finished
0202 02:52:36.018712 (+ 69us) write_op.cc:301] APPLY: starting
0202 02:52:36.018722 (+ 10us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:36.018750 (+ 28us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:36.018780 (+ 30us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:36.018862 (+ 82us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:36.018888 (+ 26us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0202 02:52:36.018903 (+ 15us) write_op.cc:312] APPLY: finished
0202 02:52:36.018953 (+ 50us) log.cc:844] Serialized 29 byte log entry
0202 02:52:36.018998 (+ 45us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:36.019013 (+ 15us) write_op.cc:454] Released schema lock
0202 02:52:36.019018 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":31,"apply.run_cpu_time_us":354,"apply.run_wall_time_us":355,"num_ops":2,"prepare.queue_time_us":140988,"prepare.run_cpu_time_us":1886,"prepare.run_wall_time_us":6170,"replication_time_us":5307974,"spinlock_wait_cycles":326912}]]}
I20260202 02:52:35.843590 901 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41736 (request call id 702) took 5302 ms. Trace:
I20260202 02:52:45.666277 901 rpcz_store.cc:276] 0202 02:52:30.541308 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.547436 (+ 6128us) service_pool.cc:225] Handling call
0202 02:52:30.593891 (+ 46455us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.593935 (+ 44us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:35.804025 (+5210090us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:35.843064 (+ 39039us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:35.843467 (+ 403us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:45.666322 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_38b5d3fdb2c748cba3a6aaad03bdb766"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:45.666718 901 tablet_service.cc:1505] Processing CreateTablet for tablet 16b2d80274a54f1193a3f1746c278252 (DEFAULT_TABLE table=default.Table_0cd8f9947fbd430f8992cc1543aed9f7 [id=163eefc3d38a4b28bce88b948cc5df3c]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:45.666934 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_38b5d3fdb2c748cba3a6aaad03bdb766 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:45.667068 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16b2d80274a54f1193a3f1746c278252. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:45.668009 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ffadda20dd3431b84b2657485e7dec7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:35.964224 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "33150e1456144b8e9c64cc2a2ac6dcca" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:38.873646 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdd72f995a704520a5fb1c24a53ed252. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:45.668438 903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fed7eb10ec74f009b73fc5c66084851. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:45.668514 778 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.131:34317 (request call id 240) took 9710 ms (client timeout 3108 ms). Trace:
I20260202 02:52:45.668588 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08fbd094d4df46cf82a694b4e1f30c6d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:45.668594 778 rpcz_store.cc:269] 0202 02:52:35.964054 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:35.964116 (+ 62us) service_pool.cc:225] Handling call
0202 02:52:45.668491 (+9704375us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:35.844702 900 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41736 (request call id 703) took 5303 ms. Trace:
I20260202 02:52:45.668776 900 rpcz_store.cc:276] 0202 02:52:30.541379 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.547737 (+ 6358us) service_pool.cc:225] Handling call
0202 02:52:30.594001 (+ 46264us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.594029 (+ 28us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:35.767281 (+5173252us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:35.844511 (+ 77230us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:35.844680 (+ 169us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:45.669215 900 tablet_service.cc:1505] Processing CreateTablet for tablet 4f2627dcb9904589967407c3a8756a33 (DEFAULT_TABLE table=default.Table_5c8329977b724148a1a8d5c504a4a513 [id=893bd49b291d4748872a291b7afc6f77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:45.669535 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f2627dcb9904589967407c3a8756a33. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:36.623075 512 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 8153510d5d154c7895b1ee794585612e (table default.Table_0e7ceba4956f408c8a9fc0f61fa67c90 [id=c0c6499675894350ac008b534d030c93], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 8153510d5d154c7895b1ee794585612e consensus state has no leader
W20260202 02:52:37.731947 513 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 404125def15a4f1fb1529447785fc93e (table default.Table_defe1743f1314f478ac7a7715442942e [id=3256d26f41564428b41ecf40e820ad04], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 404125def15a4f1fb1529447785fc93e consensus state has no leader
W20260202 02:52:45.671924 513 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet bcc6cd3595364680b4c39c1ff740aebb (table default.Table_8739c38e3fcd4069a2e7c623b11519b3 [id=2152128426644acaae4a2654b834064d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet bcc6cd3595364680b4c39c1ff740aebb consensus state has no leader
I20260202 02:52:36.024971 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "75569511179b41038e521fa3346a92ae" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:38.881868 1037 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49890 (request call id 698) took 8419 ms. Trace:
I20260202 02:52:45.672487 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2654ace469dd42389a0d64bf27237d75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:45.672487 775 raft_consensus.cc:2468] T 75569511179b41038e521fa3346a92ae P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:45.672451 1037 rpcz_store.cc:276] 0202 02:52:30.462555 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.462607 (+ 52us) service_pool.cc:225] Handling call
0202 02:52:30.593298 (+130691us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.593320 (+ 22us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:38.871868 (+8278548us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:38.881594 (+ 9726us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:38.881835 (+ 241us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260202 02:52:45.672637 775 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.131:34317 (request call id 241) took 9648 ms (client timeout 6617 ms). Trace:
W20260202 02:52:45.672703 775 rpcz_store.cc:269] 0202 02:52:36.024705 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:36.024818 (+ 113us) service_pool.cc:225] Handling call
0202 02:52:45.672616 (+9647798us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:36.050581 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "91ec01d25f8f441cbe3783cacbee51db" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:45.672947 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet cd1b6662da254889ae4bbe8d0b1ef1bb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cd1b6662da254889ae4bbe8d0b1ef1bb already in progress: creating tablet
W20260202 02:52:45.672998 774 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.131:34317 (request call id 242) took 9622 ms (client timeout 3285 ms). Trace:
W20260202 02:52:45.673063 774 rpcz_store.cc:269] 0202 02:52:36.050324 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:36.050443 (+ 119us) service_pool.cc:225] Handling call
0202 02:52:45.672982 (+9622539us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:36.071933 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "70306ed9dec54286885d965f2c012da4" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:45.673154 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet c977c146bac34638874b7a990604a0d5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c977c146bac34638874b7a990604a0d5 already in progress: creating tablet
W20260202 02:52:45.673275 771 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.131:34317 (request call id 243) took 9601 ms (client timeout 3158 ms). Trace:
W20260202 02:52:45.673353 771 rpcz_store.cc:269] 0202 02:52:36.071647 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:36.071774 (+ 127us) service_pool.cc:225] Handling call
0202 02:52:45.673264 (+9601490us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260202 02:52:45.673378 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet a5f864fd8a3d4c4a83e7f68abb8a0a83 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a5f864fd8a3d4c4a83e7f68abb8a0a83 already in progress: creating tablet
I20260202 02:52:36.092037 770 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "73f8db8f30f14d369d2227c9be7c4647" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:45.673565 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08fbd094d4df46cf82a694b4e1f30c6d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:45.673578 770 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.131:34317 (request call id 244) took 9581 ms (client timeout 4088 ms). Trace:
I20260202 02:52:35.871544 902 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41736 (request call id 701) took 5330 ms. Trace:
W20260202 02:52:45.673652 770 rpcz_store.cc:269] 0202 02:52:36.091768 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:36.091882 (+ 114us) service_pool.cc:225] Handling call
0202 02:52:45.673565 (+9581683us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:36.140630 769 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b76d1837813c46e8a0925b49939d8884" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:38.762341 514 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 5e20842de8cb4355b732685605c88834 (table default.Table_6e6ab095d9a6497d8a89bcc865b6f42f [id=7d8f7ac4c2ee4b72b2ff25aff060010f], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 5e20842de8cb4355b732685605c88834 consensus state has no leader
W20260202 02:52:45.673899 769 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.131:34317 (request call id 245) took 9533 ms (client timeout 4074 ms). Trace:
W20260202 02:52:45.674000 769 rpcz_store.cc:269] 0202 02:52:36.140260 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:36.140390 (+ 130us) service_pool.cc:225] Handling call
0202 02:52:45.673885 (+9533495us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:36.181985 768 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "63a9029bc39c4fbd8ac4bf4bc37d6b4e" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:45.674223 768 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.131:34317 (request call id 246) took 9492 ms (client timeout 4210 ms). Trace:
W20260202 02:52:45.674296 768 rpcz_store.cc:269] 0202 02:52:36.181652 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:36.181764 (+ 112us) service_pool.cc:225] Handling call
0202 02:52:45.674211 (+9492447us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:36.209159 767 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "026e6ee0b3ae4d938fdf1550044ed1fa" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:36.628573 2503 tablet.cc:1722] T be993c1512b04df4b92cab3195241d84 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:35.837242 2427 raft_consensus.cc:493] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
I20260202 02:52:45.674674 2427 raft_consensus.cc:515] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:45.675344 2427 leader_election.cc:290] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:35.874418 894 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41736 (request call id 711) took 5268 ms. Trace:
I20260202 02:52:45.675625 894 rpcz_store.cc:276] 0202 02:52:30.605719 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.606172 (+ 453us) service_pool.cc:225] Handling call
0202 02:52:35.763800 (+5157628us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:35.763826 (+ 26us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:35.766633 (+ 2807us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:35.874111 (+107478us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:35.874391 (+ 280us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260202 02:52:35.974442 519 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bb145df67e1a44569a3dbc91c024b584 consensus state has no leader
I20260202 02:52:45.676036 519 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:47430 (request call id 808) took 15431 ms. Trace:
I20260202 02:52:33.891783 2438 raft_consensus.cc:493] T 73f8db8f30f14d369d2227c9be7c4647 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
I20260202 02:52:45.676151 519 rpcz_store.cc:276] 0202 02:52:30.244265 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.550007 (+305742us) service_pool.cc:225] Handling call
0202 02:52:30.550296 (+ 289us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0202 02:52:30.550483 (+ 187us) catalog_manager.cc:3708] Apply alter schema
0202 02:52:30.550633 (+ 150us) catalog_manager.cc:3740] Acquired catalog manager lock
0202 02:52:30.550700 (+ 67us) catalog_manager.cc:3898] Updating metadata on disk
0202 02:52:35.974224 (+5423524us) catalog_manager.cc:3929] Committing alterations to in-memory state
0202 02:52:45.675995 (+9701771us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:30.612315 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:30.612438 (+ 123us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:30.612442 (+ 4us) write_op.cc:435] Acquired schema lock
0202 02:52:30.612444 (+ 2us) tablet.cc:660] Decoding operations
0202 02:52:30.612527 (+ 83us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:30.612536 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:30.612541 (+ 5us) tablet.cc:683] Acquiring locks for 1 operations
0202 02:52:30.612601 (+ 60us) tablet.cc:699] Row locks acquired
0202 02:52:30.612603 (+ 2us) write_op.cc:260] PREPARE: finished
0202 02:52:30.612635 (+ 32us) write_op.cc:270] Start()
0202 02:52:30.612663 (+ 28us) write_op.cc:276] Timestamp: P: 1770000750612582 usec, L: 0
0202 02:52:30.612666 (+ 3us) op_driver.cc:348] REPLICATION: starting
0202 02:52:30.612818 (+ 152us) log.cc:844] Serialized 405 byte log entry
0202 02:52:35.972796 (+5359978us) op_driver.cc:464] REPLICATION: finished
0202 02:52:35.973585 (+ 789us) write_op.cc:301] APPLY: starting
0202 02:52:35.973596 (+ 11us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:35.973627 (+ 31us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:35.973649 (+ 22us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:35.973687 (+ 38us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:35.973711 (+ 24us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0202 02:52:35.973718 (+ 7us) write_op.cc:312] APPLY: finished
0202 02:52:35.973797 (+ 79us) log.cc:844] Serialized 23 byte log entry
0202 02:52:35.973836 (+ 39us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:35.973848 (+ 12us) write_op.cc:454] Released schema lock
0202 02:52:35.973854 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":758,"apply.run_cpu_time_us":336,"apply.run_wall_time_us":336,"num_ops":1,"prepare.queue_time_us":61229,"prepare.run_cpu_time_us":555,"prepare.run_wall_time_us":554,"replication_time_us":5360113,"thread_start_us":40934,"threads_started":1}]]}
I20260202 02:52:45.676433 2438 raft_consensus.cc:515] T 73f8db8f30f14d369d2227c9be7c4647 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:45.676893 519 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_c33bb328c09b4b6ab9974ec216487b8a" } new_table_name: "default.Table_a77859a40f12489a8578b7eb1a66f0d4" modify_external_catalogs: true
I20260202 02:52:35.935643 1175 tablet_bootstrap.cc:492] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:45.677215 2377 leader_election.cc:290] T 3d1a7578e83447669e3c3972a426ad44 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:45.677405 1175 ts_tablet_manager.cc:1403] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 15.132s user 0.010s sys 0.004s
I20260202 02:52:38.872434 2435 raft_consensus.cc:515] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:45.678097 2435 leader_election.cc:290] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:35.835980 999 connection.cc:232] RPC call timeout handler was delayed by -4.237167137s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260202 02:52:45.528394 2482 raft_consensus.cc:493] T c050a2882f764dda8df2a7df6547f149 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f442ef85029461091de847118c34c0d)
I20260202 02:52:45.678351 2482 raft_consensus.cc:515] T c050a2882f764dda8df2a7df6547f149 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:38.874408 527 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.104.130:36691 (request call id 83) took 3115 ms (client timeout 1000 ms). Trace:
I20260202 02:52:45.678699 2432 raft_consensus.cc:493] T 622a113af04a41c6852c29fc8a8acd78 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
W20260202 02:52:45.678695 527 rpcz_store.cc:269] 0202 02:52:35.758749 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:35.766549 (+ 7800us) service_pool.cc:225] Handling call
0202 02:52:38.874329 (+3107780us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":7}
I20260202 02:52:45.678826 2432 raft_consensus.cc:515] T 622a113af04a41c6852c29fc8a8acd78 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:45.678979 2482 leader_election.cc:290] T c050a2882f764dda8df2a7df6547f149 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:45.679098 2482 raft_consensus.cc:493] T 06b80a1c9251420cbc16fc95ae98aca9 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:45.679136 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_c7654f22d0044539a2eb8a736163a22d" } new_table_name: "default.Table_9bfaf1dc7bd8452ba14d3aa60e4160ff" modify_external_catalogs: true
I20260202 02:52:45.679203 2482 raft_consensus.cc:515] T 06b80a1c9251420cbc16fc95ae98aca9 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:45.679304 2432 leader_election.cc:290] T 622a113af04a41c6852c29fc8a8acd78 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:45.679415 2432 raft_consensus.cc:493] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:45.679518 2432 raft_consensus.cc:515] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:45.680244 2432 leader_election.cc:290] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:35.944532 859 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 5.35153e+06us
I20260202 02:52:45.680554 2423 raft_consensus.cc:493] T d06b77ff1a6f4f78b73082b00145fcf6 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
I20260202 02:52:45.680766 2423 raft_consensus.cc:515] T d06b77ff1a6f4f78b73082b00145fcf6 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:45.681272 2423 leader_election.cc:290] T d06b77ff1a6f4f78b73082b00145fcf6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:35.964242 897 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41736 (request call id 705) took 5422 ms. Trace:
I20260202 02:52:38.876986 779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.104.131:34317 (request call id 234) took 3107 ms. Trace:
I20260202 02:52:45.681380 897 rpcz_store.cc:276] 0202 02:52:30.541506 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.548388 (+ 6882us) service_pool.cc:225] Handling call
0202 02:52:30.594285 (+ 45897us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.594310 (+ 25us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:35.961338 (+5367028us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:35.964062 (+ 2724us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:35.964219 (+ 157us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260202 02:52:35.989543 2416 tablet_replica.cc:1410] Time spent flushing the Log queue took a long time: real 0.212s user 0.000s sys 0.011s
I20260202 02:52:45.681548 779 rpcz_store.cc:276] 0202 02:52:35.769154 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:35.770320 (+ 1166us) service_pool.cc:225] Handling call
0202 02:52:35.770443 (+ 123us) raft_consensus.cc:1471] Updating replica for 0 ops
0202 02:52:38.873617 (+3103174us) raft_consensus.cc:1534] Early marking committed up to index 0
0202 02:52:38.873622 (+ 5us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0202 02:52:38.873636 (+ 14us) raft_consensus.cc:1645] Marking committed up to 2
0202 02:52:38.873642 (+ 6us) raft_consensus.cc:1697] Filling consensus response to leader.
0202 02:52:38.876769 (+ 3127us) spinlock_profiling.cc:243] Waited 3.1 s on lock 0x6160000a2154. stack: 00007f1e37b5aa86 00007f1e37b5a840 00007f1e36b353ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f1e43a2d93b 00007f1e43a296c7 00007f1e478a321d 00007f1e41a3cb46 00007f1e41a3c875 00007f1e3eb1d55e 00007f1e3eb1c43e 00007f1e3eb22675 00007f1e3eb22f9f 00007f1e3eb22dc8
0202 02:52:38.876781 (+ 12us) raft_consensus.cc:1691] UpdateReplicas() finished
0202 02:52:38.876954 (+ 173us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":7136353536}
I20260202 02:52:38.876783 2426 raft_consensus.cc:515] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:45.682191 766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.104.131:34317 (request call id 249) took 9387 ms. Trace:
I20260202 02:52:45.682381 2426 leader_election.cc:290] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:35.998857 884 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41736 (request call id 699) took 5445 ms. Trace:
I20260202 02:52:45.682284 766 rpcz_store.cc:276] 0202 02:52:36.295140 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:36.295256 (+ 116us) service_pool.cc:225] Handling call
0202 02:52:36.295462 (+ 206us) raft_consensus.cc:1471] Updating replica for 0 ops
0202 02:52:45.681963 (+9386501us) raft_consensus.cc:1534] Early marking committed up to index 0
0202 02:52:45.681968 (+ 5us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0202 02:52:45.681980 (+ 12us) raft_consensus.cc:1645] Marking committed up to 1
0202 02:52:45.681985 (+ 5us) raft_consensus.cc:1697] Filling consensus response to leader.
0202 02:52:45.682118 (+ 133us) spinlock_profiling.cc:243] Waited 9.39 s on lock 0x616000192454. stack: 00007f1e37b5aa86 00007f1e37b5a840 00007f1e36b353ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f1e43a2d93b 00007f1e43a296c7 00007f1e478a321d 00007f1e41a3cb46 00007f1e41a3c875 00007f1e3eb1d55e 00007f1e3eb1c43e 00007f1e3eb22675 00007f1e3eb22f9f 00007f1e3eb22dc8
0202 02:52:45.682124 (+ 6us) raft_consensus.cc:1691] UpdateReplicas() finished
0202 02:52:45.682172 (+ 48us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":21586595712}
W20260202 02:52:39.864621 515 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet bb145df67e1a44569a3dbc91c024b584 (table default.Table_8c1b93c2a1234d058590e574f89b770b [id=bf2ff444381f4d62a22258ffbfe1c040], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet bb145df67e1a44569a3dbc91c024b584 consensus state has no leader
I20260202 02:52:45.682538 884 rpcz_store.cc:276] 0202 02:52:30.541097 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.546964 (+ 5867us) service_pool.cc:225] Handling call
0202 02:52:30.594204 (+ 47240us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.594228 (+ 24us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:35.762872 (+5168644us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:35.960045 (+197173us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:35.979919 (+ 19874us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:45.682799 884 tablet_service.cc:1552] Processing DeleteTablet for tablet d8418a9dd0944456b4d4820dde0cde02 with delete_type TABLET_DATA_DELETED (replaced by tablet 1b2b4f429b6a484ea6ef7c19124b236b) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:45.682906 999 connection.cc:232] RPC call timeout handler was delayed by -6.2055584979999878s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.41180000000000005s for a response
I20260202 02:52:45.682963 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 91ec01d25f8f441cbe3783cacbee51db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:30 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:45.683089 884 tablet_service.cc:1552] Processing DeleteTablet for tablet fbcb4a522df648ba9542250450c08f1b with delete_type TABLET_DATA_DELETED (replaced by tablet 16b2d80274a54f1193a3f1746c278252) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:45.683208 884 tablet_service.cc:1552] Processing DeleteTablet for tablet f1fadf2cb5be4ec8980871fc96ee9730 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:30 UTC) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:38.944275 715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 3.18649e+06us
I20260202 02:52:45.683455 715 leader_election.cc:304] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:45.677954 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1dd3b956ae594e89b0599dbc9345f5c3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:45.684094 715 consensus_peers.cc:597] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2f0e8c5e2fd44370ae150a206f4e6cbc. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:45.685362 1054 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "20ea39bc5d3d46f5a4ba427277d6c6a3" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:45.685817 1053 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "78f122342dd144b48de89902faa0f94f" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:45.685993 1053 raft_consensus.cc:2468] T 78f122342dd144b48de89902faa0f94f P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
I20260202 02:52:45.686303 1052 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28a900bcaa8b41718a285e595886d805" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:33.620985 857 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.104.129:39403 timed out after 3.015s (SENT)
I20260202 02:52:45.686478 1052 raft_consensus.cc:2468] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 2.
I20260202 02:52:45.686694 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f94f25f4059a43b8babedb6b45a08a19" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:45.686744 1051 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "af5aa68e47bb420eac300754d274fa73" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:36.015506 526 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 09a8aa4dd70b4fe2806293d2620e521c
I20260202 02:52:45.687486 526 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:47430 (request call id 802) took 15443 ms. Trace:
I20260202 02:52:33.110416 2430 raft_consensus.cc:493] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260202 02:52:39.359097 840 kernel_stack_watchdog.cc:198] Thread 857 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 4490ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fb3588d9980 (unknown) at ??:0
@ 0x7fb3588d39c6 __GI___pthread_rwlock_wrlock at ??:0
@ 0x7fb35254c904 google::LogMessage::Flush() at ??:0
@ 0x7fb35254ca11 google::LogMessage::~LogMessage() at ??:0
@ 0x7fb35c1ce604 kudu::rpc::Proxy::AsyncRequest()::$_1::operator()() at ??:0
@ 0x7fb35c1cdebe std::_Function_handler<>::_M_invoke() at ??:0
@ 0xa8970d std::function<>::operator()() at ??:?
@ 0x7fb35c1b9f10 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7fb35c1bb602 kudu::rpc::OutboundCall::SetTimedOut() at ??:0
@ 0x7fb35c151453 kudu::rpc::Connection::HandleOutboundCallTimeout() at ??:0
@ 0x7fb35c150fc6 kudu::rpc::Connection::CallAwaitingResponse::HandleTimeout() at ??:0
@ 0x7fb35c169045 ev::base<>::method_thunk<>() at ??:0
@ 0x7fb353a73d5e ev_invoke_pending at ??:0
@ 0x7fb35c1d4765 kudu::rpc::ReactorThread::InvokePendingCb() at ??:0
@ 0x7fb353a74ca6 ev_run at ??:0
@ 0x7fb35c1e4361 ev::loop_ref::run() at ??:0
I20260202 02:52:45.687605 526 rpcz_store.cc:276] 0202 02:52:30.243805 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.299496 (+ 55691us) service_pool.cc:225] Handling call
0202 02:52:30.342965 (+ 43469us) spinlock_profiling.cc:243] Waited 43.2 ms on lock 0x616000041254. stack: 00007fc023f2da86 00007fc023f2d840 00007fc022f083ad 0000000000984f94 0000000000984f5d 00007fc0332576bd 00007fc0332577b3 00007fc02fe1e521 00007fc037ae96fd 00007fc037c76d86 00007fc02e776486 00007fc02e7761b5 00007fc02aef055e 00007fc02aeef43e 00007fc02aef5675 00007fc02aef5f9f
0202 02:52:30.554244 (+211279us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0202 02:52:30.554385 (+ 141us) catalog_manager.cc:2815] Modifying in-memory table state
0202 02:52:30.554447 (+ 62us) catalog_manager.cc:2824] Locking tablets
0202 02:52:30.554578 (+ 131us) catalog_manager.cc:2838] Removing table and tablets from system table
0202 02:52:36.015217 (+5460639us) catalog_manager.cc:2856] Removing table from by-name map
0202 02:52:36.015304 (+ 87us) catalog_manager.cc:2872] Committing in-memory state
0202 02:52:36.015334 (+ 30us) catalog_manager.cc:2876] Aborting table tasks
0202 02:52:45.687427 (+9672093us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:30.613396 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:30.613475 (+ 79us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:30.613479 (+ 4us) write_op.cc:435] Acquired schema lock
0202 02:52:30.613482 (+ 3us) tablet.cc:660] Decoding operations
0202 02:52:30.613563 (+ 81us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:30.613571 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:30.613575 (+ 4us) tablet.cc:683] Acquiring locks for 2 operations
0202 02:52:30.613658 (+ 83us) tablet.cc:699] Row locks acquired
0202 02:52:30.613660 (+ 2us) write_op.cc:260] PREPARE: finished
0202 02:52:30.613698 (+ 38us) write_op.cc:270] Start()
0202 02:52:30.613723 (+ 25us) write_op.cc:276] Timestamp: P: 1770000750613636 usec, L: 0
0202 02:52:30.613726 (+ 3us) op_driver.cc:348] REPLICATION: starting
0202 02:52:30.613857 (+ 131us) log.cc:844] Serialized 731 byte log entry
0202 02:52:36.014037 (+5400180us) op_driver.cc:464] REPLICATION: finished
0202 02:52:36.014774 (+ 737us) write_op.cc:301] APPLY: starting
0202 02:52:36.014785 (+ 11us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:36.014819 (+ 34us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:36.014848 (+ 29us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:36.014911 (+ 63us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:36.014938 (+ 27us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0202 02:52:36.014945 (+ 7us) write_op.cc:312] APPLY: finished
0202 02:52:36.014996 (+ 51us) log.cc:844] Serialized 29 byte log entry
0202 02:52:36.015057 (+ 61us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:36.015070 (+ 13us) write_op.cc:454] Released schema lock
0202 02:52:36.015075 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":101,"spinlock_wait_cycles":99340032,"child_traces":[["op",{"apply.queue_time_us":705,"apply.run_cpu_time_us":352,"apply.run_wall_time_us":352,"num_ops":2,"prepare.queue_time_us":58207,"prepare.run_cpu_time_us":506,"prepare.run_wall_time_us":504,"replication_time_us":5400294}]]}
I20260202 02:52:45.688468 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_46283f6baa2d4a9d9fdb8c1e501bcf89"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:45.684249 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79c73f4a92044bafb5b8384229cb3a4d" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:45.688916 1063 raft_consensus.cc:2468] T 79c73f4a92044bafb5b8384229cb3a4d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
W20260202 02:52:45.689088 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_46283f6baa2d4a9d9fdb8c1e501bcf89 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:45.684482 1058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a0b5d82e5fb94b60b4aa324b978a5378" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:45.689201 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48c820d845e144249c94cd87a0f38ec3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:45.684651 1061 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6b8f924aecf44e959e47d2f32e6b0bf5" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:45.689402 1061 raft_consensus.cc:2468] T 6b8f924aecf44e959e47d2f32e6b0bf5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
I20260202 02:52:45.689553 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0da286b7f598457ca4d0254a7073cb74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:33.969141 2448 raft_consensus.cc:493] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:45.690250 2448 raft_consensus.cc:515] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:45.690472 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39eaa2848cf4450dbe563ddd3b11df06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:45.334021 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c584f3163f1b45789492ccd27b7ad609. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:45.691097 2448 leader_election.cc:290] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:45.691258 2448 raft_consensus.cc:2804] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:45.691337 2448 raft_consensus.cc:697] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:45.691566 2448 consensus_queue.cc:237] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:45.692552 825 heartbeater.cc:344] Connected to a master server at 127.0.104.190:33981
I20260202 02:52:36.017992 523 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 75a24a1bf54d4c799938a5723ac6fa99
I20260202 02:52:45.695397 523 catalog_manager.cc:6724] ReplaceTablet: tablet 75a24a1bf54d4c799938a5723ac6fa99 deleted and replaced by tablet 6750a1212ceb4e6e82e4ffa93caab590
I20260202 02:52:45.695528 523 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:47430 (request call id 804) took 15451 ms. Trace:
I20260202 02:52:45.695914 767 raft_consensus.cc:3060] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:45.695639 523 rpcz_store.cc:276] 0202 02:52:30.243968 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.372856 (+128888us) service_pool.cc:225] Handling call
0202 02:52:45.695493 (+15322637us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:30.678146 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:30.678572 (+ 426us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:30.678581 (+ 9us) write_op.cc:435] Acquired schema lock
0202 02:52:30.678583 (+ 2us) tablet.cc:660] Decoding operations
0202 02:52:30.679012 (+ 429us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:30.679073 (+ 61us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:30.679080 (+ 7us) tablet.cc:683] Acquiring locks for 2 operations
0202 02:52:30.679451 (+ 371us) tablet.cc:699] Row locks acquired
0202 02:52:30.679454 (+ 3us) write_op.cc:260] PREPARE: finished
0202 02:52:30.679671 (+ 217us) write_op.cc:270] Start()
0202 02:52:30.679829 (+ 158us) write_op.cc:276] Timestamp: P: 1770000750679552 usec, L: 0
0202 02:52:30.679834 (+ 5us) op_driver.cc:348] REPLICATION: starting
0202 02:52:30.680913 (+ 1079us) log.cc:844] Serialized 628 byte log entry
0202 02:52:36.017253 (+5336340us) op_driver.cc:464] REPLICATION: finished
0202 02:52:36.017321 (+ 68us) write_op.cc:301] APPLY: starting
0202 02:52:36.017332 (+ 11us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:36.017373 (+ 41us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:36.017395 (+ 22us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:36.017493 (+ 98us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:36.017558 (+ 65us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0202 02:52:36.017565 (+ 7us) write_op.cc:312] APPLY: finished
0202 02:52:36.017615 (+ 50us) log.cc:844] Serialized 29 byte log entry
0202 02:52:36.017662 (+ 47us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:36.017675 (+ 13us) write_op.cc:454] Released schema lock
0202 02:52:36.017680 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":31,"apply.run_cpu_time_us":413,"apply.run_wall_time_us":413,"num_ops":2,"prepare.queue_time_us":114848,"prepare.run_cpu_time_us":1736,"prepare.run_wall_time_us":3077,"replication_time_us":5337398}]]}
I20260202 02:52:45.696215 1032 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49890 (request call id 702) took 15200 ms. Trace:
W20260202 02:52:35.764248 777 rpcz_store.cc:269] 0202 02:52:30.605015 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.605073 (+ 58us) service_pool.cc:225] Handling call
0202 02:52:35.764131 (+5159058us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:35.827077 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 494) took 10596 ms. Trace:
W20260202 02:52:45.526805 713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260202 02:52:45.696419 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_caf9a47c30ad4c6293b84e85cbde60eb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:45.696480 1180 rpcz_store.cc:276] 0202 02:52:25.230955 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:25.231039 (+ 84us) service_pool.cc:225] Handling call
0202 02:52:35.792271 (+10561232us) ts_tablet_manager.cc:1151] Deleting tablet 361c0c3832614ec78695c28720b841e0
0202 02:52:35.792278 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:35.827030 (+ 34752us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10561002}
I20260202 02:52:45.696906 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 495) took 20465 ms. Trace:
I20260202 02:52:45.696977 1180 rpcz_store.cc:276] 0202 02:52:25.231581 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:25.231629 (+ 48us) service_pool.cc:225] Handling call
0202 02:52:45.696763 (+20465134us) ts_tablet_manager.cc:1151] Deleting tablet 7f9dfac19c04475aadb45e95a6474eb4
0202 02:52:45.696774 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:45.696888 (+ 114us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":20464940}
W20260202 02:52:45.697055 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_caf9a47c30ad4c6293b84e85cbde60eb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:45.526901 2449 raft_consensus.cc:493] T 16b47a2831f54d809ed0b16deaec7e3d P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b1f05820b6034228bdad884cc348716d)
I20260202 02:52:45.697196 2449 raft_consensus.cc:515] T 16b47a2831f54d809ed0b16deaec7e3d P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:45.697304 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 496) took 20462 ms. Trace:
I20260202 02:52:45.697364 1180 rpcz_store.cc:276] 0202 02:52:25.234507 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:25.234565 (+ 58us) service_pool.cc:225] Handling call
0202 02:52:45.697170 (+20462605us) ts_tablet_manager.cc:1151] Deleting tablet a0b5d82e5fb94b60b4aa324b978a5378
0202 02:52:45.697178 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:45.697287 (+ 109us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":20462402}
I20260202 02:52:45.697649 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 497) took 20453 ms. Trace:
I20260202 02:52:45.697712 1180 rpcz_store.cc:276] 0202 02:52:25.244241 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:25.244305 (+ 64us) service_pool.cc:225] Handling call
0202 02:52:45.697547 (+20453242us) ts_tablet_manager.cc:1151] Deleting tablet ac2ab12f5040488aa306cb69b107b485
0202 02:52:45.697556 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:45.697629 (+ 73us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":20452971}
I20260202 02:52:45.697844 2449 leader_election.cc:290] T 16b47a2831f54d809ed0b16deaec7e3d P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:45.697937 1180 tablet_replica.cc:333] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:45.698118 1180 raft_consensus.cc:2243] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:34.691802 2450 raft_consensus.cc:493] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:45.698153 2449 consensus_queue.cc:1048] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:45.698284 1180 raft_consensus.cc:2272] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:45.336184 520 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:47430 (request call id 809) took 15091 ms. Trace:
I20260202 02:52:45.698841 1180 ts_tablet_manager.cc:1916] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:45.698530 520 rpcz_store.cc:276] 0202 02:52:30.261368 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.557936 (+296568us) service_pool.cc:225] Handling call
0202 02:52:30.558379 (+ 443us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0202 02:52:30.558497 (+ 118us) catalog_manager.cc:2815] Modifying in-memory table state
0202 02:52:30.558560 (+ 63us) catalog_manager.cc:2824] Locking tablets
0202 02:52:30.558714 (+ 154us) catalog_manager.cc:2838] Removing table and tablets from system table
0202 02:52:36.016003 (+5457289us) catalog_manager.cc:2856] Removing table from by-name map
0202 02:52:36.016109 (+ 106us) catalog_manager.cc:2872] Committing in-memory state
0202 02:52:36.016143 (+ 34us) catalog_manager.cc:2876] Aborting table tasks
0202 02:52:45.336120 (+9319977us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:30.613939 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:30.614019 (+ 80us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:30.614023 (+ 4us) write_op.cc:435] Acquired schema lock
0202 02:52:30.614025 (+ 2us) tablet.cc:660] Decoding operations
0202 02:52:30.614105 (+ 80us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:30.614112 (+ 7us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:30.614116 (+ 4us) tablet.cc:683] Acquiring locks for 2 operations
0202 02:52:30.614200 (+ 84us) tablet.cc:699] Row locks acquired
0202 02:52:30.614202 (+ 2us) write_op.cc:260] PREPARE: finished
0202 02:52:30.614230 (+ 28us) write_op.cc:270] Start()
0202 02:52:30.614253 (+ 23us) write_op.cc:276] Timestamp: P: 1770000750614179 usec, L: 0
0202 02:52:30.614256 (+ 3us) op_driver.cc:348] REPLICATION: starting
0202 02:52:30.614394 (+ 138us) log.cc:844] Serialized 731 byte log entry
0202 02:52:36.015541 (+5401147us) op_driver.cc:464] REPLICATION: finished
0202 02:52:36.015622 (+ 81us) write_op.cc:301] APPLY: starting
0202 02:52:36.015633 (+ 11us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:36.015665 (+ 32us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:36.015688 (+ 23us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:36.015741 (+ 53us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:36.015767 (+ 26us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0202 02:52:36.015774 (+ 7us) write_op.cc:312] APPLY: finished
0202 02:52:36.015826 (+ 52us) log.cc:844] Serialized 29 byte log entry
0202 02:52:36.015870 (+ 44us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:36.015884 (+ 14us) write_op.cc:454] Released schema lock
0202 02:52:36.015889 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":37,"apply.run_cpu_time_us":328,"apply.run_wall_time_us":328,"num_ops":2,"prepare.queue_time_us":54565,"prepare.run_cpu_time_us":503,"prepare.run_wall_time_us":502,"replication_time_us":5401261}]]}
I20260202 02:52:45.699268 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_ff96a2dc345940448683f8be23859993"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:45.699661 2449 raft_consensus.cc:493] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:45.699775 2449 raft_consensus.cc:515] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
W20260202 02:52:45.699829 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ff96a2dc345940448683f8be23859993 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:45.527498 762 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28a900bcaa8b41718a285e595886d805" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:45.527166 1182 raft_consensus.cc:2243] T 474df41e4fe24a818fa45a4ce84d5474 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:45.700312 2449 leader_election.cc:290] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:36.018581 522 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet b8f2b34b4f1e4ff2b2be4ff40aeb0c66
I20260202 02:52:45.527729 763 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "80eb3013f5354022a2dd78bb294a8497" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:45.528276 765 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "78f122342dd144b48de89902faa0f94f" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:45.528304 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26b890ed87c3471b9fc0b172c90e0a11" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:45.700937 522 catalog_manager.cc:6724] ReplaceTablet: tablet b8f2b34b4f1e4ff2b2be4ff40aeb0c66 deleted and replaced by tablet 912ac4044b5645caa7eda2a565c5b8bb
I20260202 02:52:45.700965 772 raft_consensus.cc:2468] T 26b890ed87c3471b9fc0b172c90e0a11 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:45.701045 522 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:47430 (request call id 811) took 15405 ms. Trace:
I20260202 02:52:45.528853 761 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "02b434fa058b45529ff3938b053263c6" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:45.701169 761 raft_consensus.cc:2468] T 02b434fa058b45529ff3938b053263c6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:45.701421 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8e489f59d1345549e29c4fa6e59c1c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:45.701134 522 rpcz_store.cc:276] 0202 02:52:30.295153 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.564039 (+268886us) service_pool.cc:225] Handling call
0202 02:52:45.701017 (+15136978us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:30.681433 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:30.681894 (+ 461us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:30.681967 (+ 73us) write_op.cc:435] Acquired schema lock
0202 02:52:30.681970 (+ 3us) tablet.cc:660] Decoding operations
0202 02:52:30.682478 (+ 508us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:30.682491 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:30.682696 (+ 205us) tablet.cc:683] Acquiring locks for 2 operations
0202 02:52:30.683152 (+ 456us) tablet.cc:699] Row locks acquired
0202 02:52:30.683156 (+ 4us) write_op.cc:260] PREPARE: finished
0202 02:52:30.683316 (+ 160us) write_op.cc:270] Start()
0202 02:52:30.706000 (+ 22684us) write_op.cc:276] Timestamp: P: 1770000750683248 usec, L: 0
0202 02:52:30.706010 (+ 10us) op_driver.cc:348] REPLICATION: starting
0202 02:52:30.707180 (+ 1170us) log.cc:844] Serialized 628 byte log entry
0202 02:52:36.017975 (+5310795us) op_driver.cc:464] REPLICATION: finished
0202 02:52:36.018045 (+ 70us) write_op.cc:301] APPLY: starting
0202 02:52:36.018056 (+ 11us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:36.018084 (+ 28us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:36.018113 (+ 29us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:36.018191 (+ 78us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:36.018233 (+ 42us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0202 02:52:36.018241 (+ 8us) write_op.cc:312] APPLY: finished
0202 02:52:36.018290 (+ 49us) log.cc:844] Serialized 29 byte log entry
0202 02:52:36.018336 (+ 46us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:36.018348 (+ 12us) write_op.cc:454] Released schema lock
0202 02:52:36.018354 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":30,"apply.run_cpu_time_us":366,"apply.run_wall_time_us":365,"num_ops":2,"prepare.queue_time_us":116269,"prepare.run_cpu_time_us":1953,"prepare.run_wall_time_us":25981,"replication_time_us":5311943}]]}
I20260202 02:52:45.702232 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae08e323fb9a4011ac313faf4ca70090. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:45.701802 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_06dc35989d444cccb418b9d2db62721d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:45.702980 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_06dc35989d444cccb418b9d2db62721d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:45.703171 767 raft_consensus.cc:2468] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
W20260202 02:52:45.703562 767 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.131:34317 (request call id 247) took 9494 ms (client timeout 4407 ms). Trace:
W20260202 02:52:45.703635 767 rpcz_store.cc:269] 0202 02:52:36.208837 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:36.208942 (+ 105us) service_pool.cc:225] Handling call
0202 02:52:45.703501 (+9494559us) spinlock_profiling.cc:243] Waited 21.4 ms on lock 0x616000210454. stack: 00007f1e37b5aa86 00007f1e37b5a840 00007f1e36b353ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f1e43a3889b 00007f1e478a8694 00007f1e41a3e806 00007f1e41a3e535 00007f1e3eb1d55e 00007f1e3eb1c43e 00007f1e3eb22675 00007f1e3eb22f9f 00007f1e3eb22dc8 0000000000a8970c
0202 02:52:45.703539 (+ 38us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":49213312}
I20260202 02:52:45.706188 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91a604fb877a48479914a5b60144a22d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:45.677233 2438 leader_election.cc:290] T 73f8db8f30f14d369d2227c9be7c4647 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:45.708457 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe97009dbcc64846844e381817148d7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:45.711547 753 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42950 (request call id 706) took 15166 ms. Trace:
I20260202 02:52:45.711887 494 master_runner.cc:186] Error getting master registration for 127.0.104.190:33981: OK, instance_id { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" instance_seqno: 1770000750063992 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260202 02:52:45.712121 494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:52:45.713111 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6466c63eb97a4a21b29f4f0871e8973c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:45.713836 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fed7eb10ec74f009b73fc5c66084851. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:45.714273 885 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41736 (request call id 697) took 15173 ms. Trace:
I20260202 02:52:45.704820 1037 tablet_service.cc:1552] Processing DeleteTablet for tablet 3dcf4f794ec14b2f9d181d015fc437fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:30 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:45.696312 1032 rpcz_store.cc:276] 0202 02:52:30.495322 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.684000 (+188678us) service_pool.cc:225] Handling call
0202 02:52:35.767578 (+5083578us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:35.767606 (+ 28us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:45.693384 (+9925778us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.695998 (+ 2614us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:45.696176 (+ 178us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:45.705600 998 leader_election.cc:304] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:45.712833 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 09a8aa4dd70b4fe2806293d2620e521c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:30 UTC) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:45.716672 998 leader_election.cc:343] T dada209d982d48f3b74a49905f19c5b6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:45.716056 1037 tablet_service.cc:1552] Processing DeleteTablet for tablet 75a24a1bf54d4c799938a5723ac6fa99 with delete_type TABLET_DATA_DELETED (replaced by tablet 6750a1212ceb4e6e82e4ffa93caab590) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:45.717082 998 leader_election.cc:343] T 33150e1456144b8e9c64cc2a2ac6dcca P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:45.717283 998 leader_election.cc:304] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:45.680269 2482 leader_election.cc:290] T 06b80a1c9251420cbc16fc95ae98aca9 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:45.717581 2482 raft_consensus.cc:2804] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260202 02:52:45.717649 2482 raft_consensus.cc:493] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:45.717710 2482 raft_consensus.cc:3060] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:45.718065 2433 raft_consensus.cc:2804] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:45.718138 2433 raft_consensus.cc:493] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:45.718194 2433 raft_consensus.cc:3060] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:45.721562 1173 tablet_bootstrap.cc:654] T b7880cb051c24bc998003f0effed3cf8 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:45.721805 998 leader_election.cc:304] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:45.722069 2434 raft_consensus.cc:2804] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260202 02:52:45.722141 2434 raft_consensus.cc:493] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
I20260202 02:52:45.722201 2434 raft_consensus.cc:3060] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:45.722537 998 leader_election.cc:304] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:45.722745 2427 raft_consensus.cc:2804] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:45.722802 2427 raft_consensus.cc:493] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:45.722859 2427 raft_consensus.cc:3060] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:45.723218 998 leader_election.cc:304] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:45.723457 2423 raft_consensus.cc:2804] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:45.723567 2423 raft_consensus.cc:697] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:45.723788 2423 consensus_queue.cc:237] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:45.725404 2433 raft_consensus.cc:515] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:45.726014 2433 leader_election.cc:290] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:45.728410 2434 raft_consensus.cc:515] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:45.729048 2434 leader_election.cc:290] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:45.733904 1031 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49890 (request call id 697) took 15272 ms. Trace:
I20260202 02:52:45.733947 2482 raft_consensus.cc:515] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:45.734005 1031 rpcz_store.cc:276] 0202 02:52:30.461570 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.461620 (+ 50us) service_pool.cc:225] Handling call
0202 02:52:30.593219 (+131599us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.593241 (+ 22us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:45.671376 (+15078135us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.733722 (+ 62346us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:45.733879 (+ 157us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:45.707779 1029 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49890 (request call id 695) took 15249 ms. Trace:
I20260202 02:52:45.734849 1029 rpcz_store.cc:276] 0202 02:52:30.458583 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.458632 (+ 49us) service_pool.cc:225] Handling call
0202 02:52:30.593060 (+134428us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.593084 (+ 24us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:45.671376 (+15078292us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.707562 (+ 36186us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:45.707738 (+ 176us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":10240}
I20260202 02:52:45.716056 1032 tablet_service.cc:1552] Processing DeleteTablet for tablet b8f2b34b4f1e4ff2b2be4ff40aeb0c66 with delete_type TABLET_DATA_DELETED (replaced by tablet 912ac4044b5645caa7eda2a565c5b8bb) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:45.734722 2482 leader_election.cc:290] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:45.736735 2427 raft_consensus.cc:515] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:45.737290 2427 leader_election.cc:290] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:45.737769 2372 log.cc:927] Time spent T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8: Append to log took a long time: real 0.109s user 0.002s sys 0.002s
I20260202 02:52:45.738957 1038 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49890 (request call id 688) took 15474 ms. Trace:
I20260202 02:52:45.739037 1038 rpcz_store.cc:276] 0202 02:52:30.264917 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.267678 (+ 2761us) service_pool.cc:225] Handling call
0202 02:52:30.592344 (+324666us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.592385 (+ 41us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:45.676091 (+15083706us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.738742 (+ 62651us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:45.738937 (+ 195us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:45.742179 1173 tablet_bootstrap.cc:492] T b7880cb051c24bc998003f0effed3cf8 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:45.742322 1173 ts_tablet_manager.cc:1403] T b7880cb051c24bc998003f0effed3cf8 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.411s user 0.006s sys 0.021s
I20260202 02:52:45.742589 890 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41736 (request call id 709) took 15137 ms. Trace:
I20260202 02:52:45.742691 890 rpcz_store.cc:276] 0202 02:52:30.605569 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.606283 (+ 714us) service_pool.cc:225] Handling call
0202 02:52:35.768392 (+5162109us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:35.768418 (+ 26us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:45.661830 (+9893412us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.742352 (+ 80522us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:45.742566 (+ 214us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:45.743070 1173 raft_consensus.cc:359] T b7880cb051c24bc998003f0effed3cf8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:45.743227 1173 raft_consensus.cc:385] T b7880cb051c24bc998003f0effed3cf8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:45.743266 1173 raft_consensus.cc:740] T b7880cb051c24bc998003f0effed3cf8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:45.743465 1173 consensus_queue.cc:260] T b7880cb051c24bc998003f0effed3cf8 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:45.744277 760 raft_consensus.cc:2393] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1f05820b6034228bdad884cc348716d in current term 1: Already voted for candidate 1f442ef85029461091de847118c34c0d in this term.
I20260202 02:52:45.744829 750 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42950 (request call id 697) took 15189 ms. Trace:
I20260202 02:52:45.744952 750 rpcz_store.cc:276] 0202 02:52:30.544361 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.556696 (+ 12335us) service_pool.cc:225] Handling call
0202 02:52:30.556977 (+ 281us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.556994 (+ 17us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:45.693828 (+15136834us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.733391 (+ 39563us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:45.733572 (+ 181us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:45.673673 902 rpcz_store.cc:276] 0202 02:52:30.541239 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.547159 (+ 5920us) service_pool.cc:225] Handling call
0202 02:52:30.593641 (+ 46482us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.593677 (+ 36us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:35.762896 (+5169219us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:35.871294 (+108398us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:35.871514 (+ 220us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:45.747871 897 tablet_service.cc:1552] Processing DeleteTablet for tablet 91a604fb877a48479914a5b60144a22d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:30 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:45.687881 2430 raft_consensus.cc:515] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:45.747967 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95b95947aad94b849303eb344d18fd27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:45.714360 885 rpcz_store.cc:276] 0202 02:52:30.540933 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.546497 (+ 5564us) service_pool.cc:225] Handling call
0202 02:52:30.594114 (+ 47617us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.594153 (+ 39us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:45.662596 (+15068443us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.714087 (+ 51491us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:45.714250 (+ 163us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1920}
I20260202 02:52:45.748152 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d06ebf3fe7fd4a59ad08d6d1ba3d85ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:45.748504 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b2b4f429b6a484ea6ef7c19124b236b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:45.745211 753 rpcz_store.cc:276] 0202 02:52:30.545158 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.565188 (+ 20030us) service_pool.cc:225] Handling call
0202 02:52:30.565521 (+ 333us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.565544 (+ 23us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:45.708884 (+15143340us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.711340 (+ 2456us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:45.711513 (+ 173us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:45.752831 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08fbd094d4df46cf82a694b4e1f30c6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:45.753428 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b2b4f429b6a484ea6ef7c19124b236b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:45.753937 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39eaa2848cf4450dbe563ddd3b11df06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:45.754271 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1dd3b956ae594e89b0599dbc9345f5c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:45.754614 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efe78242e0654a1c9bdbaf9cfa88b034. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:45.754868 894 tablet_service.cc:1505] Processing CreateTablet for tablet 1b2b4f429b6a484ea6ef7c19124b236b (DEFAULT_TABLE table=default.Table_2d42dcf81d2d4b2c90fe09860b9b2318 [id=e620930eae8e4707b93e7ab16369e13d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:45.698314 2450 raft_consensus.cc:515] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:45.748481 1175 raft_consensus.cc:359] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:45.755829 1175 raft_consensus.cc:385] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:45.755901 1175 raft_consensus.cc:740] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:45.756076 1175 consensus_queue.cc:260] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:45.756891 2443 raft_consensus.cc:493] T ac2ab12f5040488aa306cb69b107b485 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:45.756999 2443 raft_consensus.cc:515] T ac2ab12f5040488aa306cb69b107b485 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
W20260202 02:52:45.757088 859 connection.cc:232] RPC call timeout handler was delayed by -14.193123896999992s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260202 02:52:45.757302 2430 leader_election.cc:290] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:45.757328 859 connection.cc:232] RPC call timeout handler was delayed by -12.385408614999989s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30150000000000005s for a response
W20260202 02:52:45.757382 859 connection.cc:232] RPC call timeout handler was delayed by -12.253178147999989s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3206s for a response
I20260202 02:52:45.757478 2443 leader_election.cc:290] T ac2ab12f5040488aa306cb69b107b485 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:45.757627 2430 raft_consensus.cc:493] T d8418a9dd0944456b4d4820dde0cde02 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:45.758878 1055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.104.129:38747 (request call id 339) took 15219 ms. Trace:
I20260202 02:52:45.765256 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0da286b7f598457ca4d0254a7073cb74. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:45.900352 1389 leader_election.cc:336] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7dab2abef14445d688ce1335b1f9b2f8 (127.0.104.190:33981): Timed out: RequestConsensusVote RPC to 127.0.104.190:33981 timed out after 0.630s (SENT)
I20260202 02:52:47.407193 1389 leader_election.cc:304] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9c3e3612bb94fb7b94c308041acfaeb; no voters: 7dab2abef14445d688ce1335b1f9b2f8, 92a8191cf1e14e4d99f1c378c2d3c85b
I20260202 02:52:45.757722 2430 raft_consensus.cc:515] T d8418a9dd0944456b4d4820dde0cde02 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:45.989830 524 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:47430 (request call id 817) took 15523 ms. Trace:
W20260202 02:52:45.832692 714 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 175666us
I20260202 02:52:47.408406 2430 leader_election.cc:290] T d8418a9dd0944456b4d4820dde0cde02 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:45.990231 521 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:47430 (request call id 813) took 15524 ms. Trace:
I20260202 02:52:47.408061 524 rpcz_store.cc:276] 0202 02:52:30.466418 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.273315 (+14806897us) service_pool.cc:225] Handling call
0202 02:52:45.331413 (+ 58098us) catalog_manager.cc:2101] Acquired catalog manager lock
0202 02:52:45.331916 (+ 503us) catalog_manager.cc:2147] Created new table and tablet info
0202 02:52:45.989548 (+657632us) catalog_manager.cc:2207] Wrote table and tablets to system table
0202 02:52:45.989668 (+ 120us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0202 02:52:45.989762 (+ 94us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:45.350549 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:45.351673 (+ 1124us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:45.351681 (+ 8us) write_op.cc:435] Acquired schema lock
0202 02:52:45.351684 (+ 3us) tablet.cc:660] Decoding operations
0202 02:52:45.352796 (+ 1112us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:45.352992 (+ 196us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:45.352999 (+ 7us) tablet.cc:683] Acquiring locks for 2 operations
0202 02:52:45.353626 (+ 627us) tablet.cc:699] Row locks acquired
0202 02:52:45.353629 (+ 3us) write_op.cc:260] PREPARE: finished
0202 02:52:45.354036 (+ 407us) write_op.cc:270] Start()
0202 02:52:45.354260 (+ 224us) write_op.cc:276] Timestamp: P: 1770000765353816 usec, L: 0
0202 02:52:45.354265 (+ 5us) op_driver.cc:348] REPLICATION: starting
0202 02:52:45.356139 (+ 1874us) log.cc:844] Serialized 433 byte log entry
0202 02:52:45.988159 (+632020us) op_driver.cc:464] REPLICATION: finished
0202 02:52:45.988384 (+ 225us) write_op.cc:301] APPLY: starting
0202 02:52:45.988403 (+ 19us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:45.988454 (+ 51us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:45.988479 (+ 25us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:45.988611 (+ 132us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:45.988646 (+ 35us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0202 02:52:45.988652 (+ 6us) write_op.cc:312] APPLY: finished
0202 02:52:45.988724 (+ 72us) log.cc:844] Serialized 29 byte log entry
0202 02:52:45.988786 (+ 62us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:45.988807 (+ 21us) write_op.cc:454] Released schema lock
0202 02:52:45.988813 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0202 02:52:45.988884 (+ 71us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:45.988888 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":126,"apply.run_cpu_time_us":500,"apply.run_wall_time_us":502,"num_ops":2,"prepare.queue_time_us":17205,"prepare.run_cpu_time_us":3538,"prepare.run_wall_time_us":23091,"replication_time_us":633857,"thread_start_us":32924,"threads_started":2,"wal-append.queue_time_us":17612}]]}
I20260202 02:52:45.833479 1180 ts_tablet_manager.cc:1929] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:47.408691 1180 log.cc:1199] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/94bada52db6a47179415d302b541d01f
W20260202 02:52:47.408895 524 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.104.131:43527 (request call id 111) took 1703 ms (client timeout 1000 ms). Trace:
I20260202 02:52:47.409003 1180 ts_tablet_manager.cc:1950] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:47.408622 521 rpcz_store.cc:276] 0202 02:52:30.465816 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.526342 (+15060526us) service_pool.cc:225] Handling call
0202 02:52:45.657122 (+130780us) catalog_manager.cc:2101] Acquired catalog manager lock
0202 02:52:45.657634 (+ 512us) catalog_manager.cc:2147] Created new table and tablet info
0202 02:52:45.990039 (+332405us) catalog_manager.cc:2207] Wrote table and tablets to system table
0202 02:52:45.990120 (+ 81us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0202 02:52:45.990190 (+ 70us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:45.668169 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:45.668270 (+ 101us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:45.668277 (+ 7us) write_op.cc:435] Acquired schema lock
0202 02:52:45.668279 (+ 2us) tablet.cc:660] Decoding operations
0202 02:52:45.678284 (+ 10005us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:45.678297 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:45.678302 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
0202 02:52:45.678385 (+ 83us) tablet.cc:699] Row locks acquired
0202 02:52:45.678388 (+ 3us) write_op.cc:260] PREPARE: finished
0202 02:52:45.678426 (+ 38us) write_op.cc:270] Start()
0202 02:52:45.678462 (+ 36us) write_op.cc:276] Timestamp: P: 1770000765678396 usec, L: 0
0202 02:52:45.678465 (+ 3us) op_driver.cc:348] REPLICATION: starting
0202 02:52:45.678668 (+ 203us) log.cc:844] Serialized 433 byte log entry
0202 02:52:45.988240 (+309572us) op_driver.cc:464] REPLICATION: finished
0202 02:52:45.988950 (+ 710us) write_op.cc:301] APPLY: starting
0202 02:52:45.988959 (+ 9us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:45.988992 (+ 33us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:45.989031 (+ 39us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:45.989111 (+ 80us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:45.989136 (+ 25us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0202 02:52:45.989143 (+ 7us) write_op.cc:312] APPLY: finished
0202 02:52:45.989192 (+ 49us) log.cc:844] Serialized 29 byte log entry
0202 02:52:45.989229 (+ 37us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:45.989242 (+ 13us) write_op.cc:454] Released schema lock
0202 02:52:45.989247 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":681,"apply.run_cpu_time_us":408,"apply.run_wall_time_us":1477,"num_ops":2,"prepare.queue_time_us":9670,"prepare.run_cpu_time_us":729,"prepare.run_wall_time_us":10584,"replication_time_us":309758,"thread_start_us":18800,"threads_started":1}]]}
W20260202 02:52:47.408999 524 rpcz_store.cc:269] 0202 02:52:45.705322 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:47.408795 (+1703473us) service_pool.cc:212] Skipping call since client already timed out
0202 02:52:47.408841 (+ 46us) inbound_call.cc:173] Queueing failure response
Metrics: {}
I20260202 02:52:47.409309 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 499) took 22151 ms. Trace:
I20260202 02:52:47.409291 521 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_cbbe5e139f7e4e058d9e05816e811c98" } modify_external_catalogs: true
W20260202 02:52:46.671609 1313 kernel_stack_watchdog.cc:198] Thread 1389 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 185ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f5913bb7980 (unknown) at ??:0
@ 0x7f590c6ca255 __write at ??:0
@ 0x7f590c64528d _IO_file_write at ??:0
@ 0x7f590c645bff _IO_file_xsputn at ??:0
@ 0x7f590c639a57 fwrite at ??:0
@ 0x8a8c97 __interceptor_fwrite.part.56 at ??:?
@ 0x7f590d82e8a0 google::LogMessage::SendToLog() at ??:0
@ 0x7f590d82a7cc google::LogMessage::Flush() at ??:0
@ 0x7f590d82aa11 google::LogMessage::~LogMessage() at ??:0
@ 0x7f591c3b8676 kudu::consensus::LeaderElection::VoteResponseRpcCallback() at ??:0
@ 0x7f591c3bb07a kudu::consensus::LeaderElection::Run()::$_0::operator()() at ??:0
@ 0x7f591c3bae3e std::_Function_handler<>::_M_invoke() at ??:0
@ 0xa8970d std::function<>::operator()() at ??:?
@ 0x7f5917497f10 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f5917499602 kudu::rpc::OutboundCall::SetTimedOut() at ??:0
@ 0x7f591742f453 kudu::rpc::Connection::HandleOutboundCallTimeout() at ??:0
I20260202 02:52:47.409380 1180 rpcz_store.cc:276] 0202 02:52:25.257651 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:25.257696 (+ 45us) service_pool.cc:225] Handling call
0202 02:52:45.697898 (+20440202us) ts_tablet_manager.cc:1151] Deleting tablet 94bada52db6a47179415d302b541d01f
0202 02:52:45.697906 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:45.715166 (+ 17260us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.833369 (+118203us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:47.409292 (+1575923us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":20440001}
I20260202 02:52:47.409516 521 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_cbbe5e139f7e4e058d9e05816e811c98" } modify_external_catalogs: true
I20260202 02:52:47.409801 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 500) took 22120 ms. Trace:
I20260202 02:52:47.409863 1180 rpcz_store.cc:276] 0202 02:52:25.288887 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:25.289302 (+ 415us) service_pool.cc:225] Handling call
0202 02:52:47.409675 (+22120373us) ts_tablet_manager.cc:1151] Deleting tablet 3706562a22634eb6b72e36bb6a2d1ae5
0202 02:52:47.409683 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:47.409777 (+ 94us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":22119969}
I20260202 02:52:46.009539 525 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:47430 (request call id 814) took 15543 ms. Trace:
I20260202 02:52:47.410171 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 502) took 22056 ms. Trace:
I20260202 02:52:47.410230 1180 rpcz_store.cc:276] 0202 02:52:25.353858 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:25.353910 (+ 52us) service_pool.cc:225] Handling call
0202 02:52:47.410073 (+22056163us) ts_tablet_manager.cc:1151] Deleting tablet 474df41e4fe24a818fa45a4ce84d5474
0202 02:52:47.410083 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:47.410156 (+ 73us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":22046627}
I20260202 02:52:47.410501 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 501) took 22056 ms. Trace:
I20260202 02:52:47.410184 525 rpcz_store.cc:276] 0202 02:52:30.465907 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.666108 (+15200201us) service_pool.cc:225] Handling call
0202 02:52:45.667028 (+ 920us) catalog_manager.cc:2101] Acquired catalog manager lock
0202 02:52:45.667520 (+ 492us) catalog_manager.cc:2147] Created new table and tablet info
0202 02:52:46.008812 (+341292us) catalog_manager.cc:2207] Wrote table and tablets to system table
0202 02:52:46.008944 (+ 132us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0202 02:52:46.009054 (+ 110us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:45.678790 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:45.678891 (+ 101us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:45.678896 (+ 5us) write_op.cc:435] Acquired schema lock
0202 02:52:45.678898 (+ 2us) tablet.cc:660] Decoding operations
0202 02:52:45.678972 (+ 74us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:45.678981 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:45.678985 (+ 4us) tablet.cc:683] Acquiring locks for 2 operations
0202 02:52:45.679045 (+ 60us) tablet.cc:699] Row locks acquired
0202 02:52:45.679047 (+ 2us) write_op.cc:260] PREPARE: finished
0202 02:52:45.679074 (+ 27us) write_op.cc:270] Start()
0202 02:52:45.679100 (+ 26us) write_op.cc:276] Timestamp: P: 1770000765679047 usec, L: 0
0202 02:52:45.679103 (+ 3us) op_driver.cc:348] REPLICATION: starting
0202 02:52:45.679246 (+ 143us) log.cc:844] Serialized 433 byte log entry
0202 02:52:46.007209 (+327963us) op_driver.cc:464] REPLICATION: finished
0202 02:52:46.007405 (+ 196us) write_op.cc:301] APPLY: starting
0202 02:52:46.007426 (+ 21us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:46.007480 (+ 54us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:46.007524 (+ 44us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:46.007661 (+ 137us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:46.007701 (+ 40us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0202 02:52:46.007708 (+ 7us) write_op.cc:312] APPLY: finished
0202 02:52:46.007805 (+ 97us) log.cc:844] Serialized 29 byte log entry
0202 02:52:46.007880 (+ 75us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:46.007900 (+ 20us) write_op.cc:454] Released schema lock
0202 02:52:46.007907 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0202 02:52:46.007983 (+ 76us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:46.007987 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":97,"apply.run_cpu_time_us":579,"apply.run_wall_time_us":580,"num_ops":2,"prepare.queue_time_us":10775,"prepare.run_cpu_time_us":507,"prepare.run_wall_time_us":505,"replication_time_us":328075}]]}
I20260202 02:52:47.410562 1180 rpcz_store.cc:276] 0202 02:52:25.353742 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:25.356064 (+ 2322us) service_pool.cc:225] Handling call
0202 02:52:47.410414 (+22054350us) ts_tablet_manager.cc:1151] Deleting tablet 7921daa000f54c8dbed04473e21cf0a5
0202 02:52:47.410422 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:47.410488 (+ 66us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":22035585}
I20260202 02:52:47.410840 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 506) took 21789 ms. Trace:
I20260202 02:52:47.410894 1180 rpcz_store.cc:276] 0202 02:52:25.621500 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:25.622346 (+ 846us) service_pool.cc:225] Handling call
0202 02:52:47.410740 (+21788394us) ts_tablet_manager.cc:1151] Deleting tablet 93f958924885457ab15f2aac5509e0aa
0202 02:52:47.410747 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:47.410827 (+ 80us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":21750050}
I20260202 02:52:46.009361 519 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:47430 (request call id 815) took 15543 ms. Trace:
I20260202 02:52:47.410913 525 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ff54266e8b5a4334ad1fbeda5270f0f2" } modify_external_catalogs: true
I20260202 02:52:47.411083 525 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ff54266e8b5a4334ad1fbeda5270f0f2" } modify_external_catalogs: true
I20260202 02:52:47.411118 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 508) took 21707 ms. Trace:
I20260202 02:52:47.411171 1180 rpcz_store.cc:276] 0202 02:52:25.703593 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:25.703645 (+ 52us) service_pool.cc:225] Handling call
0202 02:52:47.411057 (+21707412us) ts_tablet_manager.cc:1151] Deleting tablet f93e8d3c4faf48eaa98c3cbf88d585e2
0202 02:52:47.411064 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:47.411107 (+ 43us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":21707123}
I20260202 02:52:47.411396 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 509) took 21510 ms. Trace:
I20260202 02:52:47.411048 519 rpcz_store.cc:276] 0202 02:52:30.465988 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.676695 (+15210707us) service_pool.cc:225] Handling call
0202 02:52:45.677074 (+ 379us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0202 02:52:45.677398 (+ 324us) catalog_manager.cc:3708] Apply alter schema
0202 02:52:45.677596 (+ 198us) catalog_manager.cc:3740] Acquired catalog manager lock
0202 02:52:45.677690 (+ 94us) catalog_manager.cc:3898] Updating metadata on disk
0202 02:52:46.008414 (+330724us) catalog_manager.cc:3929] Committing alterations to in-memory state
0202 02:52:46.009039 (+ 625us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:45.679331 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:45.679431 (+ 100us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:45.679436 (+ 5us) write_op.cc:435] Acquired schema lock
0202 02:52:45.679438 (+ 2us) tablet.cc:660] Decoding operations
0202 02:52:45.679536 (+ 98us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:45.679554 (+ 18us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:45.679559 (+ 5us) tablet.cc:683] Acquiring locks for 1 operations
0202 02:52:45.679612 (+ 53us) tablet.cc:699] Row locks acquired
0202 02:52:45.679614 (+ 2us) write_op.cc:260] PREPARE: finished
0202 02:52:45.679641 (+ 27us) write_op.cc:270] Start()
0202 02:52:45.679666 (+ 25us) write_op.cc:276] Timestamp: P: 1770000765679613 usec, L: 0
0202 02:52:45.679669 (+ 3us) op_driver.cc:348] REPLICATION: starting
0202 02:52:45.679810 (+ 141us) log.cc:844] Serialized 405 byte log entry
0202 02:52:46.007282 (+327472us) op_driver.cc:464] REPLICATION: finished
0202 02:52:46.007516 (+ 234us) write_op.cc:301] APPLY: starting
0202 02:52:46.007532 (+ 16us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:46.007583 (+ 51us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:46.007644 (+ 61us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:46.007731 (+ 87us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:46.007761 (+ 30us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0202 02:52:46.007769 (+ 8us) write_op.cc:312] APPLY: finished
0202 02:52:46.007888 (+ 119us) log.cc:844] Serialized 23 byte log entry
0202 02:52:46.007939 (+ 51us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:46.007955 (+ 16us) write_op.cc:454] Released schema lock
0202 02:52:46.007961 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0202 02:52:46.009132 (+ 1171us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:46.009138 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":175,"apply.run_cpu_time_us":515,"apply.run_wall_time_us":1460,"num_ops":1,"prepare.queue_time_us":1147,"prepare.run_cpu_time_us":525,"prepare.run_wall_time_us":524,"replication_time_us":327598}]]}
I20260202 02:52:47.411445 1180 rpcz_store.cc:276] 0202 02:52:25.901405 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:25.901467 (+ 62us) service_pool.cc:225] Handling call
0202 02:52:47.411321 (+21509854us) ts_tablet_manager.cc:1151] Deleting tablet 47e50559a7c04ae080fcc67342c631c9
0202 02:52:47.411328 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:47.411386 (+ 58us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":21509630}
I20260202 02:52:47.411679 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 516) took 21281 ms. Trace:
I20260202 02:52:47.411744 1180 rpcz_store.cc:276] 0202 02:52:26.130406 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.130617 (+ 211us) service_pool.cc:225] Handling call
0202 02:52:47.411605 (+21280988us) ts_tablet_manager.cc:1151] Deleting tablet 5e20842de8cb4355b732685605c88834
0202 02:52:47.411612 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:47.411668 (+ 56us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":21280742}
I20260202 02:52:47.411823 519 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_c6515f88263042feadae47be333ed860" } new_table_name: "default.Table_cdcc140c2b8d4aabae614d58873dc22a" modify_external_catalogs: true
I20260202 02:52:47.412005 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 517) took 21281 ms. Trace:
I20260202 02:52:47.412062 1180 rpcz_store.cc:276] 0202 02:52:26.130570 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.130919 (+ 349us) service_pool.cc:225] Handling call
0202 02:52:47.411925 (+21281006us) ts_tablet_manager.cc:1151] Deleting tablet 0ddc726efd3f49489092919c9c6e01fb
0202 02:52:47.411932 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:47.411993 (+ 61us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":21280821}
W20260202 02:52:46.009591 527 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5416eab7a0914ecb855fbccde4894285 consensus state has no leader
I20260202 02:52:45.758983 1055 rpcz_store.cc:276] 0202 02:52:30.539404 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.539935 (+ 531us) service_pool.cc:225] Handling call
0202 02:52:45.758759 (+15218824us) spinlock_profiling.cc:243] Waited 15.2 s on lock 0x6180000c388c. stack: 00007fec53205a86 00007fec53205840 00007fec521e03ad 0000000000984f94 0000000000984f5d 000000000098492a 00007fec6064fb2c 00007fec62f51ecd 00007fec62f8b3ff 00007fec62f8b2f5 00007fec62f4f296 00007fec62f4e151 00007fec5d0e7b46 00007fec5d0e7875 00007fec5a1c855e 00007fec5a1c743e
0202 02:52:45.758849 (+ 90us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":34998859520}
I20260202 02:52:45.840719 754 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42950 (request call id 702) took 15295 ms. Trace:
I20260202 02:52:47.412349 527 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:47430 (request call id 816) took 16946 ms. Trace:
I20260202 02:52:47.412387 754 rpcz_store.cc:276] 0202 02:52:30.544779 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.560396 (+ 15617us) service_pool.cc:225] Handling call
0202 02:52:30.566908 (+ 6512us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.566924 (+ 16us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:45.705536 (+15138612us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.840467 (+134931us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:45.840691 (+ 224us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:46.011765 523 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:47430 (request call id 819) took 15510 ms. Trace:
I20260202 02:52:46.012578 526 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:47430 (request call id 818) took 15519 ms. Trace:
I20260202 02:52:47.412648 1055 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "63a9029bc39c4fbd8ac4bf4bc37d6b4e" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:45.852166 2448 raft_consensus.cc:493] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b1f05820b6034228bdad884cc348716d)
I20260202 02:52:47.412851 2448 raft_consensus.cc:515] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:47.413048 1055 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "474df41e4fe24a818fa45a4ce84d5474" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:47.412740 526 rpcz_store.cc:276] 0202 02:52:30.493286 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.688225 (+15194939us) service_pool.cc:225] Handling call
0202 02:52:45.689180 (+ 955us) catalog_manager.cc:2101] Acquired catalog manager lock
0202 02:52:45.689650 (+ 470us) catalog_manager.cc:2147] Created new table and tablet info
0202 02:52:46.012317 (+322667us) catalog_manager.cc:2207] Wrote table and tablets to system table
0202 02:52:46.012424 (+ 107us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0202 02:52:46.012505 (+ 81us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:45.714959 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:45.715051 (+ 92us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:45.715056 (+ 5us) write_op.cc:435] Acquired schema lock
0202 02:52:45.715059 (+ 3us) tablet.cc:660] Decoding operations
0202 02:52:45.715134 (+ 75us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:45.715144 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:45.715149 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
0202 02:52:45.715215 (+ 66us) tablet.cc:699] Row locks acquired
0202 02:52:45.715218 (+ 3us) write_op.cc:260] PREPARE: finished
0202 02:52:45.715253 (+ 35us) write_op.cc:270] Start()
0202 02:52:45.715281 (+ 28us) write_op.cc:276] Timestamp: P: 1770000765715219 usec, L: 0
0202 02:52:45.715284 (+ 3us) op_driver.cc:348] REPLICATION: starting
0202 02:52:45.715435 (+ 151us) log.cc:844] Serialized 433 byte log entry
0202 02:52:46.010649 (+295214us) op_driver.cc:464] REPLICATION: finished
0202 02:52:46.010907 (+ 258us) write_op.cc:301] APPLY: starting
0202 02:52:46.010926 (+ 19us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:46.010975 (+ 49us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:46.011014 (+ 39us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:46.011133 (+ 119us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:46.011168 (+ 35us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0202 02:52:46.011175 (+ 7us) write_op.cc:312] APPLY: finished
0202 02:52:46.011256 (+ 81us) log.cc:844] Serialized 29 byte log entry
0202 02:52:46.011318 (+ 62us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:46.011336 (+ 18us) write_op.cc:454] Released schema lock
0202 02:52:46.011342 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0202 02:52:46.011409 (+ 67us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:46.011413 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":153,"apply.run_cpu_time_us":502,"apply.run_wall_time_us":502,"num_ops":2,"prepare.queue_time_us":24825,"prepare.run_cpu_time_us":533,"prepare.run_wall_time_us":532,"replication_time_us":295333}]]}
I20260202 02:52:45.757795 2443 raft_consensus.cc:493] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:47.413358 2443 raft_consensus.cc:515] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:47.413470 2448 leader_election.cc:290] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:47.413414 526 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_d0e51ff8e83f4da19ce3c119dbec1ea7" } new_table_name: "default.Table_48307218be9d434798a0d354f3590805" modify_external_catalogs: true
I20260202 02:52:46.013135 520 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:47430 (request call id 820) took 15481 ms. Trace:
I20260202 02:52:47.413949 2443 leader_election.cc:290] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:45.764906 1882 ts_tablet_manager.cc:1929] T 13cd1383348b458894fdecc22bf24b74 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:46.012866 522 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:47430 (request call id 821) took 15373 ms. Trace:
I20260202 02:52:47.414155 1882 log.cc:1199] T 13cd1383348b458894fdecc22bf24b74 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/13cd1383348b458894fdecc22bf24b74
I20260202 02:52:47.414479 1882 ts_tablet_manager.cc:1950] T 13cd1383348b458894fdecc22bf24b74 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:47.414222 522 rpcz_store.cc:276] 0202 02:52:30.639694 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.701609 (+15061915us) service_pool.cc:225] Handling call
0202 02:52:45.703074 (+ 1465us) catalog_manager.cc:2101] Acquired catalog manager lock
0202 02:52:45.703528 (+ 454us) catalog_manager.cc:2147] Created new table and tablet info
0202 02:52:46.012687 (+309159us) catalog_manager.cc:2207] Wrote table and tablets to system table
0202 02:52:46.012768 (+ 81us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0202 02:52:46.012834 (+ 66us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:45.726856 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:45.726993 (+ 137us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:45.726999 (+ 6us) write_op.cc:435] Acquired schema lock
0202 02:52:45.727002 (+ 3us) tablet.cc:660] Decoding operations
0202 02:52:45.727095 (+ 93us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:45.727106 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:45.727111 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
0202 02:52:45.727185 (+ 74us) tablet.cc:699] Row locks acquired
0202 02:52:45.727187 (+ 2us) write_op.cc:260] PREPARE: finished
0202 02:52:45.727218 (+ 31us) write_op.cc:270] Start()
0202 02:52:45.727242 (+ 24us) write_op.cc:276] Timestamp: P: 1770000765727184 usec, L: 0
0202 02:52:45.727245 (+ 3us) op_driver.cc:348] REPLICATION: starting
0202 02:52:45.727381 (+ 136us) log.cc:844] Serialized 433 byte log entry
0202 02:52:46.010765 (+283384us) op_driver.cc:464] REPLICATION: finished
0202 02:52:46.011475 (+ 710us) write_op.cc:301] APPLY: starting
0202 02:52:46.011485 (+ 10us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:46.011515 (+ 30us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:46.011545 (+ 30us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:46.011654 (+ 109us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:46.011679 (+ 25us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0202 02:52:46.011686 (+ 7us) write_op.cc:312] APPLY: finished
0202 02:52:46.011737 (+ 51us) log.cc:844] Serialized 29 byte log entry
0202 02:52:46.011783 (+ 46us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:46.011797 (+ 14us) write_op.cc:454] Released schema lock
0202 02:52:46.011802 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0202 02:52:46.011863 (+ 61us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:46.011867 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":680,"apply.run_cpu_time_us":378,"apply.run_wall_time_us":379,"num_ops":2,"prepare.queue_time_us":22541,"prepare.run_cpu_time_us":576,"prepare.run_wall_time_us":575,"replication_time_us":283507}]]}
I20260202 02:52:47.414776 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 543) took 20450 ms. Trace:
I20260202 02:52:47.414844 1882 rpcz_store.cc:276] 0202 02:52:26.964013 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.966107 (+ 2094us) service_pool.cc:225] Handling call
0202 02:52:29.926215 (+2960108us) ts_tablet_manager.cc:1151] Deleting tablet 13cd1383348b458894fdecc22bf24b74
0202 02:52:29.926229 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:30.284394 (+358165us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.764751 (+15480357us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:47.414761 (+1650010us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":14251,"tablet-delete.queue_time_us":2959566}
I20260202 02:52:47.414938 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_3be2f2becb3746a8a26896e806ee480a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:47.415237 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 542) took 20451 ms. Trace:
I20260202 02:52:47.415302 1882 rpcz_store.cc:276] 0202 02:52:26.963858 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.970814 (+ 6956us) service_pool.cc:225] Handling call
0202 02:52:47.415123 (+20444309us) ts_tablet_manager.cc:1151] Deleting tablet 0138ba7326674741b0729324735f1de7
0202 02:52:47.415131 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:47.415222 (+ 91us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":20443976}
W20260202 02:52:47.415537 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3be2f2becb3746a8a26896e806ee480a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:45.852470 744 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42950 (request call id 696) took 15308 ms. Trace:
I20260202 02:52:47.415602 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 553) took 20264 ms. Trace:
I20260202 02:52:47.415657 1882 rpcz_store.cc:276] 0202 02:52:27.150879 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:27.151173 (+ 294us) service_pool.cc:225] Handling call
0202 02:52:47.415494 (+20264321us) ts_tablet_manager.cc:1151] Deleting tablet 550895b4010f4088b2cc34acff50e03f
0202 02:52:47.415502 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:47.415587 (+ 85us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":20101681}
I20260202 02:52:47.415688 744 rpcz_store.cc:276] 0202 02:52:30.544249 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.556452 (+ 12203us) service_pool.cc:225] Handling call
0202 02:52:30.566534 (+ 10082us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.566554 (+ 20us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:45.801072 (+15234518us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.852267 (+ 51195us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:45.852450 (+ 183us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260202 02:52:47.206545 513 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 5416eab7a0914ecb855fbccde4894285 (table default.Table_9bfaf1dc7bd8452ba14d3aa60e4160ff [id=427f58cc70ef47a6afac8de0353975f5], current schema version=4) failed: Not found: failed to reset TS proxy: No leader found: Tablet 5416eab7a0914ecb855fbccde4894285 consensus state has no leader
I20260202 02:52:47.415922 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 555) took 20264 ms. Trace:
I20260202 02:52:47.415980 1882 rpcz_store.cc:276] 0202 02:52:27.151112 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:27.151218 (+ 106us) service_pool.cc:225] Handling call
0202 02:52:47.415835 (+20264617us) ts_tablet_manager.cc:1151] Deleting tablet b9fc33e8efc04734a951f7fcf9048142
0202 02:52:47.415843 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:47.415909 (+ 66us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":20101960}
I20260202 02:52:47.416252 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 554) took 20265 ms. Trace:
I20260202 02:52:45.765111 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95b95947aad94b849303eb344d18fd27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:45.758038 1182 raft_consensus.cc:2272] T 474df41e4fe24a818fa45a4ce84d5474 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:47.414016 520 rpcz_store.cc:276] 0202 02:52:30.531938 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.699067 (+15167129us) service_pool.cc:225] Handling call
0202 02:52:45.699927 (+ 860us) catalog_manager.cc:2101] Acquired catalog manager lock
0202 02:52:45.713492 (+ 13565us) catalog_manager.cc:2147] Created new table and tablet info
0202 02:52:46.012956 (+299464us) catalog_manager.cc:2207] Wrote table and tablets to system table
0202 02:52:46.013027 (+ 71us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0202 02:52:46.013086 (+ 59us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:45.727470 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:45.727552 (+ 82us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:45.727557 (+ 5us) write_op.cc:435] Acquired schema lock
0202 02:52:45.727559 (+ 2us) tablet.cc:660] Decoding operations
0202 02:52:45.727626 (+ 67us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:45.727634 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:45.727638 (+ 4us) tablet.cc:683] Acquiring locks for 2 operations
0202 02:52:45.727694 (+ 56us) tablet.cc:699] Row locks acquired
0202 02:52:45.727696 (+ 2us) write_op.cc:260] PREPARE: finished
0202 02:52:45.727721 (+ 25us) write_op.cc:270] Start()
0202 02:52:45.727743 (+ 22us) write_op.cc:276] Timestamp: P: 1770000765727689 usec, L: 0
0202 02:52:45.727746 (+ 3us) op_driver.cc:348] REPLICATION: starting
0202 02:52:45.727891 (+ 145us) log.cc:844] Serialized 433 byte log entry
0202 02:52:46.010809 (+282918us) op_driver.cc:464] REPLICATION: finished
0202 02:52:46.011922 (+ 1113us) write_op.cc:301] APPLY: starting
0202 02:52:46.011931 (+ 9us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:46.011959 (+ 28us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:46.011979 (+ 20us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:46.012076 (+ 97us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:46.012102 (+ 26us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0202 02:52:46.012109 (+ 7us) write_op.cc:312] APPLY: finished
0202 02:52:46.012156 (+ 47us) log.cc:844] Serialized 29 byte log entry
0202 02:52:46.012192 (+ 36us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:46.012205 (+ 13us) write_op.cc:454] Released schema lock
0202 02:52:46.012210 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1084,"apply.run_cpu_time_us":336,"apply.run_wall_time_us":335,"mutex_wait_us":49,"num_ops":2,"prepare.queue_time_us":13415,"prepare.run_cpu_time_us":466,"prepare.run_wall_time_us":465,"replication_time_us":283049,"thread_start_us":14053,"threads_started":1}]]}
I20260202 02:52:47.416931 1182 ts_tablet_manager.cc:1916] T 474df41e4fe24a818fa45a4ce84d5474 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:47.417062 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_2918d36968d74e4fb54ea2f78f039dfc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:45.796594 2501 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:49890 (request call id 691) took 15451 ms. Trace:
W20260202 02:52:47.417553 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2918d36968d74e4fb54ea2f78f039dfc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:45.817096 1025 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49890 (request call id 699) took 15353 ms. Trace:
I20260202 02:52:47.417677 1025 rpcz_store.cc:276] 0202 02:52:30.463815 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.463862 (+ 47us) service_pool.cc:225] Handling call
0202 02:52:30.593377 (+129515us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.593398 (+ 21us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:45.725367 (+15131969us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.816528 (+ 91161us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:45.817006 (+ 478us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.418136 1025 tablet_service.cc:1505] Processing CreateTablet for tablet f87ee8d6ff9042c08ba2e32724bea720 (DEFAULT_TABLE table=default.Table_48929b86815b49369aff936433cae314 [id=2be03edfb43944be9714d5aa217263f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:47.418344 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe97009dbcc64846844e381817148d7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:45.818148 1173 ts_tablet_manager.cc:1434] T b7880cb051c24bc998003f0effed3cf8 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.076s user 0.001s sys 0.001s
W20260202 02:52:47.418453 1173 ts_tablet_manager.cc:1467] T b7880cb051c24bc998003f0effed3cf8 P b1f05820b6034228bdad884cc348716d: Tablet startup took 1676ms
I20260202 02:52:46.188481 528 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet f94f25f4059a43b8babedb6b45a08a19
W20260202 02:52:47.418514 1173 ts_tablet_manager.cc:1469] T b7880cb051c24bc998003f0effed3cf8 P b1f05820b6034228bdad884cc348716d: Trace:
0202 02:52:20.537811 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:20.537872 (+ 61us) service_pool.cc:225] Handling call
0202 02:52:20.538351 (+ 479us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:20.538376 (+ 25us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:20.549291 (+ 10915us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:20.569565 (+ 20274us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:20.577725 (+ 8160us) inbound_call.cc:173] Queueing success response
0202 02:52:45.331755 (+24754030us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0202 02:52:45.742387 (+410632us) ts_tablet_manager.cc:1435] Starting tablet replica
0202 02:52:45.742547 (+ 160us) tablet_replica.cc:261] Starting instrumentation
0202 02:52:45.742683 (+ 136us) tablet_replica.cc:280] Starting consensus
Metrics: {"raft.queue_time_us":73773,"raft.run_cpu_time_us":49,"raft.run_wall_time_us":51,"tablet-open.queue_time_us":24754069}
I20260202 02:52:47.418946 1173 tablet_bootstrap.cc:492] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:47.419036 528 catalog_manager.cc:6724] ReplaceTablet: tablet f94f25f4059a43b8babedb6b45a08a19 deleted and replaced by tablet 76d3e799f5cb46638b289d06fca2f369
I20260202 02:52:47.419162 528 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:47430 (request call id 812) took 17069 ms. Trace:
I20260202 02:52:47.419229 528 rpcz_store.cc:276] 0202 02:52:30.349383 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.270559 (+14921176us) service_pool.cc:225] Handling call
0202 02:52:47.419115 (+2148556us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:45.942135 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:45.942234 (+ 99us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:45.942240 (+ 6us) write_op.cc:435] Acquired schema lock
0202 02:52:45.942243 (+ 3us) tablet.cc:660] Decoding operations
0202 02:52:45.942363 (+ 120us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:45.943442 (+ 1079us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:45.943451 (+ 9us) tablet.cc:683] Acquiring locks for 2 operations
0202 02:52:45.943550 (+ 99us) tablet.cc:699] Row locks acquired
0202 02:52:45.943563 (+ 13us) write_op.cc:260] PREPARE: finished
0202 02:52:45.943607 (+ 44us) write_op.cc:270] Start()
0202 02:52:45.943641 (+ 34us) write_op.cc:276] Timestamp: P: 1770000765943549 usec, L: 0
0202 02:52:45.943645 (+ 4us) op_driver.cc:348] REPLICATION: starting
0202 02:52:45.943859 (+ 214us) log.cc:844] Serialized 628 byte log entry
0202 02:52:46.187214 (+243355us) op_driver.cc:464] REPLICATION: finished
0202 02:52:46.187395 (+ 181us) write_op.cc:301] APPLY: starting
0202 02:52:46.187415 (+ 20us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:46.187477 (+ 62us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:46.187538 (+ 61us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:46.187725 (+ 187us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:46.187783 (+ 58us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0202 02:52:46.187790 (+ 7us) write_op.cc:312] APPLY: finished
0202 02:52:46.187910 (+ 120us) log.cc:844] Serialized 29 byte log entry
0202 02:52:46.188018 (+ 108us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:46.188046 (+ 28us) write_op.cc:454] Released schema lock
0202 02:52:46.188053 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0202 02:52:46.188136 (+ 83us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:46.188140 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":73,"apply.run_cpu_time_us":730,"apply.run_wall_time_us":729,"num_ops":2,"prepare.queue_time_us":751,"prepare.run_cpu_time_us":781,"prepare.run_wall_time_us":1806,"replication_time_us":243528}]]}
I20260202 02:52:47.420009 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_025d64cc432643acbbef5c3fc8203457"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:45.828644 1043 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49890 (request call id 692) took 15375 ms. Trace:
I20260202 02:52:45.850277 1027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49890 (request call id 701) took 15356 ms. Trace:
W20260202 02:52:47.420567 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_025d64cc432643acbbef5c3fc8203457 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:45.853483 757 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42950 (request call id 701) took 15308 ms. Trace:
I20260202 02:52:47.412439 527 rpcz_store.cc:276] 0202 02:52:30.466047 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.678957 (+15212910us) service_pool.cc:225] Handling call
0202 02:52:45.679289 (+ 332us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0202 02:52:45.679506 (+ 217us) catalog_manager.cc:3708] Apply alter schema
0202 02:52:45.679653 (+ 147us) catalog_manager.cc:3740] Acquired catalog manager lock
0202 02:52:45.679712 (+ 59us) catalog_manager.cc:3898] Updating metadata on disk
0202 02:52:46.008419 (+328707us) catalog_manager.cc:3929] Committing alterations to in-memory state
0202 02:52:47.412274 (+1403855us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:45.714132 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:45.714250 (+ 118us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:45.714258 (+ 8us) write_op.cc:435] Acquired schema lock
0202 02:52:45.714260 (+ 2us) tablet.cc:660] Decoding operations
0202 02:52:45.714364 (+ 104us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:45.714377 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:45.714381 (+ 4us) tablet.cc:683] Acquiring locks for 1 operations
0202 02:52:45.714446 (+ 65us) tablet.cc:699] Row locks acquired
0202 02:52:45.714448 (+ 2us) write_op.cc:260] PREPARE: finished
0202 02:52:45.714489 (+ 41us) write_op.cc:270] Start()
0202 02:52:45.714526 (+ 37us) write_op.cc:276] Timestamp: P: 1770000765714454 usec, L: 0
0202 02:52:45.714530 (+ 4us) op_driver.cc:348] REPLICATION: starting
0202 02:52:45.714738 (+ 208us) log.cc:844] Serialized 405 byte log entry
0202 02:52:46.007338 (+292600us) op_driver.cc:464] REPLICATION: finished
0202 02:52:46.008055 (+ 717us) write_op.cc:301] APPLY: starting
0202 02:52:46.008065 (+ 10us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:46.008096 (+ 31us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:46.008119 (+ 23us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:46.008158 (+ 39us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:46.008180 (+ 22us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0202 02:52:46.008187 (+ 7us) write_op.cc:312] APPLY: finished
0202 02:52:46.008239 (+ 52us) log.cc:844] Serialized 23 byte log entry
0202 02:52:46.008277 (+ 38us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:46.008289 (+ 12us) write_op.cc:454] Released schema lock
0202 02:52:46.008294 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":685,"apply.run_cpu_time_us":312,"apply.run_wall_time_us":312,"num_ops":1,"prepare.queue_time_us":33942,"prepare.run_cpu_time_us":771,"prepare.run_wall_time_us":769,"replication_time_us":292793,"thread_start_us":3158,"threads_started":1}]]}
I20260202 02:52:47.420382 1043 rpcz_store.cc:276] 0202 02:52:30.453099 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.453176 (+ 77us) service_pool.cc:225] Handling call
0202 02:52:30.592741 (+139565us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.592765 (+ 24us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:45.825639 (+15232874us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.828419 (+ 2780us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:45.828614 (+ 195us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.427971 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_7950bdb0cab248acb118e0ebdf92d83d" } new_table_name: "default.Table_4e4d461d49574c68a7d93ac891db67c6" modify_external_catalogs: true
I20260202 02:52:45.869608 1039 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49890 (request call id 706) took 15368 ms. Trace:
I20260202 02:52:45.758057 2445 raft_consensus.cc:493] T fbcb4a522df648ba9542250450c08f1b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:47.428161 1043 tablet_service.cc:1505] Processing CreateTablet for tablet 76d3e799f5cb46638b289d06fca2f369 (DEFAULT_TABLE table=default.Table_22445ccfd7b64298b92a0f70733f3625 [id=300ea019d2724bee8211d7462710dc26]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:45.926667 1116 tablet.cc:2378] T b7880cb051c24bc998003f0effed3cf8 P b1f05820b6034228bdad884cc348716d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260202 02:52:47.428345 2445 raft_consensus.cc:515] T fbcb4a522df648ba9542250450c08f1b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:47.428184 1039 rpcz_store.cc:276] 0202 02:52:30.500756 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.704466 (+203710us) service_pool.cc:225] Handling call
0202 02:52:35.768174 (+5063708us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:35.768200 (+ 26us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:45.751670 (+9983470us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.869448 (+117778us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:45.869589 (+ 141us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.429093 2445 leader_election.cc:290] T fbcb4a522df648ba9542250450c08f1b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:45.758132 2441 raft_consensus.cc:493] T c050a2882f764dda8df2a7df6547f149 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f442ef85029461091de847118c34c0d)
I20260202 02:52:45.758175 1175 ts_tablet_manager.cc:1434] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.081s user 0.002s sys 0.000s
I20260202 02:52:47.429318 1039 tablet_service.cc:1505] Processing CreateTablet for tablet 9636b45987f34d16b717a8d918cc4bd6 (DEFAULT_TABLE table=default.Table_38b5d3fdb2c748cba3a6aaad03bdb766 [id=9cb73f418ae84400871b2e81ff676c8d]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:47.429363 1175 ts_tablet_manager.cc:1467] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5: Tablet startup took 1751ms
I20260202 02:52:47.429339 2441 raft_consensus.cc:515] T c050a2882f764dda8df2a7df6547f149 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:45.943202 1032 tablet_service.cc:1505] Processing CreateTablet for tablet 6b4bfc4df8ab487a924c0e04e97d2c09 (DEFAULT_TABLE table=default.Table_a2c0065f3593412fa5958d59d354f437 [id=ca66b8adfee44b6ab8dd973e36a26597]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:47.366228 2372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=434 overwritten=0 applied=430 ignored=0} inserts{seen=399 ignored=0} mutations{seen=468 ignored=0} orphaned_commits=0. Pending: 4 replicates
W20260202 02:52:47.442009 1389 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.190:33981, user_credentials={real_user=slave}} blocked reactor thread for 1.54062e+06us
I20260202 02:52:45.943230 1038 tablet_service.cc:1505] Processing CreateTablet for tablet 32c3fece7c2145fb86bbc11c4c221f27 (DEFAULT_TABLE table=default.Table_31ed23ec1b894b32953e20125e57d70a [id=0fb5920dae13430c81292070d04bb203]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:47.442418 2618 raft_consensus.cc:2749] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260202 02:52:47.686399 2372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8: Bootstrap complete.
I20260202 02:52:45.945782 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 6750a1212ceb4e6e82e4ffa93caab590 (DEFAULT_TABLE table=default.Table_52a2f389faba40fea93e2bca972e81e1 [id=70ac373200604907b00132eb4727ea5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:45.951733 1031 tablet_service.cc:1505] Processing CreateTablet for tablet 912ac4044b5645caa7eda2a565c5b8bb (DEFAULT_TABLE table=default.Table_0ee7c010ec144c2e9d633447e4d84dcf [id=812637d0b1da4789850022a699e6bbd1]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:47.429478 1175 ts_tablet_manager.cc:1469] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5: Trace:
0202 02:52:21.545426 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:21.560415 (+ 14989us) service_pool.cc:225] Handling call
0202 02:52:21.560936 (+ 521us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:21.560963 (+ 27us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:21.564364 (+ 3401us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:21.567154 (+ 2790us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:21.580620 (+ 13466us) inbound_call.cc:173] Queueing success response
0202 02:52:30.544999 (+8964379us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0202 02:52:45.677486 (+15132487us) ts_tablet_manager.cc:1435] Starting tablet replica
0202 02:52:45.747924 (+ 70438us) tablet_replica.cc:261] Starting instrumentation
0202 02:52:45.748058 (+ 134us) tablet_replica.cc:280] Starting consensus
Metrics: {"mutex_wait_us":1641,"raft.queue_time_us":2087,"raft.run_cpu_time_us":39,"raft.run_wall_time_us":38,"tablet-open.queue_time_us":8977725}
I20260202 02:52:47.412678 523 rpcz_store.cc:276] 0202 02:52:30.501575 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.696172 (+15194597us) service_pool.cc:225] Handling call
0202 02:52:45.697138 (+ 966us) catalog_manager.cc:2101] Acquired catalog manager lock
0202 02:52:45.697625 (+ 487us) catalog_manager.cc:2147] Created new table and tablet info
0202 02:52:46.011479 (+313854us) catalog_manager.cc:2207] Wrote table and tablets to system table
0202 02:52:46.011607 (+ 128us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0202 02:52:46.011714 (+ 107us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:45.715526 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:45.715609 (+ 83us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:45.715614 (+ 5us) write_op.cc:435] Acquired schema lock
0202 02:52:45.715616 (+ 2us) tablet.cc:660] Decoding operations
0202 02:52:45.715682 (+ 66us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:45.715690 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:45.715695 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
0202 02:52:45.726408 (+ 10713us) tablet.cc:699] Row locks acquired
0202 02:52:45.726412 (+ 4us) write_op.cc:260] PREPARE: finished
0202 02:52:45.726470 (+ 58us) write_op.cc:270] Start()
0202 02:52:45.726514 (+ 44us) write_op.cc:276] Timestamp: P: 1770000765726431 usec, L: 0
0202 02:52:45.726518 (+ 4us) op_driver.cc:348] REPLICATION: starting
0202 02:52:45.726727 (+ 209us) log.cc:844] Serialized 433 byte log entry
0202 02:52:46.010724 (+283997us) op_driver.cc:464] REPLICATION: finished
0202 02:52:46.011052 (+ 328us) write_op.cc:301] APPLY: starting
0202 02:52:46.011065 (+ 13us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:46.011103 (+ 38us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:46.011134 (+ 31us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:46.011235 (+ 101us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:46.011265 (+ 30us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0202 02:52:46.011272 (+ 7us) write_op.cc:312] APPLY: finished
0202 02:52:46.011335 (+ 63us) log.cc:844] Serialized 29 byte log entry
0202 02:52:46.011380 (+ 45us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:46.011395 (+ 15us) write_op.cc:454] Released schema lock
0202 02:52:46.011401 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0202 02:52:46.011897 (+ 496us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:46.011902 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":285,"apply.run_cpu_time_us":419,"apply.run_wall_time_us":837,"num_ops":2,"prepare.queue_time_us":17335,"prepare.run_cpu_time_us":664,"prepare.run_wall_time_us":11280,"replication_time_us":284190}]]}
I20260202 02:52:47.409333 524 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_c6e682a233904983978e7abb23565e41" } modify_external_catalogs: true
I20260202 02:52:47.688656 1175 tablet_bootstrap.cc:492] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:45.853932 747 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42950 (request call id 715) took 15308 ms. Trace:
I20260202 02:52:47.688676 523 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_a15c97bb32284dcfb36cdc47921b2346" } modify_external_catalogs: true
I20260202 02:52:45.953536 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 461ecb9ee90f4e94b12934c56de475a7 (DEFAULT_TABLE table=default.Table_f468fb0750cf421e833dea1a00d62640 [id=88ea44b25a404b15901d1a267de4f03a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:47.688827 747 rpcz_store.cc:276] 0202 02:52:30.545881 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.628966 (+ 83085us) service_pool.cc:225] Handling call
0202 02:52:35.768999 (+5140033us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:35.769021 (+ 22us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:45.711303 (+9942282us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.853796 (+142493us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:45.853918 (+ 122us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.688889 524 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_c6e682a233904983978e7abb23565e41" } modify_external_catalogs: true
I20260202 02:52:45.758257 2431 raft_consensus.cc:493] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b1f05820b6034228bdad884cc348716d)
I20260202 02:52:47.688898 523 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_a15c97bb32284dcfb36cdc47921b2346" } modify_external_catalogs: true
I20260202 02:52:47.689162 2431 raft_consensus.cc:515] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:47.426307 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 13cd1383348b458894fdecc22bf24b74 (table default.Table_f92d6341db3143e490f6d64418ef6ff2 [id=ebc82aae55584b8ba2ba5c4223d9e258]) successfully deleted
W20260202 02:52:47.689469 515 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 263142us
W20260202 02:52:47.689714 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 0138ba7326674741b0729324735f1de7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0138ba7326674741b0729324735f1de7 already in progress: creating tablet
I20260202 02:52:47.689805 2431 leader_election.cc:290] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:45.856006 743 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42950 (request call id 704) took 15311 ms. Trace:
W20260202 02:52:47.689941 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 550895b4010f4088b2cc34acff50e03f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 550895b4010f4088b2cc34acff50e03f already in progress: creating tablet
I20260202 02:52:47.689953 743 rpcz_store.cc:276] 0202 02:52:30.544936 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.564406 (+ 19470us) service_pool.cc:225] Handling call
0202 02:52:30.566680 (+ 2274us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.566696 (+ 16us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:45.800459 (+15233763us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.855813 (+ 55354us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:45.855984 (+ 171us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260202 02:52:47.690099 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet b9fc33e8efc04734a951f7fcf9048142 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b9fc33e8efc04734a951f7fcf9048142 already in progress: creating tablet
I20260202 02:52:46.074481 1063 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "75569511179b41038e521fa3346a92ae" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:47.690313 1063 raft_consensus.cc:2393] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f442ef85029461091de847118c34c0d in current term 1: Already voted for candidate b1f05820b6034228bdad884cc348716d in this term.
I20260202 02:52:47.690454 1063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.129:38747 (request call id 351) took 1616 ms. Trace:
I20260202 02:52:47.690515 1063 rpcz_store.cc:276] 0202 02:52:46.074313 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:46.074365 (+ 52us) service_pool.cc:225] Handling call
0202 02:52:47.690430 (+1616065us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:45.766441 2450 leader_election.cc:290] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:45.857385 2486 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:42950 (request call id 665) took 16592 ms. Trace:
I20260202 02:52:47.690866 2486 rpcz_store.cc:276] 0202 02:52:29.264808 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.265107 (+ 299us) service_pool.cc:225] Handling call
0202 02:52:45.857368 (+16592261us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:29.322906 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0202 02:52:29.323047 (+ 141us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0202 02:52:30.280399 (+957352us) alter_schema_op.cc:66] Acquired schema lock
0202 02:52:30.280413 (+ 14us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0202 02:52:30.280488 (+ 75us) alter_schema_op.cc:128] START. Timestamp: 1770000750280438
0202 02:52:30.280510 (+ 22us) op_driver.cc:348] REPLICATION: starting
0202 02:52:30.280797 (+ 287us) log.cc:844] Serialized 173 byte log entry
0202 02:52:35.775499 (+5494702us) op_driver.cc:464] REPLICATION: finished
0202 02:52:35.943677 (+168178us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0202 02:52:45.662430 (+9718753us) log.cc:844] Serialized 94 byte log entry
0202 02:52:45.857321 (+194891us) alter_schema_op.cc:73] Released schema lock
0202 02:52:45.857327 (+ 6us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":168094,"mutex_wait_us":3,"prepare.queue_time_us":56,"prepare.run_cpu_time_us":65833,"prepare.run_wall_time_us":958037,"raft.queue_time_us":319,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":213,"replication_time_us":5494952,"thread_start_us":411788,"threads_started":2,"wal-append.queue_time_us":194354,"wal-append.run_cpu_time_us":280,"wal-append.run_wall_time_us":1000349}]]}
I20260202 02:52:46.074947 1051 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "25765fb687294e9db1289f76530dd877" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:47.691704 1051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.129:38747 (request call id 352) took 1616 ms. Trace:
I20260202 02:52:47.691780 1051 rpcz_store.cc:276] 0202 02:52:46.074755 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:46.074804 (+ 49us) service_pool.cc:225] Handling call
0202 02:52:47.691682 (+1616878us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:46.075703 1052 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cb634d4bae554ef3b39eeaa9958adc6c" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:47.692034 1052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.129:38747 (request call id 355) took 1616 ms. Trace:
I20260202 02:52:47.692096 1052 rpcz_store.cc:276] 0202 02:52:46.075588 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:46.075632 (+ 44us) service_pool.cc:225] Handling call
0202 02:52:47.692022 (+1616390us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:46.076188 1057 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "622a113af04a41c6852c29fc8a8acd78" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:47.692293 1057 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.129:38747 (request call id 357) took 1616 ms. Trace:
I20260202 02:52:47.692378 1057 rpcz_store.cc:276] 0202 02:52:46.076047 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:46.076095 (+ 48us) service_pool.cc:225] Handling call
0202 02:52:47.692281 (+1616186us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:46.076900 1061 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "02b434fa058b45529ff3938b053263c6" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:47.692612 1061 raft_consensus.cc:2393] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f442ef85029461091de847118c34c0d in current term 1: Already voted for candidate b1f05820b6034228bdad884cc348716d in this term.
I20260202 02:52:47.692698 1061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.129:38747 (request call id 350) took 1618 ms. Trace:
I20260202 02:52:47.692749 1061 rpcz_store.cc:276] 0202 02:52:46.074181 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:46.076795 (+ 2614us) service_pool.cc:225] Handling call
0202 02:52:47.692686 (+1615891us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:45.767164 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b2b4f429b6a484ea6ef7c19124b236b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:46.087510 1053 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f6746ccbf37e477196a9706396d1e3fd" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:47.693078 1053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.129:38747 (request call id 361) took 1605 ms. Trace:
I20260202 02:52:47.693138 1053 rpcz_store.cc:276] 0202 02:52:46.087307 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:46.087404 (+ 97us) service_pool.cc:225] Handling call
0202 02:52:47.693063 (+1605659us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:45.767925 900 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41736 (request call id 720) took 15135 ms. Trace:
I20260202 02:52:47.693289 900 rpcz_store.cc:276] 0202 02:52:30.632576 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.668992 (+15036416us) service_pool.cc:225] Handling call
0202 02:52:45.669389 (+ 397us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:45.669418 (+ 29us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:45.725220 (+ 55802us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.767720 (+ 42500us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:45.767901 (+ 181us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:46.088397 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "20ea39bc5d3d46f5a4ba427277d6c6a3" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:47.693655 1060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.129:38747 (request call id 362) took 1605 ms. Trace:
I20260202 02:52:47.693723 1060 rpcz_store.cc:276] 0202 02:52:46.088134 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:46.088284 (+ 150us) service_pool.cc:225] Handling call
0202 02:52:47.693638 (+1605354us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:46.089622 1056 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "be993c1512b04df4b92cab3195241d84" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:47.693964 1056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.129:38747 (request call id 363) took 1605 ms. Trace:
I20260202 02:52:47.694026 1056 rpcz_store.cc:276] 0202 02:52:46.089320 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:46.089530 (+ 210us) service_pool.cc:225] Handling call
0202 02:52:47.693950 (+1604420us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:46.090600 1058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "91ec01d25f8f441cbe3783cacbee51db" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:47.694227 1058 raft_consensus.cc:2468] T 91ec01d25f8f441cbe3783cacbee51db P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:47.694331 1058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.129:38747 (request call id 358) took 1607 ms. Trace:
I20260202 02:52:47.694386 1058 rpcz_store.cc:276] 0202 02:52:46.086527 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:46.090524 (+ 3997us) service_pool.cc:225] Handling call
0202 02:52:47.694319 (+1603795us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:46.113456 1054 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b76d1837813c46e8a0925b49939d8884" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:47.694582 1054 raft_consensus.cc:2468] T b76d1837813c46e8a0925b49939d8884 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:47.694695 1054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.129:38747 (request call id 364) took 1604 ms. Trace:
I20260202 02:52:47.694749 1054 rpcz_store.cc:276] 0202 02:52:46.113067 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:46.113290 (+ 223us) service_pool.cc:225] Handling call
0202 02:52:47.694683 (+1581393us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:46.145606 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3d1a7578e83447669e3c3972a426ad44" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:47.694962 1059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.129:38747 (request call id 365) took 1580 ms. Trace:
I20260202 02:52:47.695022 1059 rpcz_store.cc:276] 0202 02:52:46.144547 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:46.145477 (+ 930us) service_pool.cc:225] Handling call
0202 02:52:47.694948 (+1549471us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:46.156509 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "73f8db8f30f14d369d2227c9be7c4647" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:47.695231 1064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.129:38747 (request call id 366) took 1549 ms. Trace:
I20260202 02:52:47.695293 1064 rpcz_store.cc:276] 0202 02:52:46.145977 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:46.156396 (+ 10419us) service_pool.cc:225] Handling call
0202 02:52:47.695218 (+1538822us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:46.158766 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e1284fde67bc4d65b067a54c416ccc33" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:47.695513 1062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.129:38747 (request call id 367) took 1538 ms. Trace:
I20260202 02:52:47.695632 1062 rpcz_store.cc:276] 0202 02:52:46.156995 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:46.158647 (+ 1652us) service_pool.cc:225] Handling call
0202 02:52:47.695500 (+1536853us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:46.188853 1050 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26eeef6eebbf4f2cb44f69255fb37594" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:47.695820 1050 raft_consensus.cc:3060] T 26eeef6eebbf4f2cb44f69255fb37594 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:45.860266 1174 tablet_bootstrap.cc:654] T 33150e1456144b8e9c64cc2a2ac6dcca P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:45.860984 748 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42950 (request call id 707) took 15315 ms. Trace:
I20260202 02:52:47.696440 748 rpcz_store.cc:276] 0202 02:52:30.545257 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.565631 (+ 20374us) service_pool.cc:225] Handling call
0202 02:52:30.565949 (+ 318us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.565968 (+ 19us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:45.832047 (+15266079us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.859268 (+ 27221us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:45.860965 (+ 1697us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.696879 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39eaa2848cf4450dbe563ddd3b11df06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:45.777931 886 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41736 (request call id 718) took 15145 ms. Trace:
W20260202 02:52:45.784350 857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 1.21623e+07us
I20260202 02:52:47.697199 886 rpcz_store.cc:276] 0202 02:52:30.632450 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.664657 (+15032207us) service_pool.cc:225] Handling call
0202 02:52:45.665099 (+ 442us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:45.665126 (+ 27us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:45.711894 (+ 46768us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.777705 (+ 65811us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:45.777901 (+ 196us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:45.868005 751 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42950 (request call id 708) took 15322 ms. Trace:
I20260202 02:52:47.697435 751 rpcz_store.cc:276] 0202 02:52:30.545334 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.569801 (+ 24467us) service_pool.cc:225] Handling call
0202 02:52:30.570170 (+ 369us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.570192 (+ 22us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:45.858074 (+15287882us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.866497 (+ 8423us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:45.867930 (+ 1433us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.697809 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet ccd6bc8bd0504bf18d108c73685ef9ff (table default.Table_5ecb81038c9f48438e74fe874fc83e74 [id=c28e30416db3407b8c20c68091f392b3]) successfully deleted
W20260202 02:52:45.785293 971 tablet.cc:2378] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260202 02:52:45.869606 742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42950 (request call id 714) took 15323 ms. Trace:
I20260202 02:52:47.698568 1050 raft_consensus.cc:2468] T 26eeef6eebbf4f2cb44f69255fb37594 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:47.698493 742 rpcz_store.cc:276] 0202 02:52:30.545801 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.624940 (+ 79139us) service_pool.cc:225] Handling call
0202 02:52:35.771462 (+5146522us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:35.771484 (+ 22us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:45.865315 (+10093831us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.869427 (+ 4112us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:45.869587 (+ 160us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:45.873515 749 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42950 (request call id 718) took 15327 ms. Trace:
I20260202 02:52:47.698707 1050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.129:38747 (request call id 368) took 1528 ms. Trace:
I20260202 02:52:47.698732 749 rpcz_store.cc:276] 0202 02:52:30.546108 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.682029 (+135921us) service_pool.cc:225] Handling call
0202 02:52:35.772659 (+5090630us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:35.772682 (+ 23us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:45.866721 (+10094039us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.873312 (+ 6591us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:45.873491 (+ 179us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.698770 1050 rpcz_store.cc:276] 0202 02:52:46.188148 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:46.188714 (+ 566us) service_pool.cc:225] Handling call
0202 02:52:47.698688 (+1509974us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:46.207499 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "026e6ee0b3ae4d938fdf1550044ed1fa" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:45.874310 741 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42950 (request call id 710) took 15328 ms. Trace:
I20260202 02:52:47.698938 741 rpcz_store.cc:276] 0202 02:52:30.545503 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.570723 (+ 25220us) service_pool.cc:225] Handling call
0202 02:52:30.571057 (+ 334us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.571076 (+ 19us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:45.861206 (+15290130us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.874165 (+ 12959us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:45.874292 (+ 127us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.699060 1049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.129:38747 (request call id 369) took 1503 ms. Trace:
I20260202 02:52:47.699124 1049 rpcz_store.cc:276] 0202 02:52:46.206197 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:46.207365 (+ 1168us) service_pool.cc:225] Handling call
0202 02:52:47.699044 (+1491679us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.699366 2445 raft_consensus.cc:493] T 70306ed9dec54286885d965f2c012da4 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f442ef85029461091de847118c34c0d)
I20260202 02:52:46.217660 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "16b47a2831f54d809ed0b16deaec7e3d" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:47.699533 2445 raft_consensus.cc:515] T 70306ed9dec54286885d965f2c012da4 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:47.699638 1048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.129:38747 (request call id 370) took 1488 ms. Trace:
I20260202 02:52:47.699709 1048 rpcz_store.cc:276] 0202 02:52:46.216952 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:46.217512 (+ 560us) service_pool.cc:225] Handling call
0202 02:52:47.699623 (+1482111us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.700167 768 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a0b5d82e5fb94b60b4aa324b978a5378" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:47.700217 2445 leader_election.cc:290] T 70306ed9dec54286885d965f2c012da4 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:47.700384 2445 raft_consensus.cc:493] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b1f05820b6034228bdad884cc348716d)
I20260202 02:52:47.700534 2445 raft_consensus.cc:515] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:45.789166 901 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41736 (request call id 719) took 15156 ms. Trace:
I20260202 02:52:47.700599 769 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6b8f924aecf44e959e47d2f32e6b0bf5" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:47.700690 901 rpcz_store.cc:276] 0202 02:52:30.632513 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.666499 (+15033986us) service_pool.cc:225] Handling call
0202 02:52:45.666906 (+ 407us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:45.666937 (+ 31us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:45.786171 (+119234us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.788943 (+ 2772us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:45.789137 (+ 194us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:45.790019 895 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41736 (request call id 715) took 15157 ms. Trace:
I20260202 02:52:47.700886 769 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "63a9029bc39c4fbd8ac4bf4bc37d6b4e" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:47.700912 895 rpcz_store.cc:276] 0202 02:52:30.632173 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.632250 (+ 77us) service_pool.cc:225] Handling call
0202 02:52:35.770413 (+5138163us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:35.770435 (+ 22us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:45.769926 (+9999491us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.789768 (+ 19842us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:45.790000 (+ 232us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:45.808672 887 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41736 (request call id 707) took 15203 ms. Trace:
I20260202 02:52:46.314332 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "874ca4ba4ebc4721bb4b60cba8e01841" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:47.701110 887 rpcz_store.cc:276] 0202 02:52:30.605416 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.607704 (+ 2288us) service_pool.cc:225] Handling call
0202 02:52:35.770189 (+5162485us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:35.770219 (+ 30us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:45.667177 (+9896958us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.808304 (+141127us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:45.808611 (+ 307us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.701165 769 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ac2ab12f5040488aa306cb69b107b485" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:45.812136 892 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41736 (request call id 714) took 15186 ms. Trace:
I20260202 02:52:47.701227 1046 raft_consensus.cc:2468] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:47.701308 892 rpcz_store.cc:276] 0202 02:52:30.626090 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.631558 (+ 5468us) service_pool.cc:225] Handling call
0202 02:52:35.770329 (+5138771us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:35.770351 (+ 22us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:45.664168 (+9893817us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.811784 (+147616us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:45.812073 (+ 289us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.701350 1046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.129:38747 (request call id 375) took 1392 ms. Trace:
I20260202 02:52:45.814203 903 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41736 (request call id 713) took 15207 ms. Trace:
I20260202 02:52:47.701413 1046 rpcz_store.cc:276] 0202 02:52:46.313184 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:46.314196 (+ 1012us) service_pool.cc:225] Handling call
0202 02:52:47.701333 (+1387137us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.701439 769 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d0ce2121d65b4a228596e0ca9018883d" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:47.701486 903 rpcz_store.cc:276] 0202 02:52:30.607158 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.625192 (+ 18034us) service_pool.cc:225] Handling call
0202 02:52:35.770500 (+5145308us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:35.770515 (+ 15us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:45.671209 (+9900694us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.813840 (+142631us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:45.814154 (+ 314us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":896}
I20260202 02:52:45.842376 893 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41736 (request call id 706) took 15300 ms. Trace:
I20260202 02:52:47.701676 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79c73f4a92044bafb5b8384229cb3a4d" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:47.701721 769 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8bb14bf551f441dcbaaff7057b51b2be" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:45.757952 859 leader_election.cc:343] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:45.970984 840 kernel_stack_watchdog.cc:198] Thread 859 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 130ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fb3588d9980 (unknown) at ??:0
@ 0x7fb3588d39c6 __GI___pthread_rwlock_wrlock at ??:0
@ 0x7fb35254c904 google::LogMessage::Flush() at ??:0
@ 0x7fb35254ca11 google::LogMessage::~LogMessage() at ??:0
@ 0x7fb3610da503 kudu::consensus::LeaderElection::VoteResponseRpcCallback() at ??:0
@ 0x7fb3610dd07a kudu::consensus::LeaderElection::Run()::$_0::operator()() at ??:0
@ 0x7fb3610dce3e std::_Function_handler<>::_M_invoke() at ??:0
@ 0xa8970d std::function<>::operator()() at ??:?
@ 0x7fb35c1ce2df kudu::rpc::Proxy::AsyncRequest()::$_1::operator()() at ??:0
@ 0x7fb35c1cdebe std::_Function_handler<>::_M_invoke() at ??:0
@ 0xa8970d std::function<>::operator()() at ??:?
@ 0x7fb35c1b9f10 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7fb35c1ba45c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7fb35c15581a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7fb35c1541d6 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7fb35c1666a5 ev::base<>::method_thunk<>() at ??:0
I20260202 02:52:45.984773 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "25765fb687294e9db1289f76530dd877" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:47.701988 769 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1d8143f529b74e6ab779dd695920413c" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:47.702011 776 raft_consensus.cc:2468] T 25765fb687294e9db1289f76530dd877 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:47.702078 764 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f94f25f4059a43b8babedb6b45a08a19" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:47.702126 776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.131:34317 (request call id 269) took 1718 ms. Trace:
I20260202 02:52:47.702229 769 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c050a2882f764dda8df2a7df6547f149" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:47.702194 776 rpcz_store.cc:276] 0202 02:52:45.984313 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.984651 (+ 338us) service_pool.cc:225] Handling call
0202 02:52:47.702113 (+1717462us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.702320 764 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d8418a9dd0944456b4d4820dde0cde02" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:47.702339 859 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 1.94418e+06us
I20260202 02:52:47.702569 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fbcb4a522df648ba9542250450c08f1b" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:46.315971 2601 tablet.cc:1717] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d: Skipping requested alter to schema version 1, tablet already version 1
I20260202 02:52:47.702685 769 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "29c996781cb947f3a1aee5dd0a9c4828" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:47.702792 859 leader_election.cc:343] T d0ce2121d65b4a228596e0ca9018883d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:46.390661 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3d1a7578e83447669e3c3972a426ad44" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:47.703250 2430 raft_consensus.cc:493] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b1f05820b6034228bdad884cc348716d)
I20260202 02:52:47.703280 1045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.130:37027 (request call id 273) took 1316 ms. Trace:
I20260202 02:52:47.703286 770 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "20ea39bc5d3d46f5a4ba427277d6c6a3" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:47.703358 1045 rpcz_store.cc:276] 0202 02:52:46.390014 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:46.390527 (+ 513us) service_pool.cc:225] Handling call
0202 02:52:47.703247 (+1312720us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.703389 2430 raft_consensus.cc:515] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:46.093734 999 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.104.130:38539 timed out after 4.118s (SENT)
I20260202 02:52:47.703567 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "78f122342dd144b48de89902faa0f94f" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:47.704104 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28a900bcaa8b41718a285e595886d805" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:47.704219 2430 leader_election.cc:290] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:47.704492 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "af5aa68e47bb420eac300754d274fa73" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:47.704854 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3d1a7578e83447669e3c3972a426ad44" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:46.423053 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cb634d4bae554ef3b39eeaa9958adc6c" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:47.705161 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "474df41e4fe24a818fa45a4ce84d5474" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:47.705317 1047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.130:37027 (request call id 274) took 1291 ms. Trace:
I20260202 02:52:46.472939 2427 raft_consensus.cc:493] T b7880cb051c24bc998003f0effed3cf8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:47.705394 1047 rpcz_store.cc:276] 0202 02:52:46.421680 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:46.422910 (+ 1230us) service_pool.cc:225] Handling call
0202 02:52:47.705298 (+1282388us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.705549 2427 raft_consensus.cc:515] T b7880cb051c24bc998003f0effed3cf8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:47.713267 2427 leader_election.cc:290] T b7880cb051c24bc998003f0effed3cf8 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:46.713587 998 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 45154.1us
I20260202 02:52:46.893420 1044 tablet_service.cc:1505] Processing CreateTablet for tablet a402b238ae2b4a379598019914c68469 (DEFAULT_TABLE table=default.Table_33c051aaf8e1494fa559ae70367f89e9 [id=8e5a555f4f44492aa00b05de07ca2fed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:47.420400 1027 rpcz_store.cc:276] 0202 02:52:30.494002 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.619415 (+125413us) service_pool.cc:225] Handling call
0202 02:52:35.767780 (+5148365us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:35.767803 (+ 23us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:45.718925 (+9951122us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.850158 (+131233us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:45.850263 (+ 105us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:45.986388 760 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c050a2882f764dda8df2a7df6547f149" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:47.713949 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 8a206159afa84ae19f8c7763a2fe43d4 (DEFAULT_TABLE table=default.Table_06dc35989d444cccb418b9d2db62721d [id=e5f4256ed9434c52b0b71d84c58c0508]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:45.849964 1030 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49890 (request call id 696) took 15390 ms. Trace:
I20260202 02:52:47.714211 760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.131:34317 (request call id 270) took 1728 ms. Trace:
I20260202 02:52:47.714293 760 rpcz_store.cc:276] 0202 02:52:45.985916 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.986241 (+ 325us) service_pool.cc:225] Handling call
0202 02:52:47.714188 (+1727947us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.714267 1030 rpcz_store.cc:276] 0202 02:52:30.459762 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.459812 (+ 50us) service_pool.cc:225] Handling call
0202 02:52:30.593142 (+133330us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.593163 (+ 21us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:45.733976 (+15140813us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.849741 (+115765us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:45.849932 (+ 191us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:45.987370 767 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "622a113af04a41c6852c29fc8a8acd78" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:45.992602 761 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "06b80a1c9251420cbc16fc95ae98aca9" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:47.714563 767 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.131:34317 (request call id 271) took 1727 ms. Trace:
I20260202 02:52:47.714627 767 rpcz_store.cc:276] 0202 02:52:45.987076 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.987278 (+ 202us) service_pool.cc:225] Handling call
0202 02:52:47.714548 (+1727270us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.420572 1882 rpcz_store.cc:276] 0202 02:52:27.150995 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:27.151384 (+ 389us) service_pool.cc:225] Handling call
0202 02:52:47.416147 (+20264763us) ts_tablet_manager.cc:1151] Deleting tablet c977cd5aa66a4f6c8eeb37bf67f18fa7
0202 02:52:47.416154 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:47.416228 (+ 74us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":20102229}
I20260202 02:52:47.714712 761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.131:34317 (request call id 272) took 1724 ms. Trace:
I20260202 02:52:47.714689 1030 tablet_service.cc:1505] Processing CreateTablet for tablet c438e69c40c24655b37f2e73b4b3b845 (DEFAULT_TABLE table=default.Table_ff96a2dc345940448683f8be23859993 [id=8e592b0d560f48ed86a5c3886f178c2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:47.714802 761 rpcz_store.cc:276] 0202 02:52:45.991221 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.992463 (+ 1242us) service_pool.cc:225] Handling call
0202 02:52:47.714697 (+1722234us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:45.993022 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "874ca4ba4ebc4721bb4b60cba8e01841" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:47.701695 893 rpcz_store.cc:276] 0202 02:52:30.541572 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.548663 (+ 7091us) service_pool.cc:225] Handling call
0202 02:52:35.757912 (+5209249us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:35.757947 (+ 35us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:45.760551 (+10002604us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.842185 (+ 81634us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:45.842353 (+ 168us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.715023 772 raft_consensus.cc:2468] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:47.715042 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 556) took 20553 ms. Trace:
I20260202 02:52:47.715129 772 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.131:34317 (request call id 273) took 1723 ms. Trace:
I20260202 02:52:47.715109 1882 rpcz_store.cc:276] 0202 02:52:27.161817 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:27.161884 (+ 67us) service_pool.cc:225] Handling call
0202 02:52:47.714890 (+20553006us) ts_tablet_manager.cc:1151] Deleting tablet 940e656cd34e4e53896fda6d38132b6e
0202 02:52:47.714898 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:47.715023 (+ 125us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":20400927}
I20260202 02:52:47.715180 772 rpcz_store.cc:276] 0202 02:52:45.992757 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.992936 (+ 179us) service_pool.cc:225] Handling call
0202 02:52:47.715118 (+1722182us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:45.994298 763 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d06b77ff1a6f4f78b73082b00145fcf6" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:45.997381 765 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "75569511179b41038e521fa3346a92ae" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:47.429855 2441 leader_election.cc:290] T c050a2882f764dda8df2a7df6547f149 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:47.715404 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 557) took 20502 ms. Trace:
I20260202 02:52:47.715420 765 raft_consensus.cc:3060] T 75569511179b41038e521fa3346a92ae P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:45.999138 762 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26b890ed87c3471b9fc0b172c90e0a11" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:47.715463 1882 rpcz_store.cc:276] 0202 02:52:27.213040 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:27.213096 (+ 56us) service_pool.cc:225] Handling call
0202 02:52:47.715310 (+20502214us) ts_tablet_manager.cc:1151] Deleting tablet 85f754f006d6432bb1cba87043903ac2
0202 02:52:47.715318 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:47.715391 (+ 73us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":20385460}
I20260202 02:52:47.715616 762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.131:34317 (request call id 277) took 1717 ms. Trace:
I20260202 02:52:47.715677 762 rpcz_store.cc:276] 0202 02:52:45.998989 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.999036 (+ 47us) service_pool.cc:225] Handling call
0202 02:52:47.715604 (+1716568us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.715730 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 558) took 20497 ms. Trace:
I20260202 02:52:46.000313 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fe53a8bdbd13417f9ce195c54a52a93c" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:47.715787 1882 rpcz_store.cc:276] 0202 02:52:27.218294 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:27.218351 (+ 57us) service_pool.cc:225] Handling call
0202 02:52:47.715644 (+20497293us) ts_tablet_manager.cc:1151] Deleting tablet 7b45a2e6b4b14d3fa96a7131afb16208
0202 02:52:47.715651 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:47.715718 (+ 67us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":20385739}
I20260202 02:52:47.715891 777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.131:34317 (request call id 278) took 1716 ms. Trace:
I20260202 02:52:47.715953 777 rpcz_store.cc:276] 0202 02:52:46.000046 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:46.000244 (+ 198us) service_pool.cc:225] Handling call
0202 02:52:47.715879 (+1715635us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:46.001576 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "02b434fa058b45529ff3938b053263c6" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:47.716054 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 559) took 20495 ms. Trace:
W20260202 02:52:46.343708 715 consensus_peers.cc:597] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 761f5a51335a4de5977f5f7fef0d4260. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:47.716135 766 raft_consensus.cc:3060] T 02b434fa058b45529ff3938b053263c6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:47.716110 1882 rpcz_store.cc:276] 0202 02:52:27.220507 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:27.220557 (+ 50us) service_pool.cc:225] Handling call
0202 02:52:47.715969 (+20495412us) ts_tablet_manager.cc:1151] Deleting tablet 25f1d698c4ee4be7a33380184b5e2b4d
0202 02:52:47.715977 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:47.716041 (+ 64us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":20386020}
W20260202 02:52:47.716174 715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 1.37239e+06us
I20260202 02:52:47.716354 1882 tablet_replica.cc:333] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:47.420879 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c584f3163f1b45789492ccd27b7ad609. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:47.717489 2619 consensus_queue.cc:1048] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:47.718159 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b4bfc4df8ab487a924c0e04e97d2c09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:47.718152 765 raft_consensus.cc:2468] T 75569511179b41038e521fa3346a92ae P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:47.718267 765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.131:34317 (request call id 276) took 1721 ms. Trace:
I20260202 02:52:47.718323 765 rpcz_store.cc:276] 0202 02:52:45.997089 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.997280 (+ 191us) service_pool.cc:225] Handling call
0202 02:52:47.718253 (+1720973us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260202 02:52:47.188194 700 kernel_stack_watchdog.cc:198] Thread 715 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 113ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f1e3b197980 (unknown) at ??:0
@ 0x7f1e3b1919c6 __GI___pthread_rwlock_wrlock at ??:0
@ 0x7f1e34e0a904 google::LogMessage::Flush() at ??:0
@ 0x7f1e34e0aa11 google::LogMessage::~LogMessage() at ??:0
@ 0x7f1e439263de kudu::consensus::Peer::ProcessResponseErrorUnlocked() at ??:0
@ 0x7f1e4392549f kudu::consensus::Peer::ProcessResponse() at ??:0
@ 0x7f1e43925bd5 kudu::consensus::Peer::ProcessSingleResponse() at ??:0
@ 0x7f1e4392d761 kudu::consensus::Peer::SendNextRequest()::$_6::operator()() at ??:0
@ 0x7f1e4392d56e std::_Function_handler<>::_M_invoke() at ??:0
@ 0xa8970d std::function<>::operator()() at ??:?
@ 0x7f1e3ea8c2df kudu::rpc::Proxy::AsyncRequest()::$_1::operator()() at ??:0
@ 0x7f1e3ea8bebe std::_Function_handler<>::_M_invoke() at ??:0
@ 0xa8970d std::function<>::operator()() at ??:?
@ 0x7f1e3ea77f10 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f1e3ea7845c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f1e3ea1381a kudu::rpc::Connection::HandleCallResponse() at ??:0
I20260202 02:52:47.718463 766 raft_consensus.cc:2468] T 02b434fa058b45529ff3938b053263c6 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:47.718588 766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.131:34317 (request call id 279) took 1717 ms. Trace:
I20260202 02:52:47.420733 757 rpcz_store.cc:276] 0202 02:52:30.544690 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.568392 (+ 23702us) service_pool.cc:225] Handling call
0202 02:52:30.568731 (+ 339us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.568753 (+ 22us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:45.749435 (+15180682us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.853245 (+103810us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:45.853465 (+ 220us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.718650 766 rpcz_store.cc:276] 0202 02:52:46.001306 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:46.001480 (+ 174us) service_pool.cc:225] Handling call
0202 02:52:47.718571 (+1717091us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:45.758198 2439 raft_consensus.cc:493] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b1f05820b6034228bdad884cc348716d)
I20260202 02:52:47.718849 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a402b238ae2b4a379598019914c68469. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:47.718890 2439 raft_consensus.cc:515] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:47.719508 2439 leader_election.cc:290] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:47.715433 763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.104.131:34317 (request call id 274) took 1721 ms. Trace:
I20260202 02:52:47.719722 763 rpcz_store.cc:276] 0202 02:52:45.993997 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.994199 (+ 202us) service_pool.cc:225] Handling call
0202 02:52:47.715421 (+1721222us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.423904 1028 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49890 (request call id 703) took 16927 ms. Trace:
I20260202 02:52:47.720508 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76d3e799f5cb46638b289d06fca2f369. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:47.426607 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16b2d80274a54f1193a3f1746c278252. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:47.417429 2501 rpcz_store.cc:276] 0202 02:52:30.345065 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.345124 (+ 59us) service_pool.cc:225] Handling call
0202 02:52:45.796392 (+15451268us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:30.345569 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0202 02:52:30.345697 (+ 128us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0202 02:52:30.345707 (+ 10us) alter_schema_op.cc:66] Acquired schema lock
0202 02:52:30.345715 (+ 8us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0202 02:52:30.345768 (+ 53us) alter_schema_op.cc:128] START. Timestamp: 1770000750345675
0202 02:52:30.345772 (+ 4us) op_driver.cc:348] REPLICATION: starting
0202 02:52:30.577450 (+231678us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x6160000b5cb8. stack: 00007fec53205a86 00007fec53205840 00007fec521e03ad 0000000000984f94 0000000000984f5d 000000000098492a 00007fec5f009685 00007fec5f0edf1b 00007fec5f0c8f18 00007fec5f0c87c5 00007fec604ab875 00007fec604aa399 00007fec604b27c0 00007fec604b25cd 0000000000a8970c 00007fec5324bdaf
0202 02:52:30.577641 (+ 191us) log.cc:844] Serialized 173 byte log entry
0202 02:52:36.533901 (+5956260us) op_driver.cc:464] REPLICATION: finished
0202 02:52:36.567737 (+ 33836us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0202 02:52:45.672975 (+9105238us) log.cc:844] Serialized 5 byte log entry
0202 02:52:45.789665 (+116690us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:45.790054 (+ 389us) log.cc:844] Serialized 14 byte log entry
0202 02:52:45.796030 (+ 5976us) alter_schema_op.cc:73] Released schema lock
0202 02:52:45.796039 (+ 9us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":33732,"prepare.queue_time_us":65,"prepare.run_cpu_time_us":7393,"prepare.run_wall_time_us":385775,"replication_time_us":6188080,"spinlock_wait_cycles":532311936,"thread_start_us":316740,"threads_started":3,"wal-append.queue_time_us":254500,"wal-append.run_cpu_time_us":1785,"wal-append.run_wall_time_us":2016843}]]}
I20260202 02:52:47.420295 1173 tablet_bootstrap.cc:654] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:47.722985 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 912ac4044b5645caa7eda2a565c5b8bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:47.426311 1026 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49890 (request call id 679) took 17162 ms. Trace:
W20260202 02:52:47.484570 986 kernel_stack_watchdog.cc:198] Thread 999 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 299ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fec56842980 (unknown) at ??:0
@ 0x7fec5683c9c6 __GI___pthread_rwlock_wrlock at ??:0
@ 0x7fec504b5904 google::LogMessage::Flush() at ??:0
@ 0x7fec504b5a11 google::LogMessage::~LogMessage() at ??:0
@ 0x7fec5a137604 kudu::rpc::Proxy::AsyncRequest()::$_1::operator()() at ??:0
@ 0x7fec5a136ebe std::_Function_handler<>::_M_invoke() at ??:0
@ 0xa8970d std::function<>::operator()() at ??:?
@ 0x7fec5a122f10 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7fec5a124602 kudu::rpc::OutboundCall::SetTimedOut() at ??:0
@ 0x7fec5a0ba453 kudu::rpc::Connection::HandleOutboundCallTimeout() at ??:0
@ 0x7fec5a0b9fc6 kudu::rpc::Connection::CallAwaitingResponse::HandleTimeout() at ??:0
@ 0x7fec5a0d2045 ev::base<>::method_thunk<>() at ??:0
@ 0x7fec519dcd5e ev_invoke_pending at ??:0
@ 0x7fec5a13d765 kudu::rpc::ReactorThread::InvokePendingCb() at ??:0
@ 0x7fec519ddca6 ev_run at ??:0
@ 0x7fec5a14d361 ev::loop_ref::run() at ??:0
I20260202 02:52:47.412281 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 518) took 21273 ms. Trace:
I20260202 02:52:47.723729 1026 rpcz_store.cc:276] 0202 02:52:30.264178 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.267997 (+ 3819us) service_pool.cc:225] Handling call
0202 02:52:30.592624 (+324627us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.592656 (+ 32us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:47.419231 (+16826575us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:47.426109 (+ 6878us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:47.426288 (+ 179us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.723839 1180 rpcz_store.cc:276] 0202 02:52:26.138720 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.143242 (+ 4522us) service_pool.cc:225] Handling call
0202 02:52:47.412212 (+21268970us) ts_tablet_manager.cc:1151] Deleting tablet 1116872cc25a4ab5b1a3a6bbba1f0da7
0202 02:52:47.412218 (+ 6us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:47.412271 (+ 53us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":21268394}
I20260202 02:52:47.724220 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 521) took 21564 ms. Trace:
I20260202 02:52:47.724188 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 15366555b90a408c9f46222b44e7efa6 (DEFAULT_TABLE table=default.Table_46283f6baa2d4a9d9fdb8c1e501bcf89 [id=1cd4342c241a4072a65393b0be0c26a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:47.724496 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f2627dcb9904589967407c3a8756a33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:47.724550 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15366555b90a408c9f46222b44e7efa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:47.725028 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6750a1212ceb4e6e82e4ffa93caab590. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:47.724287 1180 rpcz_store.cc:276] 0202 02:52:26.160143 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.160200 (+ 57us) service_pool.cc:225] Handling call
0202 02:52:47.724093 (+21563893us) ts_tablet_manager.cc:1151] Deleting tablet 8ab5c6fbec394b9bab322a197181a618
0202 02:52:47.724105 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:47.724204 (+ 99us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":21563620}
I20260202 02:52:47.725265 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 520) took 21568 ms. Trace:
I20260202 02:52:47.725399 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 461ecb9ee90f4e94b12934c56de475a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:47.725353 1180 rpcz_store.cc:276] 0202 02:52:26.156638 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.165851 (+ 9213us) service_pool.cc:225] Handling call
0202 02:52:47.725152 (+21559301us) ts_tablet_manager.cc:1151] Deleting tablet f6c2bc531d194a4b8f3062c9b40e3da6
0202 02:52:47.725162 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:47.725251 (+ 89us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":21559061}
I20260202 02:52:47.436699 745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42950 (request call id 700) took 16892 ms. Trace:
I20260202 02:52:47.725570 1180 tablet_replica.cc:333] T ef073880898049e2be93d2cb6dc0c0bb P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:47.441877 756 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42950 (request call id 713) took 16896 ms. Trace:
I20260202 02:52:46.492072 2482 consensus_queue.cc:1048] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:47.725730 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32c3fece7c2145fb86bbc11c4c221f27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:47.725669 756 rpcz_store.cc:276] 0202 02:52:30.545725 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.621249 (+ 75524us) service_pool.cc:225] Handling call
0202 02:52:35.768617 (+5147368us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:35.768656 (+ 39us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:47.410095 (+11641439us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:47.441486 (+ 31391us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:47.441847 (+ 361us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.453508 740 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42950 (request call id 705) took 16908 ms. Trace:
I20260202 02:52:47.725847 740 rpcz_store.cc:276] 0202 02:52:30.545067 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.564710 (+ 19643us) service_pool.cc:225] Handling call
0202 02:52:30.565066 (+ 356us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.565089 (+ 23us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:47.423646 (+16858557us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:47.453279 (+ 29633us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:47.453480 (+ 201us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260202 02:52:47.461215 714 connection.cc:232] RPC call timeout handler was delayed by -6.8553233800000157s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.33610000000000007s for a response
W20260202 02:52:47.726002 714 connection.cc:232] RPC call timeout handler was delayed by -6.8274233800000275s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3392s for a response
W20260202 02:52:47.726063 714 connection.cc:232] RPC call timeout handler was delayed by -5.8995233800000051s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.4423s for a response
I20260202 02:52:47.725629 745 rpcz_store.cc:276] 0202 02:52:30.544604 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.564024 (+ 19420us) service_pool.cc:225] Handling call
0202 02:52:30.564497 (+ 473us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.564520 (+ 23us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:47.423628 (+16859108us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:47.436300 (+ 12672us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:47.436648 (+ 348us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.726854 1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16b2d80274a54f1193a3f1746c278252. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:47.727181 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 361c0c3832614ec78695c28720b841e0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 361c0c3832614ec78695c28720b841e0 already in progress: creating tablet
W20260202 02:52:47.727427 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 7f9dfac19c04475aadb45e95a6474eb4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7f9dfac19c04475aadb45e95a6474eb4 already in progress: creating tablet
W20260202 02:52:47.727638 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet a0b5d82e5fb94b60b4aa324b978a5378 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a0b5d82e5fb94b60b4aa324b978a5378 already in progress: creating tablet
W20260202 02:52:47.727803 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet ac2ab12f5040488aa306cb69b107b485 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ac2ab12f5040488aa306cb69b107b485 already in progress: creating tablet
I20260202 02:52:47.728129 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 94bada52db6a47179415d302b541d01f (table default.Table_086abcdd67e94b38ac7f5d98777f5c6b [id=32fe1d8734de41cfae07a4662814987e]) successfully deleted
W20260202 02:52:47.728314 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 3706562a22634eb6b72e36bb6a2d1ae5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3706562a22634eb6b72e36bb6a2d1ae5 already in progress: creating tablet
I20260202 02:52:47.728511 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9636b45987f34d16b717a8d918cc4bd6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:47.728593 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 474df41e4fe24a818fa45a4ce84d5474 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 474df41e4fe24a818fa45a4ce84d5474 already in progress: creating tablet
W20260202 02:52:47.728816 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 7921daa000f54c8dbed04473e21cf0a5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7921daa000f54c8dbed04473e21cf0a5 already in progress: creating tablet
W20260202 02:52:47.729038 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 93f958924885457ab15f2aac5509e0aa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 93f958924885457ab15f2aac5509e0aa already in progress: creating tablet
W20260202 02:52:47.729205 513 catalog_manager.cc:4952] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet f93e8d3c4faf48eaa98c3cbf88d585e2 because the tablet was not found. No further retry: Not found: Tablet not found: f93e8d3c4faf48eaa98c3cbf88d585e2
W20260202 02:52:47.729385 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 47e50559a7c04ae080fcc67342c631c9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 47e50559a7c04ae080fcc67342c631c9 already in progress: creating tablet
W20260202 02:52:47.729535 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 5e20842de8cb4355b732685605c88834 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5e20842de8cb4355b732685605c88834 already in progress: creating tablet
W20260202 02:52:47.729702 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 0ddc726efd3f49489092919c9c6e01fb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0ddc726efd3f49489092919c9c6e01fb already in progress: creating tablet
I20260202 02:52:47.729729 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c438e69c40c24655b37f2e73b4b3b845. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:47.730659 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 1116872cc25a4ab5b1a3a6bbba1f0da7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1116872cc25a4ab5b1a3a6bbba1f0da7 already in progress: creating tablet
W20260202 02:52:47.730923 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet f6c2bc531d194a4b8f3062c9b40e3da6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f6c2bc531d194a4b8f3062c9b40e3da6 already in progress: creating tablet
I20260202 02:52:47.701994 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "16b47a2831f54d809ed0b16deaec7e3d" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:47.712175 768 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cb634d4bae554ef3b39eeaa9958adc6c" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:47.731345 779 raft_consensus.cc:2468] T 16b47a2831f54d809ed0b16deaec7e3d P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
I20260202 02:52:47.731621 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f2627dcb9904589967407c3a8756a33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:47.731954 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 6d475fa5186243229c1db0fbdc531f4e (DEFAULT_TABLE table=default.Table_caf9a47c30ad4c6293b84e85cbde60eb [id=68f222efa4cf43b3804c194400d89112]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:47.732262 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d475fa5186243229c1db0fbdc531f4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:47.701088 2445 leader_election.cc:290] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:47.720409 1028 rpcz_store.cc:276] 0202 02:52:30.496603 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.687603 (+191000us) service_pool.cc:225] Handling call
0202 02:52:35.768349 (+5080746us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:35.768367 (+ 18us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:47.421258 (+11652891us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:47.423677 (+ 2419us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:47.423878 (+ 201us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.736474 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdd72f995a704520a5fb1c24a53ed252. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:47.736630 1028 tablet_service.cc:1552] Processing DeleteTablet for tablet f94f25f4059a43b8babedb6b45a08a19 with delete_type TABLET_DATA_DELETED (replaced by tablet 76d3e799f5cb46638b289d06fca2f369) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:47.711364 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c584f3163f1b45789492ccd27b7ad609. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:47.699366 857 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.104.129:39403 timed out after 3.206s (SENT)
I20260202 02:52:47.737389 1180 raft_consensus.cc:2243] T ef073880898049e2be93d2cb6dc0c0bb P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:47.737596 1180 raft_consensus.cc:2272] T ef073880898049e2be93d2cb6dc0c0bb P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:47.738155 1180 ts_tablet_manager.cc:1916] T ef073880898049e2be93d2cb6dc0c0bb P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:47.741645 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ffadda20dd3431b84b2657485e7dec7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:47.743018 894 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41736 (request call id 721) took 17110 ms. Trace:
I20260202 02:52:47.743147 894 rpcz_store.cc:276] 0202 02:52:30.632638 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.675855 (+15043217us) service_pool.cc:225] Handling call
0202 02:52:45.755097 (+ 79242us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:45.755133 (+ 36us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:47.696107 (+1940974us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:47.742810 (+ 46703us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:47.742974 (+ 164us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.744541 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b7880cb051c24bc998003f0effed3cf8" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:47.712373 764 raft_consensus.cc:3060] T 26b890ed87c3471b9fc0b172c90e0a11 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:47.747366 1031 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49890 (request call id 725) took 1796 ms. Trace:
I20260202 02:52:47.747455 1031 rpcz_store.cc:276] 0202 02:52:45.951379 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.951463 (+ 84us) service_pool.cc:225] Handling call
0202 02:52:47.688054 (+1736591us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:47.688082 (+ 28us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:47.744561 (+ 56479us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:47.747173 (+ 2612us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:47.747341 (+ 168us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.751232 1882 raft_consensus.cc:2243] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:47.751407 1882 raft_consensus.cc:2272] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:47.751902 1882 ts_tablet_manager.cc:1916] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:47.751940 1032 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49890 (request call id 723) took 1809 ms. Trace:
I20260202 02:52:47.752004 1032 rpcz_store.cc:276] 0202 02:52:45.942834 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.942897 (+ 63us) service_pool.cc:225] Handling call
0202 02:52:47.430036 (+1487139us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:47.430060 (+ 24us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:47.749463 (+319403us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:47.751780 (+ 2317us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:47.751924 (+ 144us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260202 02:52:47.756361 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet c977cd5aa66a4f6c8eeb37bf67f18fa7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c977cd5aa66a4f6c8eeb37bf67f18fa7 already in progress: creating tablet
W20260202 02:52:47.756613 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 940e656cd34e4e53896fda6d38132b6e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 940e656cd34e4e53896fda6d38132b6e already in progress: creating tablet
W20260202 02:52:47.756810 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 85f754f006d6432bb1cba87043903ac2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 85f754f006d6432bb1cba87043903ac2 already in progress: creating tablet
I20260202 02:52:47.756938 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdd72f995a704520a5fb1c24a53ed252. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:47.756989 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 7b45a2e6b4b14d3fa96a7131afb16208 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7b45a2e6b4b14d3fa96a7131afb16208 already in progress: creating tablet
W20260202 02:52:47.757169 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 25f1d698c4ee4be7a33380184b5e2b4d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 25f1d698c4ee4be7a33380184b5e2b4d already in progress: creating tablet
I20260202 02:52:47.759945 755 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42950 (request call id 716) took 17214 ms. Trace:
I20260202 02:52:47.838613 521 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet dd8f36d7a82f47adae1bd3c1f8628b6e
I20260202 02:52:47.850731 2372 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [term 2 FOLLOWER]: Replica starting. Triggering 4 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } }
I20260202 02:52:47.763860 1037 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49890 (request call id 726) took 1810 ms. Trace:
I20260202 02:52:47.934531 1037 rpcz_store.cc:276] 0202 02:52:45.953252 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.953309 (+ 57us) service_pool.cc:225] Handling call
0202 02:52:47.689041 (+1735732us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:47.689067 (+ 26us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:47.747009 (+ 57942us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:47.763677 (+ 16668us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:47.763835 (+ 158us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.767804 1175 tablet_bootstrap.cc:654] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:47.842216 525 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet abfe7f76b2424c4d9ab07b29b66ec103
I20260202 02:52:47.858163 526 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:47430 (request call id 826) took 1969 ms. Trace:
I20260202 02:52:47.859550 527 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:47430 (request call id 830) took 1962 ms. Trace:
W20260202 02:52:47.860580 519 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 528d4269f91a44679acfad16e9fbbe32 consensus state has no leader
I20260202 02:52:47.887270 522 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:47430 (request call id 827) took 1997 ms. Trace:
I20260202 02:52:47.890893 520 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:47430 (request call id 828) took 1992 ms. Trace:
I20260202 02:52:47.891911 528 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:47430 (request call id 829) took 1993 ms. Trace:
I20260202 02:52:47.892019 524 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 842e54adae2a452a98521de24326eaff
I20260202 02:52:47.892601 523 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 79c73f4a92044bafb5b8384229cb3a4d
I20260202 02:52:47.760040 755 rpcz_store.cc:276] 0202 02:52:30.545948 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.643327 (+ 97379us) service_pool.cc:225] Handling call
0202 02:52:35.771625 (+5128298us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:35.771653 (+ 28us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:47.723614 (+11951961us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:47.759743 (+ 36129us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:47.759921 (+ 178us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.772181 746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42950 (request call id 717) took 17226 ms. Trace:
I20260202 02:52:47.778610 759 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42950 (request call id 712) took 17232 ms. Trace:
I20260202 02:52:47.765379 1042 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49890 (request call id 683) took 17500 ms. Trace:
I20260202 02:52:47.936935 759 rpcz_store.cc:276] 0202 02:52:30.545647 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.573333 (+ 27686us) service_pool.cc:225] Handling call
0202 02:52:30.573699 (+ 366us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.573724 (+ 25us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:47.775774 (+17202050us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:47.778386 (+ 2612us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:47.778582 (+ 196us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.936967 1042 rpcz_store.cc:276] 0202 02:52:30.264537 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.266115 (+ 1578us) service_pool.cc:225] Handling call
0202 02:52:30.576363 (+310248us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.576411 (+ 48us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:47.719336 (+17142925us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:47.765193 (+ 45857us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:47.765362 (+ 169us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.782567 758 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42950 (request call id 709) took 17237 ms. Trace:
I20260202 02:52:47.805262 896 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41736 (request call id 700) took 17264 ms. Trace:
I20260202 02:52:47.823985 1180 ts_tablet_manager.cc:1929] T ef073880898049e2be93d2cb6dc0c0bb P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:47.937201 758 rpcz_store.cc:276] 0202 02:52:30.545428 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.570290 (+ 24862us) service_pool.cc:225] Handling call
0202 02:52:30.570612 (+ 322us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.570631 (+ 19us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:47.771879 (+17201248us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:47.782364 (+ 10485us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:47.782531 (+ 167us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.769521 1029 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49890 (request call id 724) took 1824 ms. Trace:
I20260202 02:52:47.937356 1180 log.cc:1199] T ef073880898049e2be93d2cb6dc0c0bb P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/ef073880898049e2be93d2cb6dc0c0bb
I20260202 02:52:47.937270 896 rpcz_store.cc:276] 0202 02:52:30.541165 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.546964 (+ 5799us) service_pool.cc:225] Handling call
0202 02:52:30.594372 (+ 47408us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.594392 (+ 20us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:47.739715 (+17145323us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:47.804988 (+ 65273us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:47.805199 (+ 211us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.814412 888 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41736 (request call id 712) took 17208 ms. Trace:
I20260202 02:52:47.937381 1029 rpcz_store.cc:276] 0202 02:52:45.945447 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.945522 (+ 75us) service_pool.cc:225] Handling call
0202 02:52:47.687250 (+1741728us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:47.687279 (+ 29us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:47.734721 (+ 47442us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:47.769316 (+ 34595us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:47.769495 (+ 179us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.937539 888 rpcz_store.cc:276] 0202 02:52:30.605793 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.606673 (+ 880us) service_pool.cc:225] Handling call
0202 02:52:35.769176 (+5162503us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:35.769197 (+ 21us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:47.810823 (+12041626us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:47.814169 (+ 3346us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:47.814362 (+ 193us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.770618 1036 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49890 (request call id 704) took 17272 ms. Trace:
I20260202 02:52:47.936888 746 rpcz_store.cc:276] 0202 02:52:30.546010 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.644006 (+ 97996us) service_pool.cc:225] Handling call
0202 02:52:35.770584 (+5126578us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:35.770607 (+ 23us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:47.735411 (+11964804us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:47.772000 (+ 36589us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:47.772160 (+ 160us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.937753 1180 ts_tablet_manager.cc:1950] T ef073880898049e2be93d2cb6dc0c0bb P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:47.937728 1036 rpcz_store.cc:276] 0202 02:52:30.498295 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.698462 (+200167us) service_pool.cc:225] Handling call
0202 02:52:35.767838 (+5069376us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:35.767866 (+ 28us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:47.729649 (+11961783us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:47.770444 (+ 40795us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:47.770599 (+ 155us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.815213 891 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41736 (request call id 696) took 17274 ms. Trace:
I20260202 02:52:47.937980 891 rpcz_store.cc:276] 0202 02:52:30.540829 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.544415 (+ 3586us) service_pool.cc:225] Handling call
0202 02:52:30.594060 (+ 49645us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.594105 (+ 45us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:47.775436 (+17181331us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:47.814983 (+ 39547us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:47.815167 (+ 184us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.771881 1033 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49890 (request call id 705) took 17272 ms. Trace:
I20260202 02:52:47.938180 1033 rpcz_store.cc:276] 0202 02:52:30.499074 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.701589 (+202515us) service_pool.cc:225] Handling call
0202 02:52:35.768022 (+5066433us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:35.768044 (+ 22us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:47.734256 (+11966212us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:47.771670 (+ 37414us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:47.771859 (+ 189us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.938243 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 526) took 21721 ms. Trace:
I20260202 02:52:47.779553 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d06ebf3fe7fd4a59ad08d6d1ba3d85ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:47.938313 1180 rpcz_store.cc:276] 0202 02:52:26.216872 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.219674 (+ 2802us) service_pool.cc:225] Handling call
0202 02:52:47.725531 (+21505857us) ts_tablet_manager.cc:1151] Deleting tablet ef073880898049e2be93d2cb6dc0c0bb
0202 02:52:47.725538 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:47.747012 (+ 21474us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:47.823859 (+ 76847us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:47.938223 (+114364us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":11579,"tablet-delete.queue_time_us":21505480}
I20260202 02:52:47.938578 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f87ee8d6ff9042c08ba2e32724bea720. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:47.781221 1038 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49890 (request call id 722) took 1838 ms. Trace:
I20260202 02:52:47.938710 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 525) took 21721 ms. Trace:
I20260202 02:52:47.938699 1038 rpcz_store.cc:276] 0202 02:52:45.942671 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.942948 (+ 277us) service_pool.cc:225] Handling call
0202 02:52:47.686143 (+1743195us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:47.686181 (+ 38us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:47.750740 (+ 64559us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:47.781023 (+ 30283us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:47.781202 (+ 179us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":15656}
I20260202 02:52:47.938778 1180 rpcz_store.cc:276] 0202 02:52:26.216763 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.227860 (+ 11097us) service_pool.cc:225] Handling call
0202 02:52:47.938550 (+21710690us) ts_tablet_manager.cc:1151] Deleting tablet 3191a80374724effa8389adc842e3448
0202 02:52:47.938560 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:47.938693 (+ 133us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":21710333}
I20260202 02:52:47.939080 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 524) took 21722 ms. Trace:
I20260202 02:52:47.939145 1180 rpcz_store.cc:276] 0202 02:52:26.216635 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.250916 (+ 34281us) service_pool.cc:225] Handling call
0202 02:52:47.938978 (+21688062us) ts_tablet_manager.cc:1151] Deleting tablet 2616e1bbce4049dbb584550ccc150288
0202 02:52:47.938987 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:47.939065 (+ 78us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":21687744}
I20260202 02:52:47.939416 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 533) took 21639 ms. Trace:
I20260202 02:52:47.939472 1180 rpcz_store.cc:276] 0202 02:52:26.300485 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.300724 (+ 239us) service_pool.cc:225] Handling call
0202 02:52:47.939324 (+21638600us) ts_tablet_manager.cc:1151] Deleting tablet 7ea7a5d8a92b4ac0903483627258ae66
0202 02:52:47.939333 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:47.939402 (+ 69us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":21638325}
I20260202 02:52:47.939744 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 534) took 21639 ms. Trace:
I20260202 02:52:47.939798 1180 rpcz_store.cc:276] 0202 02:52:26.300640 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.300862 (+ 222us) service_pool.cc:225] Handling call
0202 02:52:47.939652 (+21638790us) ts_tablet_manager.cc:1151] Deleting tablet 8ae6390a856d4c838e0689cd0c6c5c92
0202 02:52:47.939660 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:47.939731 (+ 71us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":21638600}
I20260202 02:52:47.940038 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 536) took 21616 ms. Trace:
I20260202 02:52:47.940095 1180 rpcz_store.cc:276] 0202 02:52:26.323965 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.324017 (+ 52us) service_pool.cc:225] Handling call
0202 02:52:47.939972 (+21615955us) ts_tablet_manager.cc:1151] Deleting tablet c63a670d5c5e489897bc6dbb142b844d
0202 02:52:47.939980 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:47.940026 (+ 46us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":21615722}
W20260202 02:52:47.824172 714 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 80860.2us
I20260202 02:52:47.941629 745 tablet_service.cc:1505] Processing CreateTablet for tablet 32c3fece7c2145fb86bbc11c4c221f27 (DEFAULT_TABLE table=default.Table_31ed23ec1b894b32953e20125e57d70a [id=0fb5920dae13430c81292070d04bb203]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:47.941872 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8e489f59d1345549e29c4fa6e59c1c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:47.942029 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32c3fece7c2145fb86bbc11c4c221f27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:47.942265 743 tablet_service.cc:1505] Processing CreateTablet for tablet 9636b45987f34d16b717a8d918cc4bd6 (DEFAULT_TABLE table=default.Table_38b5d3fdb2c748cba3a6aaad03bdb766 [id=9cb73f418ae84400871b2e81ff676c8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:47.942603 743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9636b45987f34d16b717a8d918cc4bd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:47.817266 899 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41736 (request call id 717) took 17184 ms. Trace:
I20260202 02:52:47.943486 899 rpcz_store.cc:276] 0202 02:52:30.632381 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.677693 (+ 45312us) service_pool.cc:225] Handling call
0202 02:52:35.772870 (+5095177us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:35.772901 (+ 31us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:47.710214 (+11937313us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:47.817052 (+106838us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:47.817209 (+ 157us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260202 02:52:47.828063 715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 110968us
W20260202 02:52:47.944008 715 leader_election.cc:343] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Not found: Tablet not found: 94bada52db6a47179415d302b541d01f
W20260202 02:52:47.944190 715 leader_election.cc:343] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Not found: Tablet not found: 8774c6bc0fe9414ea2f17ed859d9392a
W20260202 02:52:47.823170 999 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 1.72939e+06us
I20260202 02:52:47.829702 1174 tablet_bootstrap.cc:492] T 33150e1456144b8e9c64cc2a2ac6dcca P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
W20260202 02:52:47.945008 999 connection.cc:232] RPC call timeout handler was delayed by -1.3566297520000035s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260202 02:52:47.841004 1173 tablet_bootstrap.cc:492] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:47.946067 1173 ts_tablet_manager.cc:1403] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.527s user 0.013s sys 0.001s
I20260202 02:52:47.946393 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 532681264d6b4ac8940f5bc79eab3d1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:47.946887 1173 raft_consensus.cc:359] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:47.947045 1173 raft_consensus.cc:385] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:47.947095 1173 raft_consensus.cc:740] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
W20260202 02:52:47.820737 859 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 117612us
I20260202 02:52:47.947279 1173 consensus_queue.cc:260] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:47.947623 859 leader_election.cc:304] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:47.947978 859 leader_election.cc:304] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
W20260202 02:52:47.948210 859 leader_election.cc:343] T f94f25f4059a43b8babedb6b45a08a19 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:47.948463 859 leader_election.cc:304] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
W20260202 02:52:47.948725 859 leader_election.cc:343] T a0b5d82e5fb94b60b4aa324b978a5378 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:47.941664 757 tablet_service.cc:1505] Processing CreateTablet for tablet 912ac4044b5645caa7eda2a565c5b8bb (DEFAULT_TABLE table=default.Table_0ee7c010ec144c2e9d633447e4d84dcf [id=812637d0b1da4789850022a699e6bbd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:47.962585 526 rpcz_store.cc:276] 0202 02:52:45.888452 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:47.413228 (+1524776us) service_pool.cc:225] Handling call
0202 02:52:47.413579 (+ 351us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0202 02:52:47.413762 (+ 183us) catalog_manager.cc:3708] Apply alter schema
0202 02:52:47.430705 (+ 16943us) catalog_manager.cc:3740] Acquired catalog manager lock
0202 02:52:47.430869 (+ 164us) catalog_manager.cc:3898] Updating metadata on disk
0202 02:52:47.857501 (+426632us) catalog_manager.cc:3929] Committing alterations to in-memory state
0202 02:52:47.858102 (+ 601us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:47.480694 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:47.481432 (+ 738us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:47.481557 (+ 125us) write_op.cc:435] Acquired schema lock
0202 02:52:47.481561 (+ 4us) tablet.cc:660] Decoding operations
0202 02:52:47.482152 (+ 591us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:47.482284 (+ 132us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:47.482290 (+ 6us) tablet.cc:683] Acquiring locks for 1 operations
0202 02:52:47.482728 (+ 438us) tablet.cc:699] Row locks acquired
0202 02:52:47.482730 (+ 2us) write_op.cc:260] PREPARE: finished
0202 02:52:47.483010 (+ 280us) write_op.cc:270] Start()
0202 02:52:47.483283 (+ 273us) write_op.cc:276] Timestamp: P: 1770000767482613 usec, L: 0
0202 02:52:47.483289 (+ 6us) op_driver.cc:348] REPLICATION: starting
0202 02:52:47.484740 (+ 1451us) log.cc:844] Serialized 405 byte log entry
0202 02:52:47.852039 (+367299us) op_driver.cc:464] REPLICATION: finished
0202 02:52:47.856534 (+ 4495us) write_op.cc:301] APPLY: starting
0202 02:52:47.856571 (+ 37us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:47.856623 (+ 52us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:47.856672 (+ 49us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:47.856747 (+ 75us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:47.856790 (+ 43us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0202 02:52:47.856798 (+ 8us) write_op.cc:312] APPLY: finished
0202 02:52:47.856909 (+ 111us) log.cc:844] Serialized 23 byte log entry
0202 02:52:47.857330 (+ 421us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:47.857360 (+ 30us) write_op.cc:454] Released schema lock
0202 02:52:47.857400 (+ 40us) write_op.cc:341] FINISH: Updating metrics
0202 02:52:47.857566 (+ 166us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:47.857571 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":4352,"apply.run_cpu_time_us":657,"apply.run_wall_time_us":995,"num_ops":1,"prepare.queue_time_us":49129,"prepare.run_cpu_time_us":1366,"prepare.run_wall_time_us":4896,"replication_time_us":368666}]]}
I20260202 02:52:48.033320 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 912ac4044b5645caa7eda2a565c5b8bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:47.971192 2372 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dab2abef14445d688ce1335b1f9b2f8, State: Initialized, Role: FOLLOWER
I20260202 02:52:47.847370 998 leader_election.cc:304] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:48.034453 2372 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 430, Last appended: 2.434, Last appended by leader: 434, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } }
W20260202 02:52:48.034705 998 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 187384us
I20260202 02:52:48.035142 998 leader_election.cc:304] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:48.035583 998 leader_election.cc:304] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:48.035823 998 leader_election.cc:304] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:47.948889 859 leader_election.cc:304] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:48.036478 2482 raft_consensus.cc:2804] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260202 02:52:47.827749 857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 128399us
I20260202 02:52:47.834360 889 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41736 (request call id 710) took 17228 ms. Trace:
I20260202 02:52:48.036579 2482 raft_consensus.cc:493] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
W20260202 02:52:48.036564 859 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 87669us
I20260202 02:52:48.036648 2482 raft_consensus.cc:3060] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Advancing to term 2
W20260202 02:52:48.036686 857 connection.cc:232] RPC call timeout handler was delayed by -13.895803661000002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260202 02:52:48.036922 859 leader_election.cc:343] T 474df41e4fe24a818fa45a4ce84d5474 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:48.037159 2450 raft_consensus.cc:2804] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:48.037328 2450 raft_consensus.cc:493] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:47.839550 1182 ts_tablet_manager.cc:1929] T 474df41e4fe24a818fa45a4ce84d5474 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:48.037549 1182 log.cc:1199] T 474df41e4fe24a818fa45a4ce84d5474 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/474df41e4fe24a818fa45a4ce84d5474
I20260202 02:52:48.037938 1182 ts_tablet_manager.cc:1950] T 474df41e4fe24a818fa45a4ce84d5474 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:48.038295 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 682) took 18523 ms. Trace:
I20260202 02:52:48.038388 1182 rpcz_store.cc:276] 0202 02:52:29.514810 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.514860 (+ 50us) service_pool.cc:225] Handling call
0202 02:52:35.760060 (+6245200us) ts_tablet_manager.cc:1151] Deleting tablet 474df41e4fe24a818fa45a4ce84d5474
0202 02:52:35.760071 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:45.757947 (+9997876us) spinlock_profiling.cc:243] Waited 9.77 s on lock 0x6160000f4c54. stack: 00007fb35529ca86 00007fb35529c840 00007fb3542773ad 0000000000984f94 0000000000984f5d 000000000098492a 00007fb36118a1e0 00007fb3624d80b3 00007fb365098757 00007fb3650cb744 00007fb3650ac240 00007fb3650abfcd 0000000000a8970c 00007fb3552e2daf 00007fb3552e60cb 00007fb3552e5f48
0202 02:52:47.804749 (+2046802us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:47.839391 (+ 34642us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.038253 (+198862us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":22461529856,"tablet-delete.queue_time_us":6244966}
W20260202 02:52:47.944314 715 leader_election.cc:343] T fe9ea4c0502f4976b440465992f2ea2c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Not found: Tablet not found: fe9ea4c0502f4976b440465992f2ea2c
I20260202 02:52:48.038801 1182 tablet_replica.cc:333] T 1f379a29e2454f689d68f5951837512b P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
W20260202 02:52:48.038872 715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 94548.4us
I20260202 02:52:48.038992 1182 raft_consensus.cc:2243] T 1f379a29e2454f689d68f5951837512b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:48.039160 1182 raft_consensus.cc:2272] T 1f379a29e2454f689d68f5951837512b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:48.039175 715 leader_election.cc:343] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Not found: Tablet not found: 38fdbc6df7ec40b9afe4e0dfe0119e3e
I20260202 02:52:48.039186 2482 raft_consensus.cc:515] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:48.039330 715 leader_election.cc:343] T ef073880898049e2be93d2cb6dc0c0bb P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Not found: Tablet not found: ef073880898049e2be93d2cb6dc0c0bb
I20260202 02:52:48.039710 1182 ts_tablet_manager.cc:1916] T 1f379a29e2454f689d68f5951837512b P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:47.963253 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet ef073880898049e2be93d2cb6dc0c0bb (table default.Table_e8e9b4cdaf434b6cbbfed1a65bff1fc0 [id=78a7fb3a2eed4565b55d5dacb8a3d7df]) successfully deleted
W20260202 02:52:48.040002 513 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 76779.5us
I20260202 02:52:48.040072 2604 raft_consensus.cc:2804] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:48.040148 2604 raft_consensus.cc:493] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:48.040210 2604 raft_consensus.cc:3060] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:48.040203 2441 raft_consensus.cc:493] T 842e54adae2a452a98521de24326eaff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:48.040340 2441 raft_consensus.cc:515] T 842e54adae2a452a98521de24326eaff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:47.827008 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0da286b7f598457ca4d0254a7073cb74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.040966 2441 leader_election.cc:290] T 842e54adae2a452a98521de24326eaff P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:48.041707 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ac2ab12f5040488aa306cb69b107b485" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:48.042433 1050 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "29c996781cb947f3a1aee5dd0a9c4828" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:48.042761 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d8418a9dd0944456b4d4820dde0cde02" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:47.948916 2439 raft_consensus.cc:2804] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260202 02:52:48.042920 2439 raft_consensus.cc:493] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1f442ef85029461091de847118c34c0d)
I20260202 02:52:48.042901 2604 raft_consensus.cc:515] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:48.043035 2439 raft_consensus.cc:3060] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:48.043093 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8bb14bf551f441dcbaaff7057b51b2be" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:48.043329 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fbcb4a522df648ba9542250450c08f1b" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:48.043627 1054 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "16b47a2831f54d809ed0b16deaec7e3d" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:48.043744 2632 raft_consensus.cc:2804] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:48.043910 1058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "70306ed9dec54286885d965f2c012da4" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:48.044183 1056 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e1284fde67bc4d65b067a54c416ccc33" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:48.044472 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f6746ccbf37e477196a9706396d1e3fd" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:48.044759 1053 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "be993c1512b04df4b92cab3195241d84" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:47.955533 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a206159afa84ae19f8c7763a2fe43d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.045624 2439 raft_consensus.cc:515] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:48.046224 2439 leader_election.cc:290] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:48.046789 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1d8143f529b74e6ab779dd695920413c" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:48.036698 889 rpcz_store.cc:276] 0202 02:52:30.605643 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.606430 (+ 787us) service_pool.cc:225] Handling call
0202 02:52:35.768831 (+5162401us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:35.768852 (+ 21us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:47.774855 (+12006003us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:47.833521 (+ 58666us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:47.834333 (+ 812us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:48.048652 1049 raft_consensus.cc:1240] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Rejecting Update request from peer 1521a6d262b7451bbef8debe970dd6b5 for earlier term 1. Current term is 2. Ops: []
I20260202 02:52:48.049003 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c050a2882f764dda8df2a7df6547f149" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:47.944470 1174 ts_tablet_manager.cc:1403] T 33150e1456144b8e9c64cc2a2ac6dcca P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 17.372s user 0.303s sys 0.789s
I20260202 02:52:47.958161 1040 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49890 (request call id 694) took 17501 ms. Trace:
I20260202 02:52:48.049256 1040 rpcz_store.cc:276] 0202 02:52:30.457014 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.457065 (+ 51us) service_pool.cc:225] Handling call
0202 02:52:30.592965 (+135900us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.592993 (+ 28us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:47.955341 (+17362348us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:47.957962 (+ 2621us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:47.958135 (+ 173us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3328}
I20260202 02:52:47.962556 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet abfe7f76b2424c4d9ab07b29b66ec103 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:47 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:47.964686 1033 tablet_service.cc:1552] Processing DeleteTablet for tablet 842e54adae2a452a98521de24326eaff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:47 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:47.964919 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet 79c73f4a92044bafb5b8384229cb3a4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:47 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:47.969113 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet dd8f36d7a82f47adae1bd3c1f8628b6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:47 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:48.050024 1174 raft_consensus.cc:359] T 33150e1456144b8e9c64cc2a2ac6dcca P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:48.050189 1174 raft_consensus.cc:385] T 33150e1456144b8e9c64cc2a2ac6dcca P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:48.050235 1174 raft_consensus.cc:740] T 33150e1456144b8e9c64cc2a2ac6dcca P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:48.050424 1174 consensus_queue.cc:260] T 33150e1456144b8e9c64cc2a2ac6dcca P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:47.965157 519 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:47430 (request call id 825) took 11970 ms. Trace:
I20260202 02:52:47.949184 2443 raft_consensus.cc:2804] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260202 02:52:47.940300 1180 tablet_replica.cc:333] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:47.952353 746 tablet_service.cc:1552] Processing DeleteTablet for tablet 3dcf4f794ec14b2f9d181d015fc437fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:30 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:47.965245 520 rpcz_store.cc:276] 0202 02:52:45.889668 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:47.416874 (+1527206us) service_pool.cc:225] Handling call
0202 02:52:47.431838 (+ 14964us) catalog_manager.cc:2101] Acquired catalog manager lock
0202 02:52:47.432404 (+ 566us) catalog_manager.cc:2147] Created new table and tablet info
0202 02:52:47.882312 (+449908us) catalog_manager.cc:2207] Wrote table and tablets to system table
0202 02:52:47.882445 (+ 133us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0202 02:52:47.882565 (+ 120us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:47.496183 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:47.497032 (+ 849us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:47.497039 (+ 7us) write_op.cc:435] Acquired schema lock
0202 02:52:47.497041 (+ 2us) tablet.cc:660] Decoding operations
0202 02:52:47.497692 (+ 651us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:47.497820 (+ 128us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:47.497826 (+ 6us) tablet.cc:683] Acquiring locks for 2 operations
0202 02:52:47.498749 (+ 923us) tablet.cc:699] Row locks acquired
0202 02:52:47.498756 (+ 7us) write_op.cc:260] PREPARE: finished
0202 02:52:47.499037 (+ 281us) write_op.cc:270] Start()
0202 02:52:47.499201 (+ 164us) write_op.cc:276] Timestamp: P: 1770000767498640 usec, L: 0
0202 02:52:47.499317 (+ 116us) op_driver.cc:348] REPLICATION: starting
0202 02:52:47.500673 (+ 1356us) log.cc:844] Serialized 433 byte log entry
0202 02:52:47.871778 (+371105us) op_driver.cc:464] REPLICATION: finished
0202 02:52:47.881322 (+ 9544us) write_op.cc:301] APPLY: starting
0202 02:52:47.881346 (+ 24us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:47.881388 (+ 42us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:47.881419 (+ 31us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:47.881562 (+ 143us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:47.881602 (+ 40us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0202 02:52:47.881608 (+ 6us) write_op.cc:312] APPLY: finished
0202 02:52:47.881703 (+ 95us) log.cc:844] Serialized 29 byte log entry
0202 02:52:47.882006 (+ 303us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:47.882029 (+ 23us) write_op.cc:454] Released schema lock
0202 02:52:47.882036 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0202 02:52:47.882334 (+ 298us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:47.882345 (+ 11us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":9437,"apply.run_cpu_time_us":764,"apply.run_wall_time_us":978,"num_ops":2,"prepare.queue_time_us":63227,"prepare.run_cpu_time_us":1514,"prepare.run_wall_time_us":5319,"replication_time_us":372423}]]}
I20260202 02:52:47.966034 522 rpcz_store.cc:276] 0202 02:52:45.889564 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:47.414731 (+1525167us) service_pool.cc:225] Handling call
0202 02:52:47.432853 (+ 18122us) catalog_manager.cc:2101] Acquired catalog manager lock
0202 02:52:47.433312 (+ 459us) catalog_manager.cc:2147] Created new table and tablet info
0202 02:52:47.886821 (+453509us) catalog_manager.cc:2207] Wrote table and tablets to system table
0202 02:52:47.887076 (+ 255us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0202 02:52:47.887206 (+ 130us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:47.508373 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:47.509239 (+ 866us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:47.509247 (+ 8us) write_op.cc:435] Acquired schema lock
0202 02:52:47.509250 (+ 3us) tablet.cc:660] Decoding operations
0202 02:52:47.509849 (+ 599us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:47.509980 (+ 131us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:47.509986 (+ 6us) tablet.cc:683] Acquiring locks for 2 operations
0202 02:52:47.510645 (+ 659us) tablet.cc:699] Row locks acquired
0202 02:52:47.510648 (+ 3us) write_op.cc:260] PREPARE: finished
0202 02:52:47.510928 (+ 280us) write_op.cc:270] Start()
0202 02:52:47.511131 (+ 203us) write_op.cc:276] Timestamp: P: 1770000767510530 usec, L: 0
0202 02:52:47.511137 (+ 6us) op_driver.cc:348] REPLICATION: starting
0202 02:52:47.512387 (+ 1250us) log.cc:844] Serialized 433 byte log entry
0202 02:52:47.871916 (+359529us) op_driver.cc:464] REPLICATION: finished
0202 02:52:47.883816 (+ 11900us) write_op.cc:301] APPLY: starting
0202 02:52:47.883831 (+ 15us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:47.883946 (+ 115us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:47.883977 (+ 31us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:47.885877 (+ 1900us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:47.885947 (+ 70us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0202 02:52:47.885986 (+ 39us) write_op.cc:312] APPLY: finished
0202 02:52:47.886127 (+ 141us) log.cc:844] Serialized 29 byte log entry
0202 02:52:47.886594 (+ 467us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:47.886614 (+ 20us) write_op.cc:454] Released schema lock
0202 02:52:47.886621 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0202 02:52:47.886717 (+ 96us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:47.886720 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":11817,"apply.run_cpu_time_us":854,"apply.run_wall_time_us":2930,"mutex_wait_us":5068,"num_ops":2,"prepare.queue_time_us":74451,"prepare.run_cpu_time_us":1418,"prepare.run_wall_time_us":4965,"replication_time_us":360764}]]}
I20260202 02:52:47.964171 527 rpcz_store.cc:276] 0202 02:52:45.897456 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:47.427603 (+1530147us) service_pool.cc:225] Handling call
0202 02:52:47.430858 (+ 3255us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0202 02:52:47.431103 (+ 245us) catalog_manager.cc:3708] Apply alter schema
0202 02:52:47.431306 (+ 203us) catalog_manager.cc:3740] Acquired catalog manager lock
0202 02:52:47.431380 (+ 74us) catalog_manager.cc:3898] Updating metadata on disk
0202 02:52:47.858982 (+427602us) catalog_manager.cc:3929] Committing alterations to in-memory state
0202 02:52:47.859491 (+ 509us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:47.486070 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:47.486808 (+ 738us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:47.486931 (+ 123us) write_op.cc:435] Acquired schema lock
0202 02:52:47.486934 (+ 3us) tablet.cc:660] Decoding operations
0202 02:52:47.487523 (+ 589us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:47.487653 (+ 130us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:47.487661 (+ 8us) tablet.cc:683] Acquiring locks for 1 operations
0202 02:52:47.488096 (+ 435us) tablet.cc:699] Row locks acquired
0202 02:52:47.488099 (+ 3us) write_op.cc:260] PREPARE: finished
0202 02:52:47.488375 (+ 276us) write_op.cc:270] Start()
0202 02:52:47.488538 (+ 163us) write_op.cc:276] Timestamp: P: 1770000767487982 usec, L: 0
0202 02:52:47.488542 (+ 4us) op_driver.cc:348] REPLICATION: starting
0202 02:52:47.489828 (+ 1286us) log.cc:844] Serialized 405 byte log entry
0202 02:52:47.852147 (+362319us) op_driver.cc:464] REPLICATION: finished
0202 02:52:47.857754 (+ 5607us) write_op.cc:301] APPLY: starting
0202 02:52:47.857767 (+ 13us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:47.857839 (+ 72us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:47.857899 (+ 60us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:47.857976 (+ 77us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:47.858039 (+ 63us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0202 02:52:47.858045 (+ 6us) write_op.cc:312] APPLY: finished
0202 02:52:47.858313 (+ 268us) log.cc:844] Serialized 23 byte log entry
0202 02:52:47.858798 (+ 485us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:47.858819 (+ 21us) write_op.cc:454] Released schema lock
0202 02:52:47.858856 (+ 37us) write_op.cc:341] FINISH: Updating metrics
0202 02:52:47.859012 (+ 156us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:47.859017 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":5535,"apply.run_cpu_time_us":622,"apply.run_wall_time_us":1245,"num_ops":1,"prepare.queue_time_us":53999,"prepare.run_cpu_time_us":1328,"prepare.run_wall_time_us":4618,"replication_time_us":363588}]]}
I20260202 02:52:47.965394 528 rpcz_store.cc:276] 0202 02:52:45.889743 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:47.419783 (+1530040us) service_pool.cc:225] Handling call
0202 02:52:47.431952 (+ 12169us) catalog_manager.cc:2101] Acquired catalog manager lock
0202 02:52:47.432485 (+ 533us) catalog_manager.cc:2147] Created new table and tablet info
0202 02:52:47.883508 (+451023us) catalog_manager.cc:2207] Wrote table and tablets to system table
0202 02:52:47.883611 (+ 103us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0202 02:52:47.883703 (+ 92us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:47.501799 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:47.502605 (+ 806us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:47.502724 (+ 119us) write_op.cc:435] Acquired schema lock
0202 02:52:47.502728 (+ 4us) tablet.cc:660] Decoding operations
0202 02:52:47.503329 (+ 601us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:47.503460 (+ 131us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:47.503467 (+ 7us) tablet.cc:683] Acquiring locks for 2 operations
0202 02:52:47.504444 (+ 977us) tablet.cc:699] Row locks acquired
0202 02:52:47.504447 (+ 3us) write_op.cc:260] PREPARE: finished
0202 02:52:47.504872 (+ 425us) write_op.cc:270] Start()
0202 02:52:47.505037 (+ 165us) write_op.cc:276] Timestamp: P: 1770000767504469 usec, L: 0
0202 02:52:47.505156 (+ 119us) op_driver.cc:348] REPLICATION: starting
0202 02:52:47.506938 (+ 1782us) log.cc:844] Serialized 433 byte log entry
0202 02:52:47.871870 (+364932us) op_driver.cc:464] REPLICATION: finished
0202 02:52:47.882479 (+ 10609us) write_op.cc:301] APPLY: starting
0202 02:52:47.882503 (+ 24us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:47.882567 (+ 64us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:47.882618 (+ 51us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:47.882810 (+ 192us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:47.882849 (+ 39us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0202 02:52:47.882858 (+ 9us) write_op.cc:312] APPLY: finished
0202 02:52:47.882941 (+ 83us) log.cc:844] Serialized 29 byte log entry
0202 02:52:47.883324 (+ 383us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:47.883347 (+ 23us) write_op.cc:454] Released schema lock
0202 02:52:47.883355 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0202 02:52:47.883615 (+ 260us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:47.883619 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":10566,"apply.run_cpu_time_us":727,"apply.run_wall_time_us":1028,"num_ops":2,"prepare.queue_time_us":68687,"prepare.run_cpu_time_us":1610,"prepare.run_wall_time_us":6172,"replication_time_us":366697}]]}
I20260202 02:52:47.967430 523 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:47430 (request call id 831) took 2053 ms. Trace:
I20260202 02:52:47.952106 1025 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49890 (request call id 728) took 1059 ms. Trace:
I20260202 02:52:47.967555 524 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:47430 (request call id 823) took 17166 ms. Trace:
I20260202 02:52:47.949306 2430 raft_consensus.cc:2804] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:47.967942 525 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:47430 (request call id 824) took 17166 ms. Trace:
I20260202 02:52:47.968632 521 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:47430 (request call id 822) took 17167 ms. Trace:
I20260202 02:52:48.052299 1025 rpcz_store.cc:276] 0202 02:52:46.892995 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:47.417911 (+524916us) service_pool.cc:225] Handling call
0202 02:52:47.418310 (+ 399us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:47.418333 (+ 23us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:47.949324 (+530991us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:47.951911 (+ 2587us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:47.952085 (+ 174us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:47.944677 747 tablet_service.cc:1505] Processing CreateTablet for tablet 8a206159afa84ae19f8c7763a2fe43d4 (DEFAULT_TABLE table=default.Table_06dc35989d444cccb418b9d2db62721d [id=e5f4256ed9434c52b0b71d84c58c0508]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.052901 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a206159afa84ae19f8c7763a2fe43d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.052908 2443 raft_consensus.cc:493] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1f442ef85029461091de847118c34c0d)
I20260202 02:52:48.053062 2443 raft_consensus.cc:3060] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Advancing to term 3
W20260202 02:52:48.053223 859 leader_election.cc:343] T ac2ab12f5040488aa306cb69b107b485 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:48.053555 859 leader_election.cc:343] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:48.053725 859 leader_election.cc:343] T d8418a9dd0944456b4d4820dde0cde02 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:48.053885 859 leader_election.cc:343] T fbcb4a522df648ba9542250450c08f1b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:48.054849 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "78f122342dd144b48de89902faa0f94f" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:48.055032 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "842e54adae2a452a98521de24326eaff" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:48.055387 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ffadda20dd3431b84b2657485e7dec7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.055697 2443 raft_consensus.cc:515] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:47.952493 759 tablet_service.cc:1552] Processing DeleteTablet for tablet 75a24a1bf54d4c799938a5723ac6fa99 with delete_type TABLET_DATA_DELETED (replaced by tablet 6750a1212ceb4e6e82e4ffa93caab590) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:47.944800 755 tablet_service.cc:1552] Processing DeleteTablet for tablet b8f2b34b4f1e4ff2b2be4ff40aeb0c66 with delete_type TABLET_DATA_DELETED (replaced by tablet 912ac4044b5645caa7eda2a565c5b8bb) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:47.945151 740 tablet_service.cc:1505] Processing CreateTablet for tablet 6b4bfc4df8ab487a924c0e04e97d2c09 (DEFAULT_TABLE table=default.Table_a2c0065f3593412fa5958d59d354f437 [id=ca66b8adfee44b6ab8dd973e36a26597]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:47.974047 1173 ts_tablet_manager.cc:1434] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.028s user 0.004s sys 0.003s
I20260202 02:52:48.056983 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b4bfc4df8ab487a924c0e04e97d2c09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.057019 1173 tablet_bootstrap.cc:492] T 550895b4010f4088b2cc34acff50e03f P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:48.057144 2430 raft_consensus.cc:493] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:48.057271 2430 raft_consensus.cc:3060] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:48.058200 1027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49890 (request call id 731) took 1138 ms. Trace:
I20260202 02:52:48.058290 1027 rpcz_store.cc:276] 0202 02:52:46.919986 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:47.713652 (+793666us) service_pool.cc:225] Handling call
0202 02:52:47.714131 (+ 479us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:47.714153 (+ 22us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:48.050197 (+336044us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.058035 (+ 7838us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:48.058181 (+ 146us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:48.058808 1173 tablet_bootstrap.cc:654] T 550895b4010f4088b2cc34acff50e03f P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:48.059708 2482 leader_election.cc:290] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:48.059784 2430 raft_consensus.cc:515] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:47.945441 758 tablet_service.cc:1552] Processing DeleteTablet for tablet 09a8aa4dd70b4fe2806293d2620e521c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:30 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:48.060277 2604 leader_election.cc:290] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:47.945547 741 tablet_service.cc:1505] Processing CreateTablet for tablet 461ecb9ee90f4e94b12934c56de475a7 (DEFAULT_TABLE table=default.Table_f468fb0750cf421e833dea1a00d62640 [id=88ea44b25a404b15901d1a267de4f03a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.060774 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 461ecb9ee90f4e94b12934c56de475a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:47.985201 1041 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49890 (request call id 681) took 17720 ms. Trace:
I20260202 02:52:48.060894 1041 rpcz_store.cc:276] 0202 02:52:30.264374 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.265481 (+ 1107us) service_pool.cc:225] Handling call
0202 02:52:30.594740 (+329259us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.594764 (+ 24us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:47.956796 (+17362032us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:47.985030 (+ 28234us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:47.985178 (+ 148us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:48.052441 521 rpcz_store.cc:276] 0202 02:52:30.801013 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:47.409105 (+16608092us) service_pool.cc:225] Handling call
0202 02:52:47.409637 (+ 532us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0202 02:52:47.409914 (+ 277us) catalog_manager.cc:2815] Modifying in-memory table state
0202 02:52:47.409984 (+ 70us) catalog_manager.cc:2824] Locking tablets
0202 02:52:47.410167 (+ 183us) catalog_manager.cc:2838] Removing table and tablets from system table
0202 02:52:47.838422 (+428255us) catalog_manager.cc:2856] Removing table from by-name map
0202 02:52:47.838512 (+ 90us) catalog_manager.cc:2872] Committing in-memory state
0202 02:52:47.838559 (+ 47us) catalog_manager.cc:2876] Aborting table tasks
0202 02:52:47.968571 (+130012us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:47.467575 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:47.469310 (+ 1735us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:47.469318 (+ 8us) write_op.cc:435] Acquired schema lock
0202 02:52:47.469435 (+ 117us) tablet.cc:660] Decoding operations
0202 02:52:47.470654 (+ 1219us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:47.470671 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:47.470800 (+ 129us) tablet.cc:683] Acquiring locks for 2 operations
0202 02:52:47.471644 (+ 844us) tablet.cc:699] Row locks acquired
0202 02:52:47.471647 (+ 3us) write_op.cc:260] PREPARE: finished
0202 02:52:47.471943 (+ 296us) write_op.cc:270] Start()
0202 02:52:47.472222 (+ 279us) write_op.cc:276] Timestamp: P: 1770000767471545 usec, L: 0
0202 02:52:47.472227 (+ 5us) op_driver.cc:348] REPLICATION: starting
0202 02:52:47.473772 (+ 1545us) log.cc:844] Serialized 731 byte log entry
0202 02:52:47.827004 (+353232us) op_driver.cc:464] REPLICATION: finished
0202 02:52:47.828076 (+ 1072us) write_op.cc:301] APPLY: starting
0202 02:52:47.828131 (+ 55us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:47.828259 (+ 128us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:47.828370 (+ 111us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:47.828602 (+ 232us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:47.831027 (+ 2425us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0202 02:52:47.831038 (+ 11us) write_op.cc:312] APPLY: finished
0202 02:52:47.835451 (+ 4413us) log.cc:844] Serialized 29 byte log entry
0202 02:52:47.837035 (+ 1584us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:47.837598 (+ 563us) write_op.cc:454] Released schema lock
0202 02:52:47.837608 (+ 10us) write_op.cc:341] FINISH: Updating metrics
0202 02:52:47.839597 (+ 1989us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:47.839604 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":963,"apply.run_cpu_time_us":1010,"apply.run_wall_time_us":10731,"num_ops":2,"prepare.queue_time_us":56018,"prepare.run_cpu_time_us":1896,"prepare.run_wall_time_us":7320,"replication_time_us":354721,"thread_start_us":55835,"threads_started":2}]]}
I20260202 02:52:48.052551 523 rpcz_store.cc:276] 0202 02:52:45.913972 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:47.688494 (+1774522us) service_pool.cc:225] Handling call
0202 02:52:47.689177 (+ 683us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0202 02:52:47.689323 (+ 146us) catalog_manager.cc:2815] Modifying in-memory table state
0202 02:52:47.689380 (+ 57us) catalog_manager.cc:2824] Locking tablets
0202 02:52:47.846307 (+156927us) catalog_manager.cc:2838] Removing table and tablets from system table
0202 02:52:47.892405 (+ 46098us) catalog_manager.cc:2856] Removing table from by-name map
0202 02:52:47.892497 (+ 92us) catalog_manager.cc:2872] Committing in-memory state
0202 02:52:47.892536 (+ 39us) catalog_manager.cc:2876] Aborting table tasks
0202 02:52:47.966989 (+ 74453us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:47.847280 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:47.847410 (+ 130us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:47.847418 (+ 8us) write_op.cc:435] Acquired schema lock
0202 02:52:47.847420 (+ 2us) tablet.cc:660] Decoding operations
0202 02:52:47.847569 (+ 149us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:47.847582 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:47.847589 (+ 7us) tablet.cc:683] Acquiring locks for 2 operations
0202 02:52:47.847721 (+ 132us) tablet.cc:699] Row locks acquired
0202 02:52:47.847724 (+ 3us) write_op.cc:260] PREPARE: finished
0202 02:52:47.850741 (+ 3017us) spinlock_profiling.cc:243] Waited 2.63 ms on lock 0x60b000087a00. stack: 00007fc023f2da86 00007fc023f2d840 00007fc022f083ad 0000000000984f94 0000000000984f5d 000000000098492a 00007fc02fe573b6 00007fc0311d31df 00007fc0311d2399 00007fc0311da7c0 00007fc0311da5cd 0000000000a8970c 00007fc023f73daf 00007fc023f770cb 00007fc023f76f48 0000000000a8970c
0202 02:52:47.850754 (+ 13us) write_op.cc:270] Start()
0202 02:52:47.850850 (+ 96us) write_op.cc:276] Timestamp: P: 1770000767850370 usec, L: 0
0202 02:52:47.850855 (+ 5us) op_driver.cc:348] REPLICATION: starting
0202 02:52:47.872390 (+ 21535us) log.cc:844] Serialized 762 byte log entry
0202 02:52:47.872585 (+ 195us) spinlock_profiling.cc:243] Waited 21.3 ms on lock 0x616000041254. stack: 00007fc023f2da86 00007fc023f2d840 00007fc022f083ad 0000000000984f94 0000000000984f5d 000000000098492a 00007fc02fdf0922 00007fc0311d3875 00007fc0311d2399 00007fc0311da7c0 00007fc0311da5cd 0000000000a8970c 00007fc023f73daf 00007fc023f770cb 00007fc023f76f48 0000000000a8970c
0202 02:52:47.885265 (+ 12680us) op_driver.cc:464] REPLICATION: finished
0202 02:52:47.887162 (+ 1897us) write_op.cc:301] APPLY: starting
0202 02:52:47.887177 (+ 15us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:47.887221 (+ 44us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:47.887224 (+ 3us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:47.890894 (+ 3670us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:47.890928 (+ 34us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0202 02:52:47.890937 (+ 9us) write_op.cc:312] APPLY: finished
0202 02:52:47.891032 (+ 95us) log.cc:844] Serialized 29 byte log entry
0202 02:52:47.892195 (+ 1163us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:47.892217 (+ 22us) write_op.cc:454] Released schema lock
0202 02:52:47.892223 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0202 02:52:47.892435 (+ 212us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:47.892440 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1805,"apply.run_cpu_time_us":714,"apply.run_wall_time_us":5228,"num_ops":2,"prepare.queue_time_us":83,"prepare.run_cpu_time_us":1763,"prepare.run_wall_time_us":25374,"raft.queue_time_us":302,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":279,"replication_time_us":34376,"spinlock_wait_cycles":54971136}]]}
I20260202 02:52:48.021229 2653 raft_consensus.cc:493] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:48.056661 519 rpcz_store.cc:276] 0202 02:52:35.966359 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:47.411634 (+11445275us) service_pool.cc:225] Handling call
0202 02:52:47.411968 (+ 334us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0202 02:52:47.412169 (+ 201us) catalog_manager.cc:3708] Apply alter schema
0202 02:52:47.412394 (+ 225us) catalog_manager.cc:3740] Acquired catalog manager lock
0202 02:52:47.431518 (+ 19124us) catalog_manager.cc:3898] Updating metadata on disk
0202 02:52:47.860285 (+428767us) catalog_manager.cc:3929] Committing alterations to in-memory state
0202 02:52:47.936461 (+ 76176us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:47.490988 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:47.491773 (+ 785us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:47.491905 (+ 132us) write_op.cc:435] Acquired schema lock
0202 02:52:47.491908 (+ 3us) tablet.cc:660] Decoding operations
0202 02:52:47.492530 (+ 622us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:47.492543 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:47.492667 (+ 124us) tablet.cc:683] Acquiring locks for 1 operations
0202 02:52:47.493092 (+ 425us) tablet.cc:699] Row locks acquired
0202 02:52:47.493095 (+ 3us) write_op.cc:260] PREPARE: finished
0202 02:52:47.493253 (+ 158us) write_op.cc:270] Start()
0202 02:52:47.493525 (+ 272us) write_op.cc:276] Timestamp: P: 1770000767492973 usec, L: 0
0202 02:52:47.493529 (+ 4us) op_driver.cc:348] REPLICATION: starting
0202 02:52:47.494998 (+ 1469us) log.cc:844] Serialized 405 byte log entry
0202 02:52:47.852193 (+357195us) op_driver.cc:464] REPLICATION: finished
0202 02:52:47.859204 (+ 7011us) write_op.cc:301] APPLY: starting
0202 02:52:47.859219 (+ 15us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:47.859295 (+ 76us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:47.859411 (+ 116us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:47.859499 (+ 88us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:47.859583 (+ 84us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0202 02:52:47.859625 (+ 42us) write_op.cc:312] APPLY: finished
0202 02:52:47.859777 (+ 152us) log.cc:844] Serialized 23 byte log entry
0202 02:52:47.860102 (+ 325us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:47.860121 (+ 19us) write_op.cc:454] Released schema lock
0202 02:52:47.860163 (+ 42us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":6941,"apply.run_cpu_time_us":664,"apply.run_wall_time_us":1093,"num_ops":1,"prepare.queue_time_us":58925,"prepare.run_cpu_time_us":1396,"prepare.run_wall_time_us":4904,"replication_time_us":358648,"thread_start_us":19383,"threads_started":1}]]}
I20260202 02:52:48.062404 526 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet 26b890ed87c3471b9fc0b172c90e0a11 (table default.Table_044ef6a2b1af4e3cab3c8cf00505c6a6 [id=d2e4220a0cbf443e8380433da4f064f8]). Expected version 1 got 0
I20260202 02:52:47.946564 748 tablet_service.cc:1505] Processing CreateTablet for tablet 76d3e799f5cb46638b289d06fca2f369 (DEFAULT_TABLE table=default.Table_22445ccfd7b64298b92a0f70733f3625 [id=300ea019d2724bee8211d7462710dc26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.064708 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76d3e799f5cb46638b289d06fca2f369. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:48.064826 859 leader_election.cc:343] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:48.065425 2664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } } }
I20260202 02:52:48.065806 2664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:52:48.067365 2431 consensus_queue.cc:1059] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:48.067742 527 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_077acfd01c1648ef96128a866323ec3e" } modify_external_catalogs: true
I20260202 02:52:48.068033 527 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_077acfd01c1648ef96128a866323ec3e" } modify_external_catalogs: true
I20260202 02:52:47.946210 742 tablet_service.cc:1505] Processing CreateTablet for tablet a402b238ae2b4a379598019914c68469 (DEFAULT_TABLE table=default.Table_33c051aaf8e1494fa559ae70367f89e9 [id=8e5a555f4f44492aa00b05de07ca2fed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.068087 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_22e604b386714cefbe344e7f6d6c3d6e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:47.947086 753 tablet_service.cc:1505] Processing CreateTablet for tablet 6d475fa5186243229c1db0fbdc531f4e (DEFAULT_TABLE table=default.Table_caf9a47c30ad4c6293b84e85cbde60eb [id=68f222efa4cf43b3804c194400d89112]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.068480 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a402b238ae2b4a379598019914c68469. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.068512 2372 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260202 02:52:48.068658 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d475fa5186243229c1db0fbdc531f4e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:48.068724 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_22e604b386714cefbe344e7f6d6c3d6e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:48.069955 2439 raft_consensus.cc:3055] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260202 02:52:48.070073 2439 raft_consensus.cc:740] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:48.070384 2439 consensus_queue.cc:260] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:48.070883 2439 raft_consensus.cc:3060] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:47.947150 750 tablet_service.cc:1552] Processing DeleteTablet for tablet f94f25f4059a43b8babedb6b45a08a19 with delete_type TABLET_DATA_DELETED (replaced by tablet 76d3e799f5cb46638b289d06fca2f369) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:47.946914 754 tablet_service.cc:1505] Processing CreateTablet for tablet 15366555b90a408c9f46222b44e7efa6 (DEFAULT_TABLE table=default.Table_46283f6baa2d4a9d9fdb8c1e501bcf89 [id=1cd4342c241a4072a65393b0be0c26a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.072345 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15366555b90a408c9f46222b44e7efa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:47.946739 744 tablet_service.cc:1505] Processing CreateTablet for tablet c438e69c40c24655b37f2e73b4b3b845 (DEFAULT_TABLE table=default.Table_ff96a2dc345940448683f8be23859993 [id=8e592b0d560f48ed86a5c3886f178c2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.075618 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c438e69c40c24655b37f2e73b4b3b845. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:48.075640 859 leader_election.cc:343] T 842e54adae2a452a98521de24326eaff P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:47.946383 751 tablet_service.cc:1505] Processing CreateTablet for tablet f87ee8d6ff9042c08ba2e32724bea720 (DEFAULT_TABLE table=default.Table_48929b86815b49369aff936433cae314 [id=2be03edfb43944be9714d5aa217263f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.076643 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f87ee8d6ff9042c08ba2e32724bea720. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:47.945362 756 tablet_service.cc:1505] Processing CreateTablet for tablet 6750a1212ceb4e6e82e4ffa93caab590 (DEFAULT_TABLE table=default.Table_52a2f389faba40fea93e2bca972e81e1 [id=70ac373200604907b00132eb4727ea5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.043648 2427 raft_consensus.cc:2804] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:48.080946 2427 raft_consensus.cc:697] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:48.081260 2427 consensus_queue.cc:237] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:48.081820 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6750a1212ceb4e6e82e4ffa93caab590. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:47.964257 745 tablet_service.cc:1552] Processing DeleteTablet for tablet 79c73f4a92044bafb5b8384229cb3a4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:47 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:48.056439 524 rpcz_store.cc:276] 0202 02:52:30.801102 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:47.409158 (+16608056us) service_pool.cc:225] Handling call
0202 02:52:47.689050 (+279892us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0202 02:52:47.689240 (+ 190us) catalog_manager.cc:2815] Modifying in-memory table state
0202 02:52:47.689328 (+ 88us) catalog_manager.cc:2824] Locking tablets
0202 02:52:47.846561 (+157233us) catalog_manager.cc:2838] Removing table and tablets from system table
0202 02:52:47.891724 (+ 45163us) catalog_manager.cc:2856] Removing table from by-name map
0202 02:52:47.891818 (+ 94us) catalog_manager.cc:2872] Committing in-memory state
0202 02:52:47.891868 (+ 50us) catalog_manager.cc:2876] Aborting table tasks
0202 02:52:47.967108 (+ 75240us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:47.872690 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:47.872795 (+ 105us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:47.872812 (+ 17us) write_op.cc:435] Acquired schema lock
0202 02:52:47.872815 (+ 3us) tablet.cc:660] Decoding operations
0202 02:52:47.873839 (+ 1024us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:47.873852 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:47.873858 (+ 6us) tablet.cc:683] Acquiring locks for 2 operations
0202 02:52:47.873977 (+ 119us) tablet.cc:699] Row locks acquired
0202 02:52:47.873980 (+ 3us) write_op.cc:260] PREPARE: finished
0202 02:52:47.874022 (+ 42us) write_op.cc:270] Start()
0202 02:52:47.874058 (+ 36us) write_op.cc:276] Timestamp: P: 1770000767873966 usec, L: 0
0202 02:52:47.874061 (+ 3us) op_driver.cc:348] REPLICATION: starting
0202 02:52:47.874232 (+ 171us) log.cc:844] Serialized 762 byte log entry
0202 02:52:47.885600 (+ 11368us) op_driver.cc:464] REPLICATION: finished
0202 02:52:47.887365 (+ 1765us) write_op.cc:301] APPLY: starting
0202 02:52:47.887494 (+ 129us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:47.887578 (+ 84us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:47.891257 (+ 3679us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:47.891322 (+ 65us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:47.891348 (+ 26us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0202 02:52:47.891356 (+ 8us) write_op.cc:312] APPLY: finished
0202 02:52:47.891407 (+ 51us) log.cc:844] Serialized 29 byte log entry
0202 02:52:47.891601 (+ 194us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:47.891619 (+ 18us) write_op.cc:454] Released schema lock
0202 02:52:47.891626 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0202 02:52:47.891704 (+ 78us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:47.891708 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":104320,"child_traces":[["op",{"apply.queue_time_us":1714,"apply.run_cpu_time_us":576,"apply.run_wall_time_us":4328,"num_ops":2,"prepare.queue_time_us":25462,"prepare.run_cpu_time_us":731,"prepare.run_wall_time_us":1611,"replication_time_us":11513}]]}
I20260202 02:52:48.086844 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_b1e92e2bd13d4f12906c13ff7e91f109"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:48.087612 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b1e92e2bd13d4f12906c13ff7e91f109 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:48.089481 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_d5520a9602ad4f69bd95b3eeef903589"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:48.090150 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d5520a9602ad4f69bd95b3eeef903589 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:47.980290 1175 tablet_bootstrap.cc:492] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:48.037422 2450 raft_consensus.cc:3060] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:48.061547 2653 raft_consensus.cc:515] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:48.092298 1064 raft_consensus.cc:2468] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
I20260202 02:52:48.056468 525 rpcz_store.cc:276] 0202 02:52:30.801193 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:47.410741 (+16609548us) service_pool.cc:225] Handling call
0202 02:52:47.411169 (+ 428us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0202 02:52:47.411307 (+ 138us) catalog_manager.cc:2815] Modifying in-memory table state
0202 02:52:47.411380 (+ 73us) catalog_manager.cc:2824] Locking tablets
0202 02:52:47.411518 (+ 138us) catalog_manager.cc:2838] Removing table and tablets from system table
0202 02:52:47.841912 (+430394us) catalog_manager.cc:2856] Removing table from by-name map
0202 02:52:47.842015 (+ 103us) catalog_manager.cc:2872] Committing in-memory state
0202 02:52:47.842056 (+ 41us) catalog_manager.cc:2876] Aborting table tasks
0202 02:52:47.967904 (+125848us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:47.475107 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:47.475836 (+ 729us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:47.475843 (+ 7us) write_op.cc:435] Acquired schema lock
0202 02:52:47.475846 (+ 3us) tablet.cc:660] Decoding operations
0202 02:52:47.476654 (+ 808us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:47.476668 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:47.476674 (+ 6us) tablet.cc:683] Acquiring locks for 2 operations
0202 02:52:47.477508 (+ 834us) tablet.cc:699] Row locks acquired
0202 02:52:47.477514 (+ 6us) write_op.cc:260] PREPARE: finished
0202 02:52:47.477801 (+ 287us) write_op.cc:270] Start()
0202 02:52:47.478077 (+ 276us) write_op.cc:276] Timestamp: P: 1770000767477397 usec, L: 0
0202 02:52:47.478084 (+ 7us) op_driver.cc:348] REPLICATION: starting
0202 02:52:47.479384 (+ 1300us) log.cc:844] Serialized 731 byte log entry
0202 02:52:47.828661 (+349277us) op_driver.cc:464] REPLICATION: finished
0202 02:52:47.830973 (+ 2312us) write_op.cc:301] APPLY: starting
0202 02:52:47.831527 (+ 554us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:47.834922 (+ 3395us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:47.836619 (+ 1697us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:47.839532 (+ 2913us) spinlock_profiling.cc:243] Waited 889 us on lock 0x60c00067c880. stack: 00007fc023f2da86 00007fc023f2d840 00007fc022f083ad 0000000000984f94 0000000000984f5d 000000000098492a 00007fc037b1ea55 00007fc02f6c6210 00007fc023f52430 0000000000a76196 00007fc023f71a4a 00007fc023f6d46c 00007fc023f7279e 00007fc0311d5787 00007fc0311d681a 00007fc0311da29c
0202 02:52:47.839751 (+ 219us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:47.841089 (+ 1338us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0202 02:52:47.841102 (+ 13us) write_op.cc:312] APPLY: finished
0202 02:52:47.841304 (+ 202us) log.cc:844] Serialized 29 byte log entry
0202 02:52:47.841755 (+ 451us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:47.841778 (+ 23us) write_op.cc:454] Released schema lock
0202 02:52:47.841785 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0202 02:52:47.841966 (+ 181us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:47.841970 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":44,"child_traces":[["op",{"apply.queue_time_us":609,"apply.run_cpu_time_us":1175,"apply.run_wall_time_us":10969,"num_ops":2,"prepare.queue_time_us":62617,"prepare.run_cpu_time_us":1567,"prepare.run_wall_time_us":5481,"replication_time_us":349838,"spinlock_wait_cycles":2045696,"thread_start_us":608,"threads_started":1}]]}
I20260202 02:52:48.093756 2450 raft_consensus.cc:515] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:48.093850 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_bd3fed1d460846fe81f7c0f86074b2ca"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:48.094452 2450 leader_election.cc:290] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:48.094594 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bd3fed1d460846fe81f7c0f86074b2ca in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:48.094763 1175 ts_tablet_manager.cc:1403] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.406s user 0.007s sys 0.003s
I20260202 02:52:48.095602 1175 raft_consensus.cc:359] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:48.095790 1175 raft_consensus.cc:385] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:48.095860 1175 raft_consensus.cc:740] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:48.096068 1175 consensus_queue.cc:260] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:48.096709 1175 ts_tablet_manager.cc:1434] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260202 02:52:48.096660 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28a900bcaa8b41718a285e595886d805" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
W20260202 02:52:48.040285 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 3191a80374724effa8389adc842e3448 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3191a80374724effa8389adc842e3448 already in progress: creating tablet
I20260202 02:52:48.097074 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6b8f924aecf44e959e47d2f32e6b0bf5" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:48.097232 1049 raft_consensus.cc:3060] T 6b8f924aecf44e959e47d2f32e6b0bf5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:48.097230 513 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 56918.8us
W20260202 02:52:48.097584 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 7ea7a5d8a92b4ac0903483627258ae66 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7ea7a5d8a92b4ac0903483627258ae66 already in progress: creating tablet
W20260202 02:52:48.097797 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 8ae6390a856d4c838e0689cd0c6c5c92 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8ae6390a856d4c838e0689cd0c6c5c92 already in progress: creating tablet
W20260202 02:52:48.098006 513 catalog_manager.cc:4952] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet c63a670d5c5e489897bc6dbb142b844d because the tablet was not found. No further retry: Not found: Tablet not found: c63a670d5c5e489897bc6dbb142b844d
I20260202 02:52:48.099097 745 tablet_service.cc:1552] Processing DeleteTablet for tablet 842e54adae2a452a98521de24326eaff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:47 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:48.099267 750 tablet_service.cc:1552] Processing DeleteTablet for tablet abfe7f76b2424c4d9ab07b29b66ec103 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:47 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:48.099316 758 tablet_service.cc:1552] Processing DeleteTablet for tablet dd8f36d7a82f47adae1bd3c1f8628b6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:47 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:48.099951 519 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_35fb6ec04ef245a88e633fedb3d9c186" } new_table_name: "default.Table_874748aec6a14c11b3787312a348ef91" modify_external_catalogs: true
I20260202 02:52:48.101671 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_ab48976710ec44e1a6b293a951fab152"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:48.102281 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ab48976710ec44e1a6b293a951fab152 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:48.106413 1180 raft_consensus.cc:2243] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:48.106681 1180 raft_consensus.cc:2272] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:48.107338 1180 ts_tablet_manager.cc:1916] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:48.109139 2330 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 861 } } ignore_live_leader: false dest_uuid: "7dab2abef14445d688ce1335b1f9b2f8" is_pre_election: true
W20260202 02:52:48.109961 2330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47296 (request call id 1) took 2838 ms (client timeout 630 ms). Trace:
W20260202 02:52:48.110137 2330 rpcz_store.cc:269] 0202 02:52:45.271227 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.271302 (+ 75us) service_pool.cc:225] Handling call
0202 02:52:48.109905 (+2838603us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:48.084266 1182 ts_tablet_manager.cc:1929] T 1f379a29e2454f689d68f5951837512b P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:48.111701 1182 log.cc:1199] T 1f379a29e2454f689d68f5951837512b P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/1f379a29e2454f689d68f5951837512b
I20260202 02:52:48.112069 1182 ts_tablet_manager.cc:1950] T 1f379a29e2454f689d68f5951837512b P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:47.983317 1035 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49890 (request call id 693) took 17528 ms. Trace:
I20260202 02:52:48.120460 1035 rpcz_store.cc:276] 0202 02:52:30.455368 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.455427 (+ 59us) service_pool.cc:225] Handling call
0202 02:52:30.592863 (+137436us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.592905 (+ 42us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:47.941751 (+17348846us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:47.983087 (+ 41336us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:47.983291 (+ 204us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:48.126030 1041 tablet_service.cc:1505] Processing CreateTablet for tablet e2378f5d9e3b48deb0d33f462994aad5 (DEFAULT_TABLE table=default.Table_025d64cc432643acbbef5c3fc8203457 [id=b73c24871d3d48b6accfe9a1a27a8071]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.126469 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2378f5d9e3b48deb0d33f462994aad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.127561 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 9f157e0fec954c198c790b10e01c6886 (DEFAULT_TABLE table=default.Table_2918d36968d74e4fb54ea2f78f039dfc [id=a9df59b6733f4ce8808a6e0b15f12d40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.130425 528 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 26b890ed87c3471b9fc0b172c90e0a11 (table default.Table_044ef6a2b1af4e3cab3c8cf00505c6a6 [id=d2e4220a0cbf443e8380433da4f064f8]). Expected version 1 got 0
I20260202 02:52:48.130831 528 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 44c8d5b74c8e4c499f7f68e8185efc17 (table default.Table_f48ed1c7e02349adad7ce22c0e51ed81 [id=46b5d47377474ff9bac2acdb26b08c65]). Expected version 3 got 0
I20260202 02:52:48.133850 758 tablet_service.cc:1505] Processing CreateTablet for tablet e2378f5d9e3b48deb0d33f462994aad5 (DEFAULT_TABLE table=default.Table_025d64cc432643acbbef5c3fc8203457 [id=b73c24871d3d48b6accfe9a1a27a8071]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.133893 750 tablet_service.cc:1505] Processing CreateTablet for tablet 9f157e0fec954c198c790b10e01c6886 (DEFAULT_TABLE table=default.Table_2918d36968d74e4fb54ea2f78f039dfc [id=a9df59b6733f4ce8808a6e0b15f12d40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.134418 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2378f5d9e3b48deb0d33f462994aad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.134662 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f157e0fec954c198c790b10e01c6886. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.139537 522 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 26b890ed87c3471b9fc0b172c90e0a11 (table default.Table_044ef6a2b1af4e3cab3c8cf00505c6a6 [id=d2e4220a0cbf443e8380433da4f064f8]). Expected version 1 got 0
I20260202 02:52:48.120393 1175 tablet_bootstrap.cc:492] T 0138ba7326674741b0729324735f1de7 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:48.148690 2430 leader_election.cc:290] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:48.149526 526 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_e4018d1c57714a24be5d3a20c1ccaef4 [id=1e291b35a6764638acfb781bb4e6adb3] alter complete (version 2)
I20260202 02:52:48.153246 2443 leader_election.cc:290] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 3 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:48.112443 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 681) took 18597 ms. Trace:
I20260202 02:52:48.155350 1182 rpcz_store.cc:276] 0202 02:52:29.514676 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.515911 (+ 1235us) service_pool.cc:225] Handling call
0202 02:52:48.038731 (+18522820us) ts_tablet_manager.cc:1151] Deleting tablet 1f379a29e2454f689d68f5951837512b
0202 02:52:48.038740 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.051576 (+ 12836us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.084093 (+ 32517us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.112410 (+ 28317us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":18522551}
I20260202 02:52:48.157269 745 tablet_service.cc:1552] Processing DeleteTablet for tablet 02b434fa058b45529ff3938b053263c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:19 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:48.157512 755 tablet_service.cc:1552] Processing DeleteTablet for tablet 874ca4ba4ebc4721bb4b60cba8e01841 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:48.157685 740 tablet_service.cc:1552] Processing DeleteTablet for tablet 026e6ee0b3ae4d938fdf1550044ed1fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:48.161226 1174 ts_tablet_manager.cc:1434] T 33150e1456144b8e9c64cc2a2ac6dcca P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.112s user 0.002s sys 0.002s
I20260202 02:52:48.161626 1174 tablet_bootstrap.cc:492] T dada209d982d48f3b74a49905f19c5b6 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
W20260202 02:52:48.163267 857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 126120us
I20260202 02:52:48.163944 1174 tablet_bootstrap.cc:654] T dada209d982d48f3b74a49905f19c5b6 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:48.165824 857 leader_election.cc:343] T a0b5d82e5fb94b60b4aa324b978a5378 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:48.165966 857 leader_election.cc:304] T a0b5d82e5fb94b60b4aa324b978a5378 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:48.166198 2445 raft_consensus.cc:2749] T a0b5d82e5fb94b60b4aa324b978a5378 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:48.166584 857 leader_election.cc:343] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:48.166965 857 leader_election.cc:304] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:48.167294 2445 raft_consensus.cc:2749] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260202 02:52:48.167685 857 leader_election.cc:343] T ac2ab12f5040488aa306cb69b107b485 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:48.167840 857 leader_election.cc:304] T ac2ab12f5040488aa306cb69b107b485 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:48.168192 2445 raft_consensus.cc:2749] T ac2ab12f5040488aa306cb69b107b485 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:48.168658 857 leader_election.cc:343] T d0ce2121d65b4a228596e0ca9018883d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:48.168814 857 leader_election.cc:304] T d0ce2121d65b4a228596e0ca9018883d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:48.169049 2445 raft_consensus.cc:2749] T d0ce2121d65b4a228596e0ca9018883d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:48.169452 857 leader_election.cc:343] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:48.169762 857 leader_election.cc:343] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:48.169953 857 leader_election.cc:343] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:48.170094 857 leader_election.cc:304] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:48.170370 2445 raft_consensus.cc:2749] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:48.170758 857 leader_election.cc:343] T f94f25f4059a43b8babedb6b45a08a19 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:48.171319 857 leader_election.cc:304] T f94f25f4059a43b8babedb6b45a08a19 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:48.173916 1182 tablet_replica.cc:333] T a0b5d82e5fb94b60b4aa324b978a5378 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:48.174347 1182 raft_consensus.cc:2243] T a0b5d82e5fb94b60b4aa324b978a5378 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:48.174594 2445 raft_consensus.cc:2749] T f94f25f4059a43b8babedb6b45a08a19 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:48.175082 1182 raft_consensus.cc:2272] T a0b5d82e5fb94b60b4aa324b978a5378 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:48.175884 1182 ts_tablet_manager.cc:1916] T a0b5d82e5fb94b60b4aa324b978a5378 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:48.176546 857 leader_election.cc:304] T c050a2882f764dda8df2a7df6547f149 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:48.179154 2445 raft_consensus.cc:2749] T c050a2882f764dda8df2a7df6547f149 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260202 02:52:48.181699 857 leader_election.cc:343] T d8418a9dd0944456b4d4820dde0cde02 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:48.182030 857 leader_election.cc:304] T d8418a9dd0944456b4d4820dde0cde02 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:48.182502 2445 raft_consensus.cc:2749] T d8418a9dd0944456b4d4820dde0cde02 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:48.184906 857 leader_election.cc:343] T fbcb4a522df648ba9542250450c08f1b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:48.185794 857 leader_election.cc:304] T fbcb4a522df648ba9542250450c08f1b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:48.186535 2445 raft_consensus.cc:2749] T fbcb4a522df648ba9542250450c08f1b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:48.187067 857 leader_election.cc:343] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:48.198208 2653 leader_election.cc:290] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:48.203735 1180 ts_tablet_manager.cc:1929] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:48.202111 1035 tablet_service.cc:1505] Processing CreateTablet for tablet 23dfa02219d748919e3cf24beb2095be (DEFAULT_TABLE table=default.Table_3be2f2becb3746a8a26896e806ee480a [id=88180dd0482c4b66a4c54cd78d74bb7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.204454 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23dfa02219d748919e3cf24beb2095be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.206226 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f157e0fec954c198c790b10e01c6886. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.217840 1049 raft_consensus.cc:2468] T 6b8f924aecf44e959e47d2f32e6b0bf5 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
I20260202 02:52:48.218729 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet b7880cb051c24bc998003f0effed3cf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:48.218907 1034 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49890 (request call id 700) took 17751 ms. Trace:
I20260202 02:52:48.218981 1034 rpcz_store.cc:276] 0202 02:52:30.467716 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.467767 (+ 51us) service_pool.cc:225] Handling call
0202 02:52:30.593452 (+125685us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:30.593493 (+ 41us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:48.091793 (+17498300us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.218693 (+126900us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:48.218891 (+ 198us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:48.227458 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79c73f4a92044bafb5b8384229cb3a4d" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:48.227685 1049 raft_consensus.cc:3060] T 79c73f4a92044bafb5b8384229cb3a4d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:48.230134 514 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 528d4269f91a44679acfad16e9fbbe32 (table default.Table_cdcc140c2b8d4aabae614d58873dc22a [id=c2c454eb645044839fa81b33b6460558], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 528d4269f91a44679acfad16e9fbbe32 consensus state has no leader
I20260202 02:52:48.230377 1049 raft_consensus.cc:2468] T 79c73f4a92044bafb5b8384229cb3a4d P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
I20260202 02:52:48.240597 1026 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49890 (request call id 734) took 1222 ms. Trace:
I20260202 02:52:48.240726 1026 rpcz_store.cc:276] 0202 02:52:47.017818 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:47.731716 (+713898us) service_pool.cc:225] Handling call
0202 02:52:47.732123 (+ 407us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:47.732146 (+ 23us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:48.066535 (+334389us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.240320 (+173785us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:48.240572 (+ 252us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260202 02:52:48.246800 998 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 210678us
W20260202 02:52:48.247491 998 leader_election.cc:343] T b7880cb051c24bc998003f0effed3cf8 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:48.247918 1173 tablet_bootstrap.cc:492] T 550895b4010f4088b2cc34acff50e03f P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:48.248055 1173 ts_tablet_manager.cc:1403] T 550895b4010f4088b2cc34acff50e03f P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.191s user 0.005s sys 0.004s
I20260202 02:52:48.208616 2669 raft_consensus.cc:493] T 33150e1456144b8e9c64cc2a2ac6dcca P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:48.224483 741 tablet_service.cc:1505] Processing CreateTablet for tablet 23dfa02219d748919e3cf24beb2095be (DEFAULT_TABLE table=default.Table_3be2f2becb3746a8a26896e806ee480a [id=88180dd0482c4b66a4c54cd78d74bb7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.249758 1180 log.cc:1199] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/2f0e8c5e2fd44370ae150a206f4e6cbc
I20260202 02:52:48.250075 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "25765fb687294e9db1289f76530dd877" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:48.249781 2669 raft_consensus.cc:515] T 33150e1456144b8e9c64cc2a2ac6dcca P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:48.250473 768 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "874ca4ba4ebc4721bb4b60cba8e01841" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:48.250588 1180 ts_tablet_manager.cc:1950] T 2f0e8c5e2fd44370ae150a206f4e6cbc P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:48.250627 768 raft_consensus.cc:3060] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:48.250937 2669 leader_election.cc:290] T 33150e1456144b8e9c64cc2a2ac6dcca P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:48.250077 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23dfa02219d748919e3cf24beb2095be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.251472 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 537) took 21926 ms. Trace:
I20260202 02:52:48.251549 1180 rpcz_store.cc:276] 0202 02:52:26.324735 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.324805 (+ 70us) service_pool.cc:225] Handling call
0202 02:52:47.940255 (+21615450us) ts_tablet_manager.cc:1151] Deleting tablet 2f0e8c5e2fd44370ae150a206f4e6cbc
0202 02:52:47.940262 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.126726 (+186464us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.203120 (+ 76394us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.251454 (+ 48334us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":49743,"tablet-delete.queue_time_us":21615193}
I20260202 02:52:48.251994 1180 tablet_replica.cc:333] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:48.252177 1180 raft_consensus.cc:2243] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:48.252367 1180 raft_consensus.cc:2272] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:48.253298 768 raft_consensus.cc:2468] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:48.253718 998 leader_election.cc:304] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:48.253985 2482 raft_consensus.cc:2804] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:48.254083 2482 raft_consensus.cc:697] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:48.254302 2482 consensus_queue.cc:237] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:48.256027 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8bb14bf551f441dcbaaff7057b51b2be" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:48.256505 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 2f0e8c5e2fd44370ae150a206f4e6cbc (table default.Table_04337834a3ce43ae8f6012b818d70d9f [id=3d1bb64023a145bb9fdcb2378a844a37]) successfully deleted
I20260202 02:52:48.257314 1173 raft_consensus.cc:359] T 550895b4010f4088b2cc34acff50e03f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:48.258234 1173 raft_consensus.cc:385] T 550895b4010f4088b2cc34acff50e03f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:48.258460 1173 raft_consensus.cc:740] T 550895b4010f4088b2cc34acff50e03f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:48.258785 1173 consensus_queue.cc:260] T 550895b4010f4088b2cc34acff50e03f P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:48.261281 1173 ts_tablet_manager.cc:1434] T 550895b4010f4088b2cc34acff50e03f P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.013s user 0.002s sys 0.000s
I20260202 02:52:48.261650 1173 tablet_bootstrap.cc:492] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:48.263645 1173 tablet_bootstrap.cc:654] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:48.263967 2331 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 861 } } ignore_live_leader: false dest_uuid: "7dab2abef14445d688ce1335b1f9b2f8" is_pre_election: true
I20260202 02:52:48.247470 1174 tablet_bootstrap.cc:492] T dada209d982d48f3b74a49905f19c5b6 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:48.264396 1174 ts_tablet_manager.cc:1403] T dada209d982d48f3b74a49905f19c5b6 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.103s user 0.004s sys 0.000s
I20260202 02:52:48.265216 1174 raft_consensus.cc:359] T dada209d982d48f3b74a49905f19c5b6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:48.265403 1174 raft_consensus.cc:385] T dada209d982d48f3b74a49905f19c5b6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:48.265467 1174 raft_consensus.cc:740] T dada209d982d48f3b74a49905f19c5b6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:48.265679 1174 consensus_queue.cc:260] T dada209d982d48f3b74a49905f19c5b6 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:48.266105 1882 ts_tablet_manager.cc:1929] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:48.266235 1882 log.cc:1199] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/fe53a8bdbd13417f9ce195c54a52a93c
I20260202 02:52:48.266535 1882 ts_tablet_manager.cc:1950] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
W20260202 02:52:48.266806 998 leader_election.cc:343] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:48.266927 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 572) took 20845 ms. Trace:
I20260202 02:52:48.267025 1882 rpcz_store.cc:276] 0202 02:52:27.421162 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:27.437033 (+ 15871us) service_pool.cc:225] Handling call
0202 02:52:47.716290 (+20279257us) ts_tablet_manager.cc:1151] Deleting tablet fe53a8bdbd13417f9ce195c54a52a93c
0202 02:52:47.716298 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.243904 (+527606us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.265981 (+ 22077us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.266878 (+ 897us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":34653,"tablet-delete.queue_time_us":20263748}
I20260202 02:52:48.267598 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 576) took 20734 ms. Trace:
I20260202 02:52:48.267704 1882 rpcz_store.cc:276] 0202 02:52:27.532951 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:27.534907 (+ 1956us) service_pool.cc:225] Handling call
0202 02:52:48.267471 (+20732564us) ts_tablet_manager.cc:1151] Deleting tablet 8cf6a52eea054ab688410e2dd029abf3
0202 02:52:48.267480 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.267584 (+ 104us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":20732301}
I20260202 02:52:48.267771 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet fe53a8bdbd13417f9ce195c54a52a93c (table default.Table_935434504bf249f8842274aa67486d8c [id=4f19275f416a425c9b9d100b2a83870a]) successfully deleted
I20260202 02:52:48.268088 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 574) took 20764 ms. Trace:
W20260202 02:52:48.268237 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 8cf6a52eea054ab688410e2dd029abf3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8cf6a52eea054ab688410e2dd029abf3 already in progress: creating tablet
I20260202 02:52:48.268187 1882 rpcz_store.cc:276] 0202 02:52:27.503957 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:27.544057 (+ 40100us) service_pool.cc:225] Handling call
0202 02:52:48.267986 (+20723929us) ts_tablet_manager.cc:1151] Deleting tablet 2c2a535de2294cc697d75561d0463642
0202 02:52:48.267995 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.268075 (+ 80us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":20723709}
I20260202 02:52:48.268529 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 573) took 20765 ms. Trace:
W20260202 02:52:48.268702 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 2c2a535de2294cc697d75561d0463642 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2c2a535de2294cc697d75561d0463642 already in progress: creating tablet
I20260202 02:52:48.268674 1882 rpcz_store.cc:276] 0202 02:52:27.503077 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:27.544317 (+ 41240us) service_pool.cc:225] Handling call
0202 02:52:48.268430 (+20724113us) ts_tablet_manager.cc:1151] Deleting tablet 4126bf5218644ca583c82190ce952267
0202 02:52:48.268439 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.268517 (+ 78us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":20723943}
I20260202 02:52:48.269009 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 581) took 20715 ms. Trace:
W20260202 02:52:48.269135 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 4126bf5218644ca583c82190ce952267 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4126bf5218644ca583c82190ce952267 already in progress: creating tablet
I20260202 02:52:48.269078 1882 rpcz_store.cc:276] 0202 02:52:27.553916 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:27.565370 (+ 11454us) service_pool.cc:225] Handling call
0202 02:52:48.268890 (+20703520us) ts_tablet_manager.cc:1151] Deleting tablet ce757ddc5c144d1fbaec84c9723cdbf2
0202 02:52:48.268898 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.268995 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":20703360}
I20260202 02:52:48.269343 1882 tablet_replica.cc:333] T 06b80a1c9251420cbc16fc95ae98aca9 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:48.269456 1174 ts_tablet_manager.cc:1434] T dada209d982d48f3b74a49905f19c5b6 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
W20260202 02:52:48.269541 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet ce757ddc5c144d1fbaec84c9723cdbf2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ce757ddc5c144d1fbaec84c9723cdbf2 already in progress: creating tablet
I20260202 02:52:48.269542 1882 raft_consensus.cc:2243] T 06b80a1c9251420cbc16fc95ae98aca9 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:48.269709 1882 raft_consensus.cc:2272] T 06b80a1c9251420cbc16fc95ae98aca9 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:48.269809 1174 tablet_bootstrap.cc:492] T 79c73f4a92044bafb5b8384229cb3a4d P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:48.270324 1882 ts_tablet_manager.cc:1916] T 06b80a1c9251420cbc16fc95ae98aca9 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:48.270439 2331 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47296 (request call id 0) took 12598 ms (client timeout 630 ms). Trace:
W20260202 02:52:48.270644 2331 rpcz_store.cc:269] 0202 02:52:35.672244 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:35.672342 (+ 98us) service_pool.cc:225] Handling call
0202 02:52:48.270335 (+12597993us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:48.275309 1174 tablet_bootstrap.cc:654] T 79c73f4a92044bafb5b8384229cb3a4d P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:48.276885 1180 ts_tablet_manager.cc:1916] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:48.217334 898 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41736 (request call id 716) took 17585 ms. Trace:
I20260202 02:52:48.279277 857 leader_election.cc:304] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:48.279351 898 rpcz_store.cc:276] 0202 02:52:30.632272 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.632666 (+ 394us) service_pool.cc:225] Handling call
0202 02:52:35.771281 (+5138615us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:35.771303 (+ 22us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:48.043993 (+12272690us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.216984 (+172991us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:48.217285 (+ 301us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:48.226511 859 leader_election.cc:304] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:48.279680 2445 raft_consensus.cc:2749] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:48.279886 2666 raft_consensus.cc:2804] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:48.280018 2666 raft_consensus.cc:493] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260202 02:52:48.280067 857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 92999us
I20260202 02:52:48.280161 2666 raft_consensus.cc:3060] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:48.280827 857 leader_election.cc:304] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:48.281128 2445 raft_consensus.cc:2749] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260202 02:52:48.281845 857 leader_election.cc:304] T af5aa68e47bb420eac300754d274fa73 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:48.282121 2445 raft_consensus.cc:2749] T af5aa68e47bb420eac300754d274fa73 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260202 02:52:48.282527 857 leader_election.cc:304] T 3d1a7578e83447669e3c3972a426ad44 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:48.282768 2445 raft_consensus.cc:2749] T 3d1a7578e83447669e3c3972a426ad44 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260202 02:52:48.283285 2666 raft_consensus.cc:515] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:48.283874 2666 leader_election.cc:290] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:48.153277 1175 tablet_bootstrap.cc:654] T 0138ba7326674741b0729324735f1de7 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:48.285178 859 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 58688.8us
I20260202 02:52:48.285401 859 leader_election.cc:304] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:48.285658 859 leader_election.cc:304] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:48.285871 2445 raft_consensus.cc:2804] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:48.285995 2445 raft_consensus.cc:697] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:48.286262 2445 consensus_queue.cc:237] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:48.287992 2443 raft_consensus.cc:2804] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:48.288134 2443 raft_consensus.cc:697] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:48.288569 2443 consensus_queue.cc:237] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:48.300913 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8bb14bf551f441dcbaaff7057b51b2be" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:48.301162 1049 raft_consensus.cc:3060] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:48.304765 1049 raft_consensus.cc:2468] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
I20260202 02:52:48.305557 859 leader_election.cc:304] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:48.306059 2666 raft_consensus.cc:2804] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:48.306387 2666 raft_consensus.cc:697] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:48.306839 2666 consensus_queue.cc:237] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:48.316078 857 leader_election.cc:343] T 474df41e4fe24a818fa45a4ce84d5474 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:48.316368 857 leader_election.cc:304] T 474df41e4fe24a818fa45a4ce84d5474 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
W20260202 02:52:48.316629 857 raft_consensus.cc:2686] T 474df41e4fe24a818fa45a4ce84d5474 P 1521a6d262b7451bbef8debe970dd6b5: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260202 02:52:48.317106 857 leader_election.cc:304] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:48.317440 2430 raft_consensus.cc:2749] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260202 02:52:48.317871 857 leader_election.cc:304] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:48.318166 2430 raft_consensus.cc:2804] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260202 02:52:48.318253 2430 raft_consensus.cc:493] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b1f05820b6034228bdad884cc348716d)
I20260202 02:52:48.318346 2430 raft_consensus.cc:3060] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:48.319869 2632 raft_consensus.cc:697] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:48.320236 2632 consensus_queue.cc:237] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
W20260202 02:52:48.321182 715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 281462us
W20260202 02:52:48.321895 715 consensus_peers.cc:597] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:48.321833 2430 raft_consensus.cc:515] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:48.322587 2430 leader_election.cc:290] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:48.323920 715 leader_election.cc:304] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:48.324179 2669 raft_consensus.cc:2804] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260202 02:52:48.324261 2669 raft_consensus.cc:493] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
I20260202 02:52:48.324369 2669 raft_consensus.cc:3060] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:48.325387 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "16b47a2831f54d809ed0b16deaec7e3d" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:48.327174 2669 raft_consensus.cc:515] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:48.328112 2669 leader_election.cc:290] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:48.333262 519 master_service.cc:954] ReplaceTablet: received request to replace tablet 118753e8cb344b47a35a745aa8fed265 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:48.336318 527 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 3338964dfed548f980f58147f0283691
I20260202 02:52:48.336927 527 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:47430 (request call id 832) took 2420 ms. Trace:
I20260202 02:52:48.337037 527 rpcz_store.cc:276] 0202 02:52:45.916727 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:48.067540 (+2150813us) service_pool.cc:225] Handling call
0202 02:52:48.068162 (+ 622us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0202 02:52:48.068307 (+ 145us) catalog_manager.cc:2815] Modifying in-memory table state
0202 02:52:48.112482 (+ 44175us) catalog_manager.cc:2824] Locking tablets
0202 02:52:48.123229 (+ 10747us) catalog_manager.cc:2838] Removing table and tablets from system table
0202 02:52:48.336066 (+212837us) catalog_manager.cc:2856] Removing table from by-name map
0202 02:52:48.336158 (+ 92us) catalog_manager.cc:2872] Committing in-memory state
0202 02:52:48.336194 (+ 36us) catalog_manager.cc:2876] Aborting table tasks
0202 02:52:48.336861 (+ 667us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:48.124188 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:48.124301 (+ 113us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:48.124307 (+ 6us) write_op.cc:435] Acquired schema lock
0202 02:52:48.124310 (+ 3us) tablet.cc:660] Decoding operations
0202 02:52:48.124470 (+ 160us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:48.124482 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:48.124487 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
0202 02:52:48.124595 (+ 108us) tablet.cc:699] Row locks acquired
0202 02:52:48.124598 (+ 3us) write_op.cc:260] PREPARE: finished
0202 02:52:48.124643 (+ 45us) write_op.cc:270] Start()
0202 02:52:48.124677 (+ 34us) write_op.cc:276] Timestamp: P: 1770000768124561 usec, L: 0
0202 02:52:48.124681 (+ 4us) op_driver.cc:348] REPLICATION: starting
0202 02:52:48.127435 (+ 2754us) log.cc:844] Serialized 731 byte log entry
0202 02:52:48.335519 (+208084us) op_driver.cc:464] REPLICATION: finished
0202 02:52:48.335607 (+ 88us) write_op.cc:301] APPLY: starting
0202 02:52:48.335619 (+ 12us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:48.335655 (+ 36us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:48.335685 (+ 30us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:48.335746 (+ 61us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:48.335777 (+ 31us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0202 02:52:48.335785 (+ 8us) write_op.cc:312] APPLY: finished
0202 02:52:48.335858 (+ 73us) log.cc:844] Serialized 29 byte log entry
0202 02:52:48.335922 (+ 64us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:48.335939 (+ 17us) write_op.cc:454] Released schema lock
0202 02:52:48.335946 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":55,"child_traces":[["op",{"apply.queue_time_us":32,"apply.run_cpu_time_us":398,"apply.run_wall_time_us":397,"num_ops":2,"prepare.queue_time_us":70,"prepare.run_cpu_time_us":835,"prepare.run_wall_time_us":3357,"replication_time_us":210811}]]}
I20260202 02:52:48.342684 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_2d8e70c77d2e4bbe994ff7493c108625"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:48.342757 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_b82204d74f4949769570a9720fe4b5af"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:48.343439 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2d8e70c77d2e4bbe994ff7493c108625 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:48.343447 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b82204d74f4949769570a9720fe4b5af in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:48.343536 1035 tablet_service.cc:1552] Processing DeleteTablet for tablet 3338964dfed548f980f58147f0283691 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:48 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:48.347940 528 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 26b890ed87c3471b9fc0b172c90e0a11 (table default.Table_044ef6a2b1af4e3cab3c8cf00505c6a6 [id=d2e4220a0cbf443e8380433da4f064f8]). Expected version 1 got 0
I20260202 02:52:48.348153 528 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 8b0368e03f644f1db28dbb57c95c48c1 (table default.Table_f8d8615b34774b738faeb83039f9601a [id=8882e1fbac2949838c99adb1290f273e]). Expected version 3 got 0
I20260202 02:52:48.349153 528 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_7913f8fb6eac487a971545a11cf257a9" } new_table_name: "default.Table_88cb19d3d7684dc1a8aa41ed28fed4b8" modify_external_catalogs: true
I20260202 02:52:48.349226 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_c8269ebb04c6479fa3643887f6f51f8d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:48.349761 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c8269ebb04c6479fa3643887f6f51f8d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:48.351418 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_c8fe5bcb63164a1980d102e0374c1d0e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:48.352088 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c8fe5bcb63164a1980d102e0374c1d0e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:48.352479 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_502b1ec344f64ad78b8613e843e3f73b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:48.352949 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_502b1ec344f64ad78b8613e843e3f73b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:48.354878 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 3338964dfed548f980f58147f0283691 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:48 UTC) from {username='slave'} at 127.0.0.1:42950
W20260202 02:52:48.356412 827 tablet.cc:2378] T 33150e1456144b8e9c64cc2a2ac6dcca P 1f442ef85029461091de847118c34c0d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260202 02:52:48.360276 525 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_5485f3195b4f4bd7b9220ff7b1459d5a" } new_table_name: "default.Table_20a23d2f97dc42fcbff9071d1b935941" modify_external_catalogs: true
I20260202 02:52:48.362982 1182 ts_tablet_manager.cc:1929] T a0b5d82e5fb94b60b4aa324b978a5378 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:48.363201 1182 log.cc:1199] T a0b5d82e5fb94b60b4aa324b978a5378 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/a0b5d82e5fb94b60b4aa324b978a5378
I20260202 02:52:48.362968 521 catalog_manager.cc:5654] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130). New cstate: current_term: 1 leader_uuid: "1521a6d262b7451bbef8debe970dd6b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: HEALTHY } } }
I20260202 02:52:48.363564 1182 ts_tablet_manager.cc:1950] T a0b5d82e5fb94b60b4aa324b978a5378 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:48.363515 521 catalog_manager.cc:5654] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130). New cstate: current_term: 1 leader_uuid: "1521a6d262b7451bbef8debe970dd6b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: HEALTHY } } }
I20260202 02:52:48.363943 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 671) took 19099 ms. Trace:
I20260202 02:52:48.364034 1182 rpcz_store.cc:276] 0202 02:52:29.264728 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.650210 (+385482us) service_pool.cc:225] Handling call
0202 02:52:48.155641 (+18505431us) ts_tablet_manager.cc:1151] Deleting tablet a0b5d82e5fb94b60b4aa324b978a5378
0202 02:52:48.173833 (+ 18192us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.273151 (+ 99318us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.362836 (+ 89685us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.363904 (+ 1068us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":420,"tablet-delete.queue_time_us":18505196}
I20260202 02:52:48.364216 523 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_58420fa37ead4e3c97dd3108c58b31a1" } modify_external_catalogs: true
I20260202 02:52:48.364450 523 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_58420fa37ead4e3c97dd3108c58b31a1" } modify_external_catalogs: true
I20260202 02:52:48.364527 1035 tablet_service.cc:1552] Processing DeleteTablet for tablet 550895b4010f4088b2cc34acff50e03f with delete_type TABLET_DATA_DELETED (replaced by tablet 5416eab7a0914ecb855fbccde4894285) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:48.364750 1182 tablet_replica.cc:333] T 3d1a7578e83447669e3c3972a426ad44 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:48.365084 1182 raft_consensus.cc:2243] T 3d1a7578e83447669e3c3972a426ad44 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:48.366890 519 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 118753e8cb344b47a35a745aa8fed265
I20260202 02:52:48.367368 519 catalog_manager.cc:6724] ReplaceTablet: tablet 118753e8cb344b47a35a745aa8fed265 deleted and replaced by tablet 0aa023e74e74480498b721bbb5d15052
I20260202 02:52:48.367856 1035 tablet_service.cc:1552] Processing DeleteTablet for tablet 118753e8cb344b47a35a745aa8fed265 with delete_type TABLET_DATA_DELETED (replaced by tablet 0aa023e74e74480498b721bbb5d15052) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:48.367794 519 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_6b5c3a97133940d98d66d07f6be3d55a" } modify_external_catalogs: true
I20260202 02:52:48.368019 519 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_6b5c3a97133940d98d66d07f6be3d55a" } modify_external_catalogs: true
W20260202 02:52:48.371413 528 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0aa023e74e74480498b721bbb5d15052 has no consensus state
I20260202 02:52:48.371876 528 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_48929b86815b49369aff936433cae314" } modify_external_catalogs: true
I20260202 02:52:48.372085 528 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_48929b86815b49369aff936433cae314" } modify_external_catalogs: true
I20260202 02:52:48.372918 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 118753e8cb344b47a35a745aa8fed265 with delete_type TABLET_DATA_DELETED (replaced by tablet 0aa023e74e74480498b721bbb5d15052) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:48.374302 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_731531d5a4d548d8b43461d6b87310dc" } new_table_name: "default.Table_65cccf45d7ea41b885fda61444c03ea5" modify_external_catalogs: true
I20260202 02:52:48.375206 744 tablet_service.cc:1552] Processing DeleteTablet for tablet 2f0e8c5e2fd44370ae150a206f4e6cbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:18 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:48.375809 522 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_12c352712c374a45b7de7a0e634a1327" } new_table_name: "default.Table_5e9068e8d78849919824ab9796e02397" modify_external_catalogs: true
W20260202 02:52:48.384356 525 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 67a8c9a1998b47b3b214a06ce62381e9 consensus state has no leader
I20260202 02:52:48.384780 519 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 48863e2d60c84460858daabb9a08dcd2
I20260202 02:52:48.385712 521 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_10aa0dafec0547069eb92ecf12b3c780" } modify_external_catalogs: true
I20260202 02:52:48.386013 521 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_10aa0dafec0547069eb92ecf12b3c780" } modify_external_catalogs: true
I20260202 02:52:48.386303 1035 tablet_service.cc:1552] Processing DeleteTablet for tablet 48863e2d60c84460858daabb9a08dcd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:48 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:48.387267 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_37ff6365170a49efb41f9a4d5db68e6a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:48.387391 1173 tablet_bootstrap.cc:492] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:48.387383 744 tablet_service.cc:1552] Processing DeleteTablet for tablet 48863e2d60c84460858daabb9a08dcd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:48 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:48.387557 1173 ts_tablet_manager.cc:1403] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.126s user 0.001s sys 0.004s
I20260202 02:52:48.388372 1173 raft_consensus.cc:359] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:48.388311 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_37ff6365170a49efb41f9a4d5db68e6a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:48.388676 744 tablet_service.cc:1552] Processing DeleteTablet for tablet dada209d982d48f3b74a49905f19c5b6 with delete_type TABLET_DATA_DELETED (replaced by tablet 67a8c9a1998b47b3b214a06ce62381e9) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:48.388815 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_fc9e04c1d0564a7896286c1a2d4f87a6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:48.389413 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fc9e04c1d0564a7896286c1a2d4f87a6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:48.389806 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 33150e1456144b8e9c64cc2a2ac6dcca with delete_type TABLET_DATA_DELETED (replaced by tablet 1d98c7a46139413b8f0d3697b74b1e52) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:48.390120 523 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 2d18485f5a4a4b11916c64a487cd28b4
I20260202 02:52:48.390798 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_063766871b1745af8bf479effa86c7f1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:48.391391 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_063766871b1745af8bf479effa86c7f1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:48.391786 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 2d18485f5a4a4b11916c64a487cd28b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:48 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:48.393049 1182 raft_consensus.cc:2272] T 3d1a7578e83447669e3c3972a426ad44 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:48.393738 1182 ts_tablet_manager.cc:1916] T 3d1a7578e83447669e3c3972a426ad44 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:48.397186 1035 tablet_service.cc:1552] Processing DeleteTablet for tablet 2d18485f5a4a4b11916c64a487cd28b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:48 UTC) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:48.397210 715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 73293.9us
I20260202 02:52:48.397455 715 leader_election.cc:304] T b76d1837813c46e8a0925b49939d8884 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:48.397926 715 leader_election.cc:304] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:48.398172 2669 raft_consensus.cc:2804] T b76d1837813c46e8a0925b49939d8884 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260202 02:52:48.398283 2669 raft_consensus.cc:493] T b76d1837813c46e8a0925b49939d8884 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
I20260202 02:52:48.398355 2669 raft_consensus.cc:3060] T b76d1837813c46e8a0925b49939d8884 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Advancing to term 2
W20260202 02:52:48.365595 857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 47746us
I20260202 02:52:48.401950 2669 raft_consensus.cc:515] T b76d1837813c46e8a0925b49939d8884 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:48.402890 2669 leader_election.cc:290] T b76d1837813c46e8a0925b49939d8884 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:48.403079 2669 raft_consensus.cc:2804] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:48.403281 2669 raft_consensus.cc:697] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:48.403510 2669 consensus_queue.cc:237] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:48.405503 1175 tablet_bootstrap.cc:492] T 0138ba7326674741b0729324735f1de7 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:48.405704 1175 ts_tablet_manager.cc:1403] T 0138ba7326674741b0729324735f1de7 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.285s user 0.006s sys 0.000s
I20260202 02:52:48.406591 1175 raft_consensus.cc:359] T 0138ba7326674741b0729324735f1de7 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:48.406813 1175 raft_consensus.cc:385] T 0138ba7326674741b0729324735f1de7 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:48.406899 1175 raft_consensus.cc:740] T 0138ba7326674741b0729324735f1de7 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:48.407228 1175 consensus_queue.cc:260] T 0138ba7326674741b0729324735f1de7 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:48.407799 1175 ts_tablet_manager.cc:1434] T 0138ba7326674741b0729324735f1de7 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:48.410903 969 heartbeater.cc:344] Connected to a master server at 127.0.104.190:33981
I20260202 02:52:48.416242 528 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet f87ee8d6ff9042c08ba2e32724bea720
I20260202 02:52:48.416576 922 raft_consensus.cc:1240] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [term 3 FOLLOWER]: Rejecting Update request from peer 1f442ef85029461091de847118c34c0d for earlier term 2. Current term is 3. Ops: []
I20260202 02:52:48.417273 1035 tablet_service.cc:1552] Processing DeleteTablet for tablet f87ee8d6ff9042c08ba2e32724bea720 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:48 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:48.419390 741 tablet_service.cc:1552] Processing DeleteTablet for tablet f87ee8d6ff9042c08ba2e32724bea720 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:48 UTC) from {username='slave'} at 127.0.0.1:42950
W20260202 02:52:48.428299 527 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cafa1d1231c54a608422695409720112 consensus state has no leader
I20260202 02:52:48.435393 521 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet d2d27ceea1bb4122b18f0fcf4d2d1bf4
I20260202 02:52:48.436403 1035 tablet_service.cc:1552] Processing DeleteTablet for tablet d2d27ceea1bb4122b18f0fcf4d2d1bf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:48 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:48.436959 741 tablet_service.cc:1552] Processing DeleteTablet for tablet d2d27ceea1bb4122b18f0fcf4d2d1bf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:48 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:48.437443 922 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "25765fb687294e9db1289f76530dd877" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.438452 922 raft_consensus.cc:1240] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Rejecting Update request from peer 1f442ef85029461091de847118c34c0d for earlier term 1. Current term is 2. Ops: []
I20260202 02:52:48.439560 920 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "622a113af04a41c6852c29fc8a8acd78" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.439929 922 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "91ec01d25f8f441cbe3783cacbee51db" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.439965 918 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f6746ccbf37e477196a9706396d1e3fd" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.440173 918 raft_consensus.cc:2468] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:48.440258 915 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b76d1837813c46e8a0925b49939d8884" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.440294 920 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "20ea39bc5d3d46f5a4ba427277d6c6a3" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.440689 920 raft_consensus.cc:2468] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:48.440689 919 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "be993c1512b04df4b92cab3195241d84" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.440883 915 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3d1a7578e83447669e3c3972a426ad44" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.440907 919 raft_consensus.cc:2468] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:48.441067 917 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cb634d4bae554ef3b39eeaa9958adc6c" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.441228 917 raft_consensus.cc:2468] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:48.441377 919 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "73f8db8f30f14d369d2227c9be7c4647" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.441895 919 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e1284fde67bc4d65b067a54c416ccc33" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.442049 919 raft_consensus.cc:2468] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:48.442371 1174 tablet_bootstrap.cc:492] T 79c73f4a92044bafb5b8384229cb3a4d P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:48.442507 1174 ts_tablet_manager.cc:1403] T 79c73f4a92044bafb5b8384229cb3a4d P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.173s user 0.013s sys 0.010s
I20260202 02:52:48.443325 1174 raft_consensus.cc:359] T 79c73f4a92044bafb5b8384229cb3a4d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:48.443481 1174 raft_consensus.cc:385] T 79c73f4a92044bafb5b8384229cb3a4d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:48.443526 1174 raft_consensus.cc:740] T 79c73f4a92044bafb5b8384229cb3a4d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:48.443717 1174 consensus_queue.cc:260] T 79c73f4a92044bafb5b8384229cb3a4d P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:48.443841 919 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "16b47a2831f54d809ed0b16deaec7e3d" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.444062 919 raft_consensus.cc:2393] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f442ef85029461091de847118c34c0d in current term 2: Already voted for candidate 1521a6d262b7451bbef8debe970dd6b5 in this term.
I20260202 02:52:48.444208 1174 ts_tablet_manager.cc:1434] T 79c73f4a92044bafb5b8384229cb3a4d P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:48.444525 1174 tablet_bootstrap.cc:492] T 8bb14bf551f441dcbaaff7057b51b2be P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:48.379040 879 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:41734 (request call id 143) took 2621 ms. Trace:
W20260202 02:52:48.445510 715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 47591.2us
I20260202 02:52:48.445595 879 rpcz_store.cc:276] 0202 02:52:45.757618 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.758402 (+ 784us) service_pool.cc:225] Handling call
0202 02:52:47.832257 (+2073855us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x6160000d9954. stack: 00007fb35529ca86 00007fb35529c840 00007fb3542773ad 0000000000984f94 0000000000984f5d 000000000098492a 00007fb3611592c0 00007fb365009b2f 00007fb35f42ec36 00007fb35f42e965 00007fb35c25f55e 00007fb35c25e43e 00007fb35c264675 00007fb35c264f9f 00007fb35c264dc8 0000000000a8970c
0202 02:52:48.378980 (+546723us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":277771136}
I20260202 02:52:48.445870 715 leader_election.cc:304] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
W20260202 02:52:48.446105 715 leader_election.cc:343] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Not found: Tablet not found: 4e79cd48fefb4ce29ef48beacfc4d26f
I20260202 02:52:48.446338 2448 raft_consensus.cc:2764] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260202 02:52:48.447814 1174 tablet_bootstrap.cc:654] T 8bb14bf551f441dcbaaff7057b51b2be P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:48.449530 1882 ts_tablet_manager.cc:1929] T 06b80a1c9251420cbc16fc95ae98aca9 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:48.449705 1882 log.cc:1199] T 06b80a1c9251420cbc16fc95ae98aca9 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/06b80a1c9251420cbc16fc95ae98aca9
I20260202 02:52:48.449999 1882 ts_tablet_manager.cc:1950] T 06b80a1c9251420cbc16fc95ae98aca9 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:48.450484 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 624) took 20299 ms. Trace:
I20260202 02:52:48.450587 1882 rpcz_store.cc:276] 0202 02:52:28.150869 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:28.192899 (+ 42030us) service_pool.cc:225] Handling call
0202 02:52:48.269296 (+20076397us) ts_tablet_manager.cc:1151] Deleting tablet 06b80a1c9251420cbc16fc95ae98aca9
0202 02:52:48.269304 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.435944 (+166640us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.449410 (+ 13466us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.450463 (+ 1053us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":20042813}
I20260202 02:52:48.451061 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 623) took 20314 ms. Trace:
I20260202 02:52:48.451143 1882 rpcz_store.cc:276] 0202 02:52:28.136578 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:28.209521 (+ 72943us) service_pool.cc:225] Handling call
0202 02:52:48.450935 (+20241414us) ts_tablet_manager.cc:1151] Deleting tablet 2c31b45dc8004eae993e972e9faa3ff8
0202 02:52:48.450945 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.451045 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":20223090}
I20260202 02:52:48.451237 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 06b80a1c9251420cbc16fc95ae98aca9 (table default.Table_84d6693620cc468e82d5cb1ec6cc69ea [id=f6625b21b6064828b11acdfdb694cd71]) successfully deleted
I20260202 02:52:48.451469 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 627) took 20187 ms. Trace:
I20260202 02:52:48.451540 1882 rpcz_store.cc:276] 0202 02:52:28.263583 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:28.263670 (+ 87us) service_pool.cc:225] Handling call
0202 02:52:48.451372 (+20187702us) ts_tablet_manager.cc:1151] Deleting tablet 425dab1cf74c4f379f10e202613093dd
0202 02:52:48.451380 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.451455 (+ 75us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":20187282}
W20260202 02:52:48.451856 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 2c31b45dc8004eae993e972e9faa3ff8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2c31b45dc8004eae993e972e9faa3ff8 already in progress: creating tablet
I20260202 02:52:48.452023 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 632) took 19876 ms. Trace:
I20260202 02:52:48.452103 1882 rpcz_store.cc:276] 0202 02:52:28.575465 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:28.575526 (+ 61us) service_pool.cc:225] Handling call
0202 02:52:48.451918 (+19876392us) ts_tablet_manager.cc:1151] Deleting tablet 83d198d90a204c6396b569fc1b65abef
0202 02:52:48.451928 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.452008 (+ 80us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":19876177}
I20260202 02:52:48.452422 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 633) took 19871 ms. Trace:
I20260202 02:52:48.452492 1882 rpcz_store.cc:276] 0202 02:52:28.580895 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:28.580967 (+ 72us) service_pool.cc:225] Handling call
0202 02:52:48.452304 (+19871337us) ts_tablet_manager.cc:1151] Deleting tablet 47edd9f65ef343b9a3be74c30708daca
0202 02:52:48.452312 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.452408 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":19871137}
W20260202 02:52:48.452713 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 425dab1cf74c4f379f10e202613093dd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 425dab1cf74c4f379f10e202613093dd already in progress: creating tablet
I20260202 02:52:48.452785 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 634) took 19871 ms. Trace:
I20260202 02:52:48.452859 1882 rpcz_store.cc:276] 0202 02:52:28.581621 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:28.581664 (+ 43us) service_pool.cc:225] Handling call
0202 02:52:48.452697 (+19871033us) ts_tablet_manager.cc:1151] Deleting tablet b05008bde58d47f7adaaf35ac5949bdc
0202 02:52:48.452705 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.452772 (+ 67us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":19870884}
W20260202 02:52:48.452955 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 83d198d90a204c6396b569fc1b65abef because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 83d198d90a204c6396b569fc1b65abef already in progress: creating tablet
I20260202 02:52:48.453166 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 631) took 19877 ms. Trace:
I20260202 02:52:48.453248 1882 rpcz_store.cc:276] 0202 02:52:28.575331 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:28.575525 (+ 194us) service_pool.cc:225] Handling call
0202 02:52:48.453055 (+19877530us) ts_tablet_manager.cc:1151] Deleting tablet 4ec86e245f8c4d9e87c3ee3409a164b3
0202 02:52:48.453066 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.453150 (+ 84us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":19857838}
I20260202 02:52:48.453519 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 639) took 19612 ms. Trace:
I20260202 02:52:48.453581 1882 rpcz_store.cc:276] 0202 02:52:28.840778 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:28.840853 (+ 75us) service_pool.cc:225] Handling call
0202 02:52:48.453443 (+19612590us) ts_tablet_manager.cc:1151] Deleting tablet 6a3dac5cd1ce47488a50b4c32eca67c7
0202 02:52:48.453449 (+ 6us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.453507 (+ 58us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":19607644}
W20260202 02:52:48.453660 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 47edd9f65ef343b9a3be74c30708daca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 47edd9f65ef343b9a3be74c30708daca already in progress: creating tablet
I20260202 02:52:48.453861 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 641) took 19508 ms. Trace:
W20260202 02:52:48.453867 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet b05008bde58d47f7adaaf35ac5949bdc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b05008bde58d47f7adaaf35ac5949bdc already in progress: creating tablet
I20260202 02:52:48.453923 1882 rpcz_store.cc:276] 0202 02:52:28.945354 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:28.945414 (+ 60us) service_pool.cc:225] Handling call
0202 02:52:48.453762 (+19508348us) ts_tablet_manager.cc:1151] Deleting tablet 79e65a55f29449a0b13349bc152c651b
0202 02:52:48.453770 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.453847 (+ 77us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":19505963}
W20260202 02:52:48.454080 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 4ec86e245f8c4d9e87c3ee3409a164b3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4ec86e245f8c4d9e87c3ee3409a164b3 already in progress: creating tablet
I20260202 02:52:48.454124 1882 tablet_replica.cc:333] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:48.454288 1882 raft_consensus.cc:2243] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:48.454525 1882 pending_rounds.cc:70] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d: Trying to abort 1 pending ops.
I20260202 02:52:48.454568 1882 pending_rounds.cc:77] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923147042156544 op_type: NO_OP noop_request { }
I20260202 02:52:48.454658 1882 raft_consensus.cc:2889] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:48.454727 1882 raft_consensus.cc:2272] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:48.454664 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 6a3dac5cd1ce47488a50b4c32eca67c7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6a3dac5cd1ce47488a50b4c32eca67c7 already in progress: creating tablet
W20260202 02:52:48.454931 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 79e65a55f29449a0b13349bc152c651b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 79e65a55f29449a0b13349bc152c651b already in progress: creating tablet
I20260202 02:52:48.455785 915 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26b890ed87c3471b9fc0b172c90e0a11" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.456039 915 raft_consensus.cc:2368] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f442ef85029461091de847118c34c0d for earlier term 1. Current term is 2.
I20260202 02:52:48.460843 1175 tablet_bootstrap.cc:492] T 939fbcaf75c7439d9208bbbf83a72717 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:48.462767 1175 tablet_bootstrap.cc:654] T 939fbcaf75c7439d9208bbbf83a72717 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:48.465057 520 master_service.cc:954] ReplaceTablet: received request to replace tablet e07e87c0182b46a49b8ca64c072f4210 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:48.465095 1882 ts_tablet_manager.cc:1916] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:48.467835 715 consensus_peers.cc:597] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:48.477178 1064 raft_consensus.cc:1240] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Rejecting Update request from peer 1521a6d262b7451bbef8debe970dd6b5 for earlier term 1. Current term is 2. Ops: []
I20260202 02:52:48.477547 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "33150e1456144b8e9c64cc2a2ac6dcca" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:48.477780 1046 raft_consensus.cc:2468] T 33150e1456144b8e9c64cc2a2ac6dcca P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:48.478660 1054 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b76d1837813c46e8a0925b49939d8884" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:48.479388 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "91ec01d25f8f441cbe3783cacbee51db" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:48.481122 715 leader_election.cc:304] T 33150e1456144b8e9c64cc2a2ac6dcca P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:48.481402 2669 raft_consensus.cc:2804] T 33150e1456144b8e9c64cc2a2ac6dcca P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:48.481484 2669 raft_consensus.cc:493] T 33150e1456144b8e9c64cc2a2ac6dcca P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:48.481555 2669 raft_consensus.cc:3060] T 33150e1456144b8e9c64cc2a2ac6dcca P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:48.487252 2669 raft_consensus.cc:515] T 33150e1456144b8e9c64cc2a2ac6dcca P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:48.487670 520 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet e07e87c0182b46a49b8ca64c072f4210
I20260202 02:52:48.488109 2669 leader_election.cc:290] T 33150e1456144b8e9c64cc2a2ac6dcca P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:48.488210 520 catalog_manager.cc:6724] ReplaceTablet: tablet e07e87c0182b46a49b8ca64c072f4210 deleted and replaced by tablet 6c4f969e31214b6ab61328ac2e8413b6
I20260202 02:52:48.488525 741 tablet_service.cc:1552] Processing DeleteTablet for tablet e07e87c0182b46a49b8ca64c072f4210 with delete_type TABLET_DATA_DELETED (replaced by tablet 6c4f969e31214b6ab61328ac2e8413b6) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:48.489667 2632 raft_consensus.cc:493] T 550895b4010f4088b2cc34acff50e03f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:48.489900 1180 ts_tablet_manager.cc:1929] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:48.489838 2632 raft_consensus.cc:515] T 550895b4010f4088b2cc34acff50e03f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:48.490135 1180 log.cc:1199] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/d67e91a022f1401ea724dff898c3b81b
I20260202 02:52:48.490466 2632 leader_election.cc:290] T 550895b4010f4088b2cc34acff50e03f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:48.490608 1180 ts_tablet_manager.cc:1950] T d67e91a022f1401ea724dff898c3b81b P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:48.492446 1035 tablet_service.cc:1552] Processing DeleteTablet for tablet e07e87c0182b46a49b8ca64c072f4210 with delete_type TABLET_DATA_DELETED (replaced by tablet 6c4f969e31214b6ab61328ac2e8413b6) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:48.493059 715 consensus_peers.cc:597] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fe53a8bdbd13417f9ce195c54a52a93c. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:48.494711 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "33150e1456144b8e9c64cc2a2ac6dcca" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:48.494920 1064 raft_consensus.cc:3060] T 33150e1456144b8e9c64cc2a2ac6dcca P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:48.497723 1064 raft_consensus.cc:2468] T 33150e1456144b8e9c64cc2a2ac6dcca P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:48.498413 715 leader_election.cc:304] T 33150e1456144b8e9c64cc2a2ac6dcca P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:48.498754 2669 raft_consensus.cc:2804] T 33150e1456144b8e9c64cc2a2ac6dcca P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:48.499008 2669 raft_consensus.cc:697] T 33150e1456144b8e9c64cc2a2ac6dcca P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:48.499290 2669 consensus_queue.cc:237] T 33150e1456144b8e9c64cc2a2ac6dcca P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:48.501221 880 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:41734 (request call id 144) took 2743 ms. Trace:
I20260202 02:52:48.501394 880 rpcz_store.cc:276] 0202 02:52:45.757731 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.757790 (+ 59us) service_pool.cc:225] Handling call
0202 02:52:47.835726 (+2077936us) spinlock_profiling.cc:243] Waited 67.8 ms on lock 0x6160000d9954. stack: 00007fb35529ca86 00007fb35529c840 00007fb3542773ad 0000000000984f94 0000000000984f5d 000000000098492a 00007fb3611592c0 00007fb365009b2f 00007fb35f42ec36 00007fb35f42e965 00007fb35c25f55e 00007fb35c25e43e 00007fb35c264675 00007fb35c264f9f 00007fb35c264dc8 0000000000a8970c
0202 02:52:48.501168 (+665442us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":155861248}
I20260202 02:52:48.506781 764 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "550895b4010f4088b2cc34acff50e03f" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:48.507434 998 leader_election.cc:343] T 550895b4010f4088b2cc34acff50e03f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:48.508800 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 538) took 22183 ms. Trace:
I20260202 02:52:48.508831 520 master_service.cc:954] ReplaceTablet: received request to replace tablet d594295561ff4b01bf13451533a5743c from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:48.509006 1180 rpcz_store.cc:276] 0202 02:52:26.325513 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.325559 (+ 46us) service_pool.cc:225] Handling call
0202 02:52:48.251949 (+21926390us) ts_tablet_manager.cc:1151] Deleting tablet d67e91a022f1401ea724dff898c3b81b
0202 02:52:48.251960 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.438650 (+186690us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.489760 (+ 51110us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.508670 (+ 18910us) spinlock_profiling.cc:243] Waited 17.5 ms on lock 0x61800000c518. stack: 00007f1e37b5aa86 00007f1e37b5a840 00007f1e36b353ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f1e37b433bd 00007f1e37b4442e 00007f1e4b7b392d 00007f1e4b731f23 00007f1e47956bc3 00007f1e47989744 00007f1e4796a240 00007f1e47969fcd 0000000000a8970c 00007f1e37ba0daf
0202 02:52:48.508777 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":40191744,"tablet-delete.queue_time_us":21926194}
I20260202 02:52:48.510224 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet d67e91a022f1401ea724dff898c3b81b (table default.Table_c245b83c1987454fabbe8646d14252d6 [id=08498a654b6a4a9ca3f968d70daad9b6]) successfully deleted
I20260202 02:52:48.510854 1180 tablet_replica.cc:333] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:48.511125 1180 raft_consensus.cc:2243] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:48.511492 1180 raft_consensus.cc:2272] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:48.512147 1180 ts_tablet_manager.cc:1916] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:48.517495 1175 tablet_bootstrap.cc:492] T 939fbcaf75c7439d9208bbbf83a72717 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:48.517778 1175 ts_tablet_manager.cc:1403] T 939fbcaf75c7439d9208bbbf83a72717 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.057s user 0.002s sys 0.001s
I20260202 02:52:48.518620 1175 raft_consensus.cc:359] T 939fbcaf75c7439d9208bbbf83a72717 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:48.518921 1175 raft_consensus.cc:385] T 939fbcaf75c7439d9208bbbf83a72717 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:48.519025 1175 raft_consensus.cc:740] T 939fbcaf75c7439d9208bbbf83a72717 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:48.519308 1175 consensus_queue.cc:260] T 939fbcaf75c7439d9208bbbf83a72717 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:48.520017 1175 ts_tablet_manager.cc:1434] T 939fbcaf75c7439d9208bbbf83a72717 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:48.522015 915 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "91ec01d25f8f441cbe3783cacbee51db" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:48.522162 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d2d27ceea1bb4122b18f0fcf4d2d1bf4" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.522266 919 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b76d1837813c46e8a0925b49939d8884" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:48.522300 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79c73f4a92044bafb5b8384229cb3a4d" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.523034 1175 tablet_bootstrap.cc:492] T 93f958924885457ab15f2aac5509e0aa P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:48.523064 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "70306ed9dec54286885d965f2c012da4" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.523088 918 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "91ec01d25f8f441cbe3783cacbee51db" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.523531 918 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "73f8db8f30f14d369d2227c9be7c4647" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.523612 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b76d1837813c46e8a0925b49939d8884" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.524011 918 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "63a9029bc39c4fbd8ac4bf4bc37d6b4e" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.524390 918 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28a900bcaa8b41718a285e595886d805" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.524482 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26b890ed87c3471b9fc0b172c90e0a11" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.524587 918 raft_consensus.cc:2393] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1f05820b6034228bdad884cc348716d in current term 3: Already voted for candidate 1521a6d262b7451bbef8debe970dd6b5 in this term.
I20260202 02:52:48.524608 923 raft_consensus.cc:2468] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 2.
I20260202 02:52:48.524922 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "80eb3013f5354022a2dd78bb294a8497" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.525465 527 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 939fbcaf75c7439d9208bbbf83a72717 (table default.Table_12d68a317c8e4d809dbbe77c095a1f6e [id=7219522ffc3849faab94d754a42a8511]). Expected version 1 got 0
W20260202 02:52:48.526050 527 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 939fbcaf75c7439d9208bbbf83a72717 (table default.Table_12d68a317c8e4d809dbbe77c095a1f6e [id=7219522ffc3849faab94d754a42a8511], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 939fbcaf75c7439d9208bbbf83a72717 consensus state has no leader
I20260202 02:52:48.527890 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "78f122342dd144b48de89902faa0f94f" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.528090 923 raft_consensus.cc:2393] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1f05820b6034228bdad884cc348716d in current term 2: Already voted for candidate 1521a6d262b7451bbef8debe970dd6b5 in this term.
I20260202 02:52:48.528532 918 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "25765fb687294e9db1289f76530dd877" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.529625 2667 raft_consensus.cc:493] T 80eb3013f5354022a2dd78bb294a8497 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
I20260202 02:52:48.529784 2667 raft_consensus.cc:515] T 80eb3013f5354022a2dd78bb294a8497 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:48.530691 527 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 939fbcaf75c7439d9208bbbf83a72717 (table default.Table_12d68a317c8e4d809dbbe77c095a1f6e [id=7219522ffc3849faab94d754a42a8511]). Expected version 1 got 0
I20260202 02:52:48.531148 2667 leader_election.cc:290] T 80eb3013f5354022a2dd78bb294a8497 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:48.531579 2667 raft_consensus.cc:493] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
I20260202 02:52:48.532032 2667 raft_consensus.cc:515] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:48.532894 2667 leader_election.cc:290] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:48.533108 520 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet d594295561ff4b01bf13451533a5743c
I20260202 02:52:48.533241 2667 raft_consensus.cc:493] T d06b77ff1a6f4f78b73082b00145fcf6 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
I20260202 02:52:48.533435 2667 raft_consensus.cc:515] T d06b77ff1a6f4f78b73082b00145fcf6 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:48.534204 2667 leader_election.cc:290] T d06b77ff1a6f4f78b73082b00145fcf6 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:48.534921 520 catalog_manager.cc:6724] ReplaceTablet: tablet d594295561ff4b01bf13451533a5743c deleted and replaced by tablet 137dae39f8f54be5af6354e2b6226adc
I20260202 02:52:48.534914 741 tablet_service.cc:1552] Processing DeleteTablet for tablet d594295561ff4b01bf13451533a5743c with delete_type TABLET_DATA_DELETED (replaced by tablet 137dae39f8f54be5af6354e2b6226adc) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:48.534943 1035 tablet_service.cc:1552] Processing DeleteTablet for tablet d594295561ff4b01bf13451533a5743c with delete_type TABLET_DATA_DELETED (replaced by tablet 137dae39f8f54be5af6354e2b6226adc) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:48.539934 521 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_cccb5565674d4157a97fa0a565bfeea2" } modify_external_catalogs: true
I20260202 02:52:48.540372 521 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_cccb5565674d4157a97fa0a565bfeea2" } modify_external_catalogs: true
I20260202 02:52:48.540570 520 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_3ee8fd9451364de0b9fbb7eb0aaa7990" } new_table_name: "default.Table_2679d310b29046beb6169593bb96e42f" modify_external_catalogs: true
I20260202 02:52:48.542174 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_99b0aec996a8431fba8dcd993bc5c846"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:48.542690 1056 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "80eb3013f5354022a2dd78bb294a8497" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:48.542806 1061 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d2d27ceea1bb4122b18f0fcf4d2d1bf4" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:48.542912 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_99b0aec996a8431fba8dcd993bc5c846 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:48.544214 1054 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d06b77ff1a6f4f78b73082b00145fcf6" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:48.548537 1175 tablet_bootstrap.cc:654] T 93f958924885457ab15f2aac5509e0aa P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:48.551738 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_7d923e5f4b2f4f5993cf00febdffa326"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:48.556164 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7d923e5f4b2f4f5993cf00febdffa326 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:48.558265 520 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 258e0afdba894f3cb2a3c7238e5634b7 consensus state has no leader
I20260202 02:52:48.562563 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_c00e5644e8f444a696c11cdb71e4e698"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:48.563045 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_d232fe693bd54d02996f4fca5e90e599"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:48.563665 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d232fe693bd54d02996f4fca5e90e599 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:48.565500 1174 tablet_bootstrap.cc:492] T 8bb14bf551f441dcbaaff7057b51b2be P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:48.565721 1174 ts_tablet_manager.cc:1403] T 8bb14bf551f441dcbaaff7057b51b2be P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.121s user 0.017s sys 0.003s
I20260202 02:52:48.566586 1174 raft_consensus.cc:359] T 8bb14bf551f441dcbaaff7057b51b2be P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:48.566880 1174 raft_consensus.cc:385] T 8bb14bf551f441dcbaaff7057b51b2be P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:48.567001 1174 raft_consensus.cc:740] T 8bb14bf551f441dcbaaff7057b51b2be P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:48.567356 1174 consensus_queue.cc:260] T 8bb14bf551f441dcbaaff7057b51b2be P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:48.572790 1182 ts_tablet_manager.cc:1929] T 3d1a7578e83447669e3c3972a426ad44 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:48.573043 1182 log.cc:1199] T 3d1a7578e83447669e3c3972a426ad44 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/3d1a7578e83447669e3c3972a426ad44
I20260202 02:52:48.574252 528 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_027e99b4f18f42948ac88f3658653753" } new_table_name: "default.Table_1366b68811c14d28b1785266d6f65a3a" modify_external_catalogs: true
I20260202 02:52:48.575723 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_5d3d1a79e21c432bbe40c8639f375fb1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:48.576346 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5d3d1a79e21c432bbe40c8639f375fb1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:48.577431 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c00e5644e8f444a696c11cdb71e4e698 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:48.593022 523 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_38b5d3fdb2c748cba3a6aaad03bdb766" } new_table_name: "default.Table_87de479e8a3f44909921b8f2f2fec67c" modify_external_catalogs: true
I20260202 02:52:48.595222 2720 tablet.cc:1722] T c050a2882f764dda8df2a7df6547f149 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:48.595955 525 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_06dc35989d444cccb418b9d2db62721d" } modify_external_catalogs: true
I20260202 02:52:48.596167 525 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_06dc35989d444cccb418b9d2db62721d" } modify_external_catalogs: true
I20260202 02:52:48.596810 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_274e129a05bd42a39cbee9731d492dac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:48.597373 712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260202 02:52:48.597383 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_274e129a05bd42a39cbee9731d492dac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:48.599300 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_a0b796c134ec461ba052a12b3cc914d1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:48.599713 1882 ts_tablet_manager.cc:1929] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:48.599836 1882 log.cc:1199] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/02b434fa058b45529ff3938b053263c6
W20260202 02:52:48.600093 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a0b796c134ec461ba052a12b3cc914d1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:48.600170 1882 ts_tablet_manager.cc:1950] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:48.604641 527 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_8fe88cb0c8f649b685a2c9f75d6ca759" } modify_external_catalogs: true
I20260202 02:52:48.604802 2724 tablet.cc:1722] T 70306ed9dec54286885d965f2c012da4 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:48.604955 527 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_8fe88cb0c8f649b685a2c9f75d6ca759" } modify_external_catalogs: true
I20260202 02:52:48.606885 2667 raft_consensus.cc:493] T 79c73f4a92044bafb5b8384229cb3a4d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:48.607059 2667 raft_consensus.cc:515] T 79c73f4a92044bafb5b8384229cb3a4d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:48.607887 2667 leader_election.cc:290] T 79c73f4a92044bafb5b8384229cb3a4d P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:48.589174 922 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c050a2882f764dda8df2a7df6547f149" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.589658 918 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "622a113af04a41c6852c29fc8a8acd78" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.591107 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d06b77ff1a6f4f78b73082b00145fcf6" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.601763 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26b890ed87c3471b9fc0b172c90e0a11" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:48.612562 2619 raft_consensus.cc:493] T dada209d982d48f3b74a49905f19c5b6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:48.612739 2619 raft_consensus.cc:515] T dada209d982d48f3b74a49905f19c5b6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:48.610431 1058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79c73f4a92044bafb5b8384229cb3a4d" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:48.613276 1058 raft_consensus.cc:2393] T 79c73f4a92044bafb5b8384229cb3a4d P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f442ef85029461091de847118c34c0d in current term 1: Already voted for candidate 1521a6d262b7451bbef8debe970dd6b5 in this term.
I20260202 02:52:48.613385 2619 leader_election.cc:290] T dada209d982d48f3b74a49905f19c5b6 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:48.610790 2721 tablet.cc:1722] T c050a2882f764dda8df2a7df6547f149 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:48.614377 1180 ts_tablet_manager.cc:1929] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:48.614501 1180 log.cc:1199] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/59990791123c4d4eb620090a1e8e31e1
I20260202 02:52:48.614925 1180 ts_tablet_manager.cc:1950] T 59990791123c4d4eb620090a1e8e31e1 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:48.614953 911 raft_consensus.cc:2468] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 2.
I20260202 02:52:48.615284 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 535) took 22292 ms. Trace:
I20260202 02:52:48.615373 1180 rpcz_store.cc:276] 0202 02:52:26.323294 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.328078 (+ 4784us) service_pool.cc:225] Handling call
0202 02:52:48.510790 (+22182712us) ts_tablet_manager.cc:1151] Deleting tablet 59990791123c4d4eb620090a1e8e31e1
0202 02:52:48.510804 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.554777 (+ 43973us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.614233 (+ 59456us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.615248 (+ 1015us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":22182482}
I20260202 02:52:48.603530 913 raft_consensus.cc:1240] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Rejecting Update request from peer b1f05820b6034228bdad884cc348716d for earlier term 1. Current term is 2. Ops: []
I20260202 02:52:48.615890 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 59990791123c4d4eb620090a1e8e31e1 (table default.Table_26642341d3d54c0fa9af19fef3a5b06d [id=9ff2eb6ee89943918019f5e899ac362b]) successfully deleted
I20260202 02:52:48.604245 919 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79c73f4a92044bafb5b8384229cb3a4d" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.604406 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "25765fb687294e9db1289f76530dd877" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:48.604769 921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8bb14bf551f441dcbaaff7057b51b2be" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.616796 1182 ts_tablet_manager.cc:1950] T 3d1a7578e83447669e3c3972a426ad44 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:48.617705 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 690) took 18968 ms. Trace:
I20260202 02:52:48.617983 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dada209d982d48f3b74a49905f19c5b6" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:48.618218 1045 raft_consensus.cc:2468] T dada209d982d48f3b74a49905f19c5b6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:48.619702 715 leader_election.cc:304] T dada209d982d48f3b74a49905f19c5b6 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:48.624384 2619 raft_consensus.cc:2804] T dada209d982d48f3b74a49905f19c5b6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:48.624514 2619 raft_consensus.cc:493] T dada209d982d48f3b74a49905f19c5b6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:48.624610 2619 raft_consensus.cc:3060] T dada209d982d48f3b74a49905f19c5b6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:48.620402 1175 tablet_bootstrap.cc:492] T 93f958924885457ab15f2aac5509e0aa P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:48.625718 889 tablet_service.cc:1552] Processing DeleteTablet for tablet ac2ab12f5040488aa306cb69b107b485 with delete_type TABLET_DATA_DELETED (replaced by tablet 898778f8f0b548ec89cb1417d6f07926) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:48.627514 1175 ts_tablet_manager.cc:1403] T 93f958924885457ab15f2aac5509e0aa P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.105s user 0.002s sys 0.004s
I20260202 02:52:48.628489 1175 raft_consensus.cc:359] T 93f958924885457ab15f2aac5509e0aa P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:48.628741 1175 raft_consensus.cc:385] T 93f958924885457ab15f2aac5509e0aa P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:48.628836 1175 raft_consensus.cc:740] T 93f958924885457ab15f2aac5509e0aa P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:48.629079 1175 consensus_queue.cc:260] T 93f958924885457ab15f2aac5509e0aa P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:48.629635 1175 ts_tablet_manager.cc:1434] T 93f958924885457ab15f2aac5509e0aa P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:48.630189 1175 tablet_bootstrap.cc:492] T 587cfd0ae62f4cfd937faf6890e78f4a P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:48.631671 1175 tablet_bootstrap.cc:654] T 587cfd0ae62f4cfd937faf6890e78f4a P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:48.632200 825 heartbeater.cc:344] Connected to a master server at 127.0.104.190:33981
I20260202 02:52:48.634171 1174 ts_tablet_manager.cc:1434] T 8bb14bf551f441dcbaaff7057b51b2be P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.054s user 0.001s sys 0.001s
I20260202 02:52:48.634516 1174 tablet_bootstrap.cc:492] T b7880cb051c24bc998003f0effed3cf8 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:48.634418 2619 raft_consensus.cc:515] T dada209d982d48f3b74a49905f19c5b6 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:48.634539 1175 tablet_bootstrap.cc:492] T 587cfd0ae62f4cfd937faf6890e78f4a P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:48.634845 1175 ts_tablet_manager.cc:1403] T 587cfd0ae62f4cfd937faf6890e78f4a P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260202 02:52:48.635082 2619 leader_election.cc:290] T dada209d982d48f3b74a49905f19c5b6 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:48.635527 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 543) took 22219 ms. Trace:
I20260202 02:52:48.635638 1175 raft_consensus.cc:359] T 587cfd0ae62f4cfd937faf6890e78f4a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:48.635699 1180 rpcz_store.cc:276] 0202 02:52:26.415692 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.418963 (+ 3271us) service_pool.cc:225] Handling call
0202 02:52:48.628881 (+22209918us) ts_tablet_manager.cc:1151] Deleting tablet 7e94cacaa0714f92ad6a2b2ab6171f44
0202 02:52:48.628894 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.635486 (+ 6592us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":22205455}
W20260202 02:52:48.636230 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 7e94cacaa0714f92ad6a2b2ab6171f44 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7e94cacaa0714f92ad6a2b2ab6171f44 already in progress: creating tablet
I20260202 02:52:48.636693 1180 tablet_replica.cc:333] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:48.636982 1180 raft_consensus.cc:2243] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:48.637367 1180 raft_consensus.cc:2272] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:48.642321 528 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet eea6bb04ddf24276b46270d009ed78b4 consensus state has no leader
I20260202 02:52:48.644490 520 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_d232fe693bd54d02996f4fca5e90e599" } new_table_name: "default.Table_339b1866686f40e689308903d8ab370e" modify_external_catalogs: true
I20260202 02:52:48.644861 526 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_b82204d74f4949769570a9720fe4b5af" } modify_external_catalogs: true
I20260202 02:52:48.645016 526 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_b82204d74f4949769570a9720fe4b5af" } modify_external_catalogs: true
I20260202 02:52:48.646211 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_f7636b54f7f34f2a9487c03e57a621a9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:48.646797 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f7636b54f7f34f2a9487c03e57a621a9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:48.640044 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dada209d982d48f3b74a49905f19c5b6" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:48.653498 1045 raft_consensus.cc:3060] T dada209d982d48f3b74a49905f19c5b6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:48.658160 1045 raft_consensus.cc:2468] T dada209d982d48f3b74a49905f19c5b6 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:48.659859 2731 tablet.cc:1722] T af5aa68e47bb420eac300754d274fa73 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:48.663961 1175 raft_consensus.cc:385] T 587cfd0ae62f4cfd937faf6890e78f4a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:48.664160 1175 raft_consensus.cc:740] T 587cfd0ae62f4cfd937faf6890e78f4a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:48.629777 1182 rpcz_store.cc:276] 0202 02:52:29.654044 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.654424 (+ 380us) service_pool.cc:225] Handling call
0202 02:52:48.364371 (+18709947us) ts_tablet_manager.cc:1151] Deleting tablet 3d1a7578e83447669e3c3972a426ad44
0202 02:52:48.364381 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.515801 (+151420us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.572662 (+ 56861us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.617687 (+ 45025us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":18709702}
I20260202 02:52:48.639120 899 tablet_service.cc:1552] Processing DeleteTablet for tablet d0ce2121d65b4a228596e0ca9018883d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:48.639436 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 3dcf4f794ec14b2f9d181d015fc437fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:30 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:48.639757 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 09a8aa4dd70b4fe2806293d2620e521c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:30 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:48.639892 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 75a24a1bf54d4c799938a5723ac6fa99 with delete_type TABLET_DATA_DELETED (replaced by tablet 6750a1212ceb4e6e82e4ffa93caab590) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:48.640022 898 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41736: Illegal state: replica 1521a6d262b7451bbef8debe970dd6b5 is not leader of this config: current role FOLLOWER
I20260202 02:52:48.665009 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 693) took 18799 ms. Trace:
I20260202 02:52:48.650950 896 tablet_service.cc:1552] Processing DeleteTablet for tablet b8f2b34b4f1e4ff2b2be4ff40aeb0c66 with delete_type TABLET_DATA_DELETED (replaced by tablet 912ac4044b5645caa7eda2a565c5b8bb) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:48.651297 894 tablet_service.cc:1505] Processing CreateTablet for tablet 32c3fece7c2145fb86bbc11c4c221f27 (DEFAULT_TABLE table=default.Table_31ed23ec1b894b32953e20125e57d70a [id=0fb5920dae13430c81292070d04bb203]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.665534 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32c3fece7c2145fb86bbc11c4c221f27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.651988 903 tablet_service.cc:1505] Processing CreateTablet for tablet 6750a1212ceb4e6e82e4ffa93caab590 (DEFAULT_TABLE table=default.Table_52a2f389faba40fea93e2bca972e81e1 [id=70ac373200604907b00132eb4727ea5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.652036 893 tablet_service.cc:1505] Processing CreateTablet for tablet 6b4bfc4df8ab487a924c0e04e97d2c09 (DEFAULT_TABLE table=default.Table_a2c0065f3593412fa5958d59d354f437 [id=ca66b8adfee44b6ab8dd973e36a26597]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.652565 892 tablet_service.cc:1505] Processing CreateTablet for tablet 912ac4044b5645caa7eda2a565c5b8bb (DEFAULT_TABLE table=default.Table_0ee7c010ec144c2e9d633447e4d84dcf [id=812637d0b1da4789850022a699e6bbd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.652689 2445 raft_consensus.cc:493] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:48.653014 887 tablet_service.cc:1505] Processing CreateTablet for tablet 461ecb9ee90f4e94b12934c56de475a7 (DEFAULT_TABLE table=default.Table_f468fb0750cf421e833dea1a00d62640 [id=88ea44b25a404b15901d1a267de4f03a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.668777 903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6750a1212ceb4e6e82e4ffa93caab590. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.668674 2445 raft_consensus.cc:515] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:48.668984 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 461ecb9ee90f4e94b12934c56de475a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.669358 2445 leader_election.cc:290] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:48.653298 895 tablet_service.cc:1505] Processing CreateTablet for tablet a402b238ae2b4a379598019914c68469 (DEFAULT_TABLE table=default.Table_33c051aaf8e1494fa559ae70367f89e9 [id=8e5a555f4f44492aa00b05de07ca2fed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.654055 2443 consensus_queue.cc:1059] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260202 02:52:48.657938 2732 tablet.cc:1722] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:48.672595 2443 raft_consensus.cc:3055] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260202 02:52:48.672648 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a402b238ae2b4a379598019914c68469. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.672696 2443 raft_consensus.cc:740] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:48.672935 2443 consensus_queue.cc:260] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:48.673290 2443 raft_consensus.cc:3060] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:48.675964 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b4bfc4df8ab487a924c0e04e97d2c09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.676559 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 912ac4044b5645caa7eda2a565c5b8bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.665149 1182 rpcz_store.cc:276] 0202 02:52:29.865849 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.865890 (+ 41us) service_pool.cc:225] Handling call
0202 02:52:48.664878 (+18798988us) ts_tablet_manager.cc:1151] Deleting tablet 1d98c7a46139413b8f0d3697b74b1e52
0202 02:52:48.664891 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.664990 (+ 99us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":18798806}
I20260202 02:52:48.682472 2445 raft_consensus.cc:493] T 939fbcaf75c7439d9208bbbf83a72717 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:48.682618 2445 raft_consensus.cc:515] T 939fbcaf75c7439d9208bbbf83a72717 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:48.683147 2445 leader_election.cc:290] T 939fbcaf75c7439d9208bbbf83a72717 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:48.684135 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "939fbcaf75c7439d9208bbbf83a72717" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:48.684589 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 645) took 19651 ms. Trace:
W20260202 02:52:48.684770 859 leader_election.cc:343] T 939fbcaf75c7439d9208bbbf83a72717 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:48.684689 1882 rpcz_store.cc:276] 0202 02:52:29.033325 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.033467 (+ 142us) service_pool.cc:225] Handling call
0202 02:52:48.454086 (+19420619us) ts_tablet_manager.cc:1151] Deleting tablet 02b434fa058b45529ff3938b053263c6
0202 02:52:48.454094 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.566950 (+112856us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.599584 (+ 32634us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.684572 (+ 84988us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":19420447}
I20260202 02:52:48.685039 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 646) took 19651 ms. Trace:
I20260202 02:52:48.685109 1882 rpcz_store.cc:276] 0202 02:52:29.033415 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.033677 (+ 262us) service_pool.cc:225] Handling call
0202 02:52:48.684955 (+19651278us) ts_tablet_manager.cc:1151] Deleting tablet 94bada52db6a47179415d302b541d01f
0202 02:52:48.684965 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.685027 (+ 62us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":19651128}
I20260202 02:52:48.685334 1882 tablet_replica.cc:333] T d30557fbe9084cd3860820f69dda8752 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:48.685496 1882 raft_consensus.cc:2243] T d30557fbe9084cd3860820f69dda8752 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:48.685647 1882 raft_consensus.cc:2272] T d30557fbe9084cd3860820f69dda8752 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:48.686510 1882 ts_tablet_manager.cc:1916] T d30557fbe9084cd3860820f69dda8752 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:48.687330 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 02b434fa058b45529ff3938b053263c6 (table default.Table_6133c617ff8d40eca5cbde96533acd82 [id=5a502f86d98b42c6a3e274de5bf08544]) successfully deleted
W20260202 02:52:48.687667 515 catalog_manager.cc:4952] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 94bada52db6a47179415d302b541d01f because the tablet was not found. No further retry: Not found: Tablet not found: 94bada52db6a47179415d302b541d01f
I20260202 02:52:48.691310 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 691) took 19000 ms. Trace:
I20260202 02:52:48.691499 1182 rpcz_store.cc:276] 0202 02:52:29.731296 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.876728 (+145432us) service_pool.cc:225] Handling call
0202 02:52:48.681143 (+18804415us) ts_tablet_manager.cc:1151] Deleting tablet 67495e8e525542fb976f675b71512ed2
0202 02:52:48.691167 (+ 10024us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.691280 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":18804195}
I20260202 02:52:48.684056 1053 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8b0368e03f644f1db28dbb57c95c48c1" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:48.695472 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 694) took 18829 ms. Trace:
I20260202 02:52:48.695633 1182 rpcz_store.cc:276] 0202 02:52:29.866189 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.883158 (+ 16969us) service_pool.cc:225] Handling call
0202 02:52:48.695304 (+18812146us) ts_tablet_manager.cc:1151] Deleting tablet 78a9a483b95c4bb5a5ab69abc899837b
0202 02:52:48.695321 (+ 17us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.695446 (+ 125us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":18810380}
I20260202 02:52:48.696283 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 692) took 18830 ms. Trace:
I20260202 02:52:48.696447 1182 rpcz_store.cc:276] 0202 02:52:29.865763 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.897360 (+ 31597us) service_pool.cc:225] Handling call
0202 02:52:48.695994 (+18798634us) ts_tablet_manager.cc:1151] Deleting tablet 8dd62de8e8234d6c9ce380b1aadc097d
0202 02:52:48.696004 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.696083 (+ 79us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":18798399}
W20260202 02:52:48.697413 859 leader_election.cc:343] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:48.697816 1182 tablet_replica.cc:333] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:48.698446 1182 raft_consensus.cc:2243] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:48.698781 1110 heartbeater.cc:344] Connected to a master server at 127.0.104.190:33981
W20260202 02:52:48.702651 523 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9636b45987f34d16b717a8d918cc4bd6 consensus state has no leader
I20260202 02:52:48.707409 1182 raft_consensus.cc:2272] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:48.708865 886 tablet_service.cc:1505] Processing CreateTablet for tablet e2378f5d9e3b48deb0d33f462994aad5 (DEFAULT_TABLE table=default.Table_025d64cc432643acbbef5c3fc8203457 [id=b73c24871d3d48b6accfe9a1a27a8071]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.709326 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2378f5d9e3b48deb0d33f462994aad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.709674 1182 ts_tablet_manager.cc:1916] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:48.709630 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_f02b61a92b1e4552968931092b466a76"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:48.710198 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f02b61a92b1e4552968931092b466a76 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:48.711809 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 79c73f4a92044bafb5b8384229cb3a4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:47 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:48.712028 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 842e54adae2a452a98521de24326eaff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:47 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:48.713692 523 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_8262b850970641b1bbe1cf9d1c3d9409" } new_table_name: "default.Table_a9d14eea240d4bca96de4ea77b5b31f2" modify_external_catalogs: true
I20260202 02:52:48.716602 2731 tablet.cc:1722] T d06b77ff1a6f4f78b73082b00145fcf6 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260202 02:52:48.718791 893 tablet_service.cc:1505] Processing CreateTablet for tablet 8a206159afa84ae19f8c7763a2fe43d4 (DEFAULT_TABLE table=default.Table_06dc35989d444cccb418b9d2db62721d [id=e5f4256ed9434c52b0b71d84c58c0508]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.702355 901 tablet_service.cc:1505] Processing CreateTablet for tablet f87ee8d6ff9042c08ba2e32724bea720 (DEFAULT_TABLE table=default.Table_48929b86815b49369aff936433cae314 [id=2be03edfb43944be9714d5aa217263f8]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:48.719422 423 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 65291.7us
I20260202 02:52:48.719686 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f87ee8d6ff9042c08ba2e32724bea720. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.727172 886 tablet_service.cc:1505] Processing CreateTablet for tablet 9f157e0fec954c198c790b10e01c6886 (DEFAULT_TABLE table=default.Table_2918d36968d74e4fb54ea2f78f039dfc [id=a9df59b6733f4ce8808a6e0b15f12d40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.727586 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f157e0fec954c198c790b10e01c6886. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.728251 1180 ts_tablet_manager.cc:1916] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:48.729487 2720 tablet.cc:1722] T 70306ed9dec54286885d965f2c012da4 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:48.703336 903 tablet_service.cc:1505] Processing CreateTablet for tablet 15366555b90a408c9f46222b44e7efa6 (DEFAULT_TABLE table=default.Table_46283f6baa2d4a9d9fdb8c1e501bcf89 [id=1cd4342c241a4072a65393b0be0c26a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.703840 894 tablet_service.cc:1505] Processing CreateTablet for tablet 6d475fa5186243229c1db0fbdc531f4e (DEFAULT_TABLE table=default.Table_caf9a47c30ad4c6293b84e85cbde60eb [id=68f222efa4cf43b3804c194400d89112]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.704468 896 tablet_service.cc:1552] Processing DeleteTablet for tablet f94f25f4059a43b8babedb6b45a08a19 with delete_type TABLET_DATA_DELETED (replaced by tablet 76d3e799f5cb46638b289d06fca2f369) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:48.704924 889 tablet_service.cc:1552] Processing DeleteTablet for tablet abfe7f76b2424c4d9ab07b29b66ec103 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:47 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:48.730810 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d475fa5186243229c1db0fbdc531f4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.705461 898 tablet_service.cc:1552] Processing DeleteTablet for tablet dd8f36d7a82f47adae1bd3c1f8628b6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:47 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:48.705482 899 tablet_service.cc:1505] Processing CreateTablet for tablet 23dfa02219d748919e3cf24beb2095be (DEFAULT_TABLE table=default.Table_3be2f2becb3746a8a26896e806ee480a [id=88180dd0482c4b66a4c54cd78d74bb7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.705785 887 tablet_service.cc:1505] Processing CreateTablet for tablet c438e69c40c24655b37f2e73b4b3b845 (DEFAULT_TABLE table=default.Table_ff96a2dc345940448683f8be23859993 [id=8e592b0d560f48ed86a5c3886f178c2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.706162 895 tablet_service.cc:1505] Processing CreateTablet for tablet 9636b45987f34d16b717a8d918cc4bd6 (DEFAULT_TABLE table=default.Table_38b5d3fdb2c748cba3a6aaad03bdb766 [id=9cb73f418ae84400871b2e81ff676c8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.706477 892 tablet_service.cc:1505] Processing CreateTablet for tablet 76d3e799f5cb46638b289d06fca2f369 (DEFAULT_TABLE table=default.Table_22445ccfd7b64298b92a0f70733f3625 [id=300ea019d2724bee8211d7462710dc26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.719213 2732 tablet.cc:1722] T d06b77ff1a6f4f78b73082b00145fcf6 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260202 02:52:48.732292 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a206159afa84ae19f8c7763a2fe43d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.700990 1175 consensus_queue.cc:260] T 587cfd0ae62f4cfd937faf6890e78f4a P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:48.732586 903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15366555b90a408c9f46222b44e7efa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.732960 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23dfa02219d748919e3cf24beb2095be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.741209 2443 raft_consensus.cc:493] T 587cfd0ae62f4cfd937faf6890e78f4a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:48.741355 2443 raft_consensus.cc:515] T 587cfd0ae62f4cfd937faf6890e78f4a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:48.742113 2443 leader_election.cc:290] T 587cfd0ae62f4cfd937faf6890e78f4a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:48.742429 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76d3e799f5cb46638b289d06fca2f369. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.742620 1053 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "587cfd0ae62f4cfd937faf6890e78f4a" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:48.742977 1175 ts_tablet_manager.cc:1434] T 587cfd0ae62f4cfd937faf6890e78f4a P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.108s user 0.000s sys 0.002s
W20260202 02:52:48.743170 859 leader_election.cc:343] T 587cfd0ae62f4cfd937faf6890e78f4a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:48.744547 1175 tablet_bootstrap.cc:492] T 65bd1c1a382e42e4a68077264563c55b P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:48.745826 1175 tablet_bootstrap.cc:654] T 65bd1c1a382e42e4a68077264563c55b P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:48.747126 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c438e69c40c24655b37f2e73b4b3b845. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.748307 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9636b45987f34d16b717a8d918cc4bd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.753196 901 tablet_service.cc:1552] Processing DeleteTablet for tablet 3338964dfed548f980f58147f0283691 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:48 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:48.754297 898 tablet_service.cc:1552] Processing DeleteTablet for tablet d594295561ff4b01bf13451533a5743c with delete_type TABLET_DATA_DELETED (replaced by tablet 137dae39f8f54be5af6354e2b6226adc) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:48.754401 899 tablet_service.cc:1552] Processing DeleteTablet for tablet 0138ba7326674741b0729324735f1de7 with delete_type TABLET_DATA_DELETED (replaced by tablet 6671d3b545eb4371bfa71fe703c214f2) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:48.753413 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 118753e8cb344b47a35a745aa8fed265 with delete_type TABLET_DATA_DELETED (replaced by tablet 0aa023e74e74480498b721bbb5d15052) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:48.754547 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 93f958924885457ab15f2aac5509e0aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:25 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:48.754616 901 tablet_service.cc:1552] Processing DeleteTablet for tablet 2d18485f5a4a4b11916c64a487cd28b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:48 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:48.754746 892 tablet_service.cc:1552] Processing DeleteTablet for tablet 48863e2d60c84460858daabb9a08dcd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:48 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:48.754891 903 tablet_service.cc:1552] Processing DeleteTablet for tablet e07e87c0182b46a49b8ca64c072f4210 with delete_type TABLET_DATA_DELETED (replaced by tablet 6c4f969e31214b6ab61328ac2e8413b6) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:48.754899 894 tablet_service.cc:1552] Processing DeleteTablet for tablet f87ee8d6ff9042c08ba2e32724bea720 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:48 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:48.756651 1174 tablet_bootstrap.cc:654] T b7880cb051c24bc998003f0effed3cf8 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:48.758384 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 474df41e4fe24a818fa45a4ce84d5474 (table default.Table_96e21d4e03c941129f0581aa558481c0 [id=14bbe505bc8b4a4bbde264a2c0dcdf50]) successfully deleted
I20260202 02:52:48.758734 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 1f379a29e2454f689d68f5951837512b (table default.Table_67c55c53ca254b03b0c326672a41b559 [id=7e80e30271434635a17d0034580aeca8]) successfully deleted
I20260202 02:52:48.759306 715 leader_election.cc:304] T dada209d982d48f3b74a49905f19c5b6 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:48.760167 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet a0b5d82e5fb94b60b4aa324b978a5378 (table default.Table_cf37f87385ee44cea1ff80cc14050ff2 [id=697b6fe62ee845d895af350881a03556]) successfully deleted
I20260202 02:52:48.760514 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f6746ccbf37e477196a9706396d1e3fd" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:48.760666 714 consensus_peers.cc:597] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 13cd1383348b458894fdecc22bf24b74. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:48.761648 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28a900bcaa8b41718a285e595886d805" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:48.761557 763 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "78f122342dd144b48de89902faa0f94f" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:48.763981 1173 raft_consensus.cc:385] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:48.764096 1173 raft_consensus.cc:740] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:48.764312 1173 consensus_queue.cc:260] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:48.764653 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "44c8d5b74c8e4c499f7f68e8185efc17" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:48.764974 763 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6b8f924aecf44e959e47d2f32e6b0bf5" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:48.765030 768 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "be993c1512b04df4b92cab3195241d84" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:48.764984 2448 raft_consensus.cc:2804] T dada209d982d48f3b74a49905f19c5b6 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:48.761432 779 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "842e54adae2a452a98521de24326eaff" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:48.765759 715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 100302us
I20260202 02:52:48.760854 2723 tablet.cc:1722] T af5aa68e47bb420eac300754d274fa73 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:48.771657 766 raft_consensus.cc:1240] T b76d1837813c46e8a0925b49939d8884 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Rejecting Update request from peer 1521a6d262b7451bbef8debe970dd6b5 for earlier term 1. Current term is 2. Ops: []
I20260202 02:52:48.765661 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "70306ed9dec54286885d965f2c012da4" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:48.771265 765 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79c73f4a92044bafb5b8384229cb3a4d" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:48.771253 2448 raft_consensus.cc:697] T dada209d982d48f3b74a49905f19c5b6 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:48.773895 2731 tablet.cc:1722] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20260202 02:52:48.774821 764 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e1284fde67bc4d65b067a54c416ccc33" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:48.777690 893 tablet_service.cc:1552] Processing DeleteTablet for tablet d2d27ceea1bb4122b18f0fcf4d2d1bf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:48 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:48.777897 1173 ts_tablet_manager.cc:1434] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.390s user 0.012s sys 0.005s
I20260202 02:52:48.778250 999 leader_election.cc:304] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
W20260202 02:52:48.778642 999 leader_election.cc:343] T dada209d982d48f3b74a49905f19c5b6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: dada209d982d48f3b74a49905f19c5b6
I20260202 02:52:48.778743 999 leader_election.cc:304] T dada209d982d48f3b74a49905f19c5b6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:48.779712 2482 raft_consensus.cc:2749] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260202 02:52:48.779886 2482 raft_consensus.cc:2749] T dada209d982d48f3b74a49905f19c5b6 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:48.780781 520 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a5a7f612cde040df956bf6741f7f2760 has no consensus state
I20260202 02:52:48.781172 520 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_4799ebe642db4543be7712203e1fe532" } modify_external_catalogs: true
I20260202 02:52:48.781360 520 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_4799ebe642db4543be7712203e1fe532" } modify_external_catalogs: true
I20260202 02:52:48.781908 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_1cbc82d8edee4ef58a1beda6efdca8bc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:48.782464 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1cbc82d8edee4ef58a1beda6efdca8bc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:48.783538 526 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet bed432e27aa841109fa13d4d7604af23
I20260202 02:52:48.783776 526 master_service.cc:954] ReplaceTablet: received request to replace tablet d0ce2121d65b4a228596e0ca9018883d from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:48.784348 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_b308f5c7056f4abda5b3c64a7a4173d3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:48.784904 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b308f5c7056f4abda5b3c64a7a4173d3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:48.788748 765 raft_consensus.cc:3060] T 79c73f4a92044bafb5b8384229cb3a4d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:48.789012 714 consensus_peers.cc:597] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d30557fbe9084cd3860820f69dda8752. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:48.793279 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 252d16b6b37346b2908862b509c4c8cc (DEFAULT_TABLE table=default.Table_22e604b386714cefbe344e7f6d6c3d6e [id=fa167d2a101641ffbb21593606f5a711]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.793476 1041 tablet_service.cc:1505] Processing CreateTablet for tablet 4552fa1c83424db2b0c7bc390c3bc2b6 (DEFAULT_TABLE table=default.Table_bd3fed1d460846fe81f7c0f86074b2ca [id=7b113a4090c24abe8376093a5b7a353f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.793896 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 14ec41dfdf2648319ba08111c5e7c5e8 (DEFAULT_TABLE table=default.Table_ab48976710ec44e1a6b293a951fab152 [id=d2f8f193d18047a4924a90fc2d4ae1de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.794255 1034 tablet_service.cc:1505] Processing CreateTablet for tablet 491784bba44c45bdbe3a762dffe85d9e (DEFAULT_TABLE table=default.Table_d5520a9602ad4f69bd95b3eeef903589 [id=1cf5b0b923754af08ba5b1eb9cbe908d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.793279 1035 tablet_service.cc:1505] Processing CreateTablet for tablet d06e48923cd4452bae0538c94c527bb2 (DEFAULT_TABLE table=default.Table_b1e92e2bd13d4f12906c13ff7e91f109 [id=a1653b0201104a06afd16cd2ae2fb0b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.795148 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 3d1a7578e83447669e3c3972a426ad44 (table default.Table_7b5f2de229cb4b2f8bd174ccfd6b9553 [id=97a0437f66864a7291b966c9b72c0eb7]) successfully deleted
I20260202 02:52:48.796052 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 252d16b6b37346b2908862b509c4c8cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.795960 524 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_e72399a9ba4b4a7a9259eec83a2f1f71" } new_table_name: "default.Table_19b44b1161d14170bb703846ec8fe6d8" modify_external_catalogs: true
I20260202 02:52:48.796523 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 491784bba44c45bdbe3a762dffe85d9e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:48.798429 514 catalog_manager.cc:5064] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): alter failed for tablet 26b890ed87c3471b9fc0b172c90e0a11 (table default.Table_044ef6a2b1af4e3cab3c8cf00505c6a6 [id=d2e4220a0cbf443e8380433da4f064f8]): Illegal state: replica 1521a6d262b7451bbef8debe970dd6b5 is not leader of this config: current role FOLLOWER
W20260202 02:52:48.799856 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 1d98c7a46139413b8f0d3697b74b1e52 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1d98c7a46139413b8f0d3697b74b1e52 already in progress: creating tablet
W20260202 02:52:48.800160 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 67495e8e525542fb976f675b71512ed2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 67495e8e525542fb976f675b71512ed2 already in progress: creating tablet
W20260202 02:52:48.800379 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 78a9a483b95c4bb5a5ab69abc899837b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 78a9a483b95c4bb5a5ab69abc899837b already in progress: creating tablet
W20260202 02:52:48.800958 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 8dd62de8e8234d6c9ce380b1aadc097d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8dd62de8e8234d6c9ce380b1aadc097d already in progress: creating tablet
I20260202 02:52:48.801151 915 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d2d27ceea1bb4122b18f0fcf4d2d1bf4" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.801187 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "16b47a2831f54d809ed0b16deaec7e3d" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:48.801491 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8bb14bf551f441dcbaaff7057b51b2be" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:48.801571 921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d06b77ff1a6f4f78b73082b00145fcf6" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.801854 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79c73f4a92044bafb5b8384229cb3a4d" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
W20260202 02:52:48.802131 999 consensus_peers.cc:597] T 3d1a7578e83447669e3c3972a426ad44 P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:48.802374 999 leader_election.cc:343] T 33150e1456144b8e9c64cc2a2ac6dcca P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 33150e1456144b8e9c64cc2a2ac6dcca
I20260202 02:52:48.802476 999 leader_election.cc:304] T 33150e1456144b8e9c64cc2a2ac6dcca P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
W20260202 02:52:48.802850 999 leader_election.cc:343] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 75569511179b41038e521fa3346a92ae
I20260202 02:52:48.802984 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "80eb3013f5354022a2dd78bb294a8497" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.803097 999 leader_election.cc:304] T 70306ed9dec54286885d965f2c012da4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:48.803421 999 leader_election.cc:304] T 91ec01d25f8f441cbe3783cacbee51db P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:48.804230 2632 raft_consensus.cc:2749] T 33150e1456144b8e9c64cc2a2ac6dcca P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:48.804289 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14ec41dfdf2648319ba08111c5e7c5e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.804399 2632 raft_consensus.cc:2749] T 70306ed9dec54286885d965f2c012da4 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260202 02:52:48.804523 2632 raft_consensus.cc:2749] T 91ec01d25f8f441cbe3783cacbee51db P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260202 02:52:48.809182 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4552fa1c83424db2b0c7bc390c3bc2b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.809350 1182 ts_tablet_manager.cc:1929] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:48.809563 1182 log.cc:1199] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/be993c1512b04df4b92cab3195241d84
I20260202 02:52:48.810240 1182 ts_tablet_manager.cc:1950] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:48.810766 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 695) took 18898 ms. Trace:
I20260202 02:52:48.814700 2731 tablet.cc:1722] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260202 02:52:48.813858 2732 tablet.cc:1722] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260202 02:52:48.816016 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d06e48923cd4452bae0538c94c527bb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.816176 895 tablet_service.cc:1505] Processing CreateTablet for tablet d06e48923cd4452bae0538c94c527bb2 (DEFAULT_TABLE table=default.Table_b1e92e2bd13d4f12906c13ff7e91f109 [id=a1653b0201104a06afd16cd2ae2fb0b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.812243 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28a900bcaa8b41718a285e595886d805" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:48.816548 923 raft_consensus.cc:2393] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1f05820b6034228bdad884cc348716d in current term 3: Already voted for candidate 1521a6d262b7451bbef8debe970dd6b5 in this term.
I20260202 02:52:48.817152 1175 tablet_bootstrap.cc:492] T 65bd1c1a382e42e4a68077264563c55b P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:48.817813 1175 ts_tablet_manager.cc:1403] T 65bd1c1a382e42e4a68077264563c55b P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.073s user 0.002s sys 0.000s
I20260202 02:52:48.817754 903 tablet_service.cc:1505] Processing CreateTablet for tablet 491784bba44c45bdbe3a762dffe85d9e (DEFAULT_TABLE table=default.Table_d5520a9602ad4f69bd95b3eeef903589 [id=1cf5b0b923754af08ba5b1eb9cbe908d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.818058 857 leader_election.cc:304] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:48.817559 893 tablet_service.cc:1505] Processing CreateTablet for tablet 252d16b6b37346b2908862b509c4c8cc (DEFAULT_TABLE table=default.Table_22e604b386714cefbe344e7f6d6c3d6e [id=fa167d2a101641ffbb21593606f5a711]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.819288 1182 rpcz_store.cc:276] 0202 02:52:29.912984 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.914746 (+ 1762us) service_pool.cc:225] Handling call
0202 02:52:48.697769 (+18783023us) ts_tablet_manager.cc:1151] Deleting tablet be993c1512b04df4b92cab3195241d84
0202 02:52:48.697778 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.718012 (+ 20234us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.809220 (+ 91208us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.810748 (+ 1528us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":18460958}
I20260202 02:52:48.819890 1182 tablet_replica.cc:333] T d8418a9dd0944456b4d4820dde0cde02 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:48.820137 1182 raft_consensus.cc:2243] T d8418a9dd0944456b4d4820dde0cde02 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:48.820370 1182 raft_consensus.cc:2272] T d8418a9dd0944456b4d4820dde0cde02 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:48.820456 1175 raft_consensus.cc:359] T 65bd1c1a382e42e4a68077264563c55b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:48.823123 2666 raft_consensus.cc:2749] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260202 02:52:48.824311 1175 raft_consensus.cc:385] T 65bd1c1a382e42e4a68077264563c55b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:48.824582 1175 raft_consensus.cc:740] T 65bd1c1a382e42e4a68077264563c55b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:48.825143 1175 consensus_queue.cc:260] T 65bd1c1a382e42e4a68077264563c55b P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:48.826417 857 leader_election.cc:304] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:48.827490 999 leader_election.cc:304] T b76d1837813c46e8a0925b49939d8884 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:48.827839 2632 raft_consensus.cc:2749] T b76d1837813c46e8a0925b49939d8884 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260202 02:52:48.828423 999 leader_election.cc:304] T 73f8db8f30f14d369d2227c9be7c4647 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:48.828747 2632 raft_consensus.cc:2749] T 73f8db8f30f14d369d2227c9be7c4647 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260202 02:52:48.829166 999 leader_election.cc:343] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 026e6ee0b3ae4d938fdf1550044ed1fa
I20260202 02:52:48.829610 999 leader_election.cc:304] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:48.829900 2632 raft_consensus.cc:2749] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260202 02:52:48.830395 999 leader_election.cc:304] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:48.830657 2632 raft_consensus.cc:2749] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260202 02:52:48.830977 999 leader_election.cc:343] T d30557fbe9084cd3860820f69dda8752 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: d30557fbe9084cd3860820f69dda8752
I20260202 02:52:48.831128 999 leader_election.cc:304] T d30557fbe9084cd3860820f69dda8752 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
W20260202 02:52:48.831321 999 raft_consensus.cc:2686] T d30557fbe9084cd3860820f69dda8752 P b1f05820b6034228bdad884cc348716d: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260202 02:52:48.831569 999 leader_election.cc:343] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 02b434fa058b45529ff3938b053263c6
W20260202 02:52:48.831903 999 leader_election.cc:343] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: fe53a8bdbd13417f9ce195c54a52a93c
I20260202 02:52:48.826640 1175 ts_tablet_manager.cc:1434] T 65bd1c1a382e42e4a68077264563c55b P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.009s user 0.003s sys 0.000s
I20260202 02:52:48.832603 857 leader_election.cc:304] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:48.833030 999 leader_election.cc:304] T 78f122342dd144b48de89902faa0f94f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:48.832227 2666 raft_consensus.cc:2749] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260202 02:52:48.833323 1175 tablet_bootstrap.cc:492] T 75a24a1bf54d4c799938a5723ac6fa99 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:48.833571 2632 raft_consensus.cc:3060] T 78f122342dd144b48de89902faa0f94f P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:48.833592 894 tablet_service.cc:1505] Processing CreateTablet for tablet 14ec41dfdf2648319ba08111c5e7c5e8 (DEFAULT_TABLE table=default.Table_ab48976710ec44e1a6b293a951fab152 [id=d2f8f193d18047a4924a90fc2d4ae1de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.824448 1182 ts_tablet_manager.cc:1916] T d8418a9dd0944456b4d4820dde0cde02 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:48.836275 2632 raft_consensus.cc:2749] T 78f122342dd144b48de89902faa0f94f P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260202 02:52:48.837005 999 leader_election.cc:343] T 26eeef6eebbf4f2cb44f69255fb37594 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 26eeef6eebbf4f2cb44f69255fb37594
I20260202 02:52:48.837288 2666 raft_consensus.cc:2749] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260202 02:52:48.837899 857 leader_election.cc:343] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:48.838372 857 leader_election.cc:304] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:48.838155 999 leader_election.cc:304] T 26eeef6eebbf4f2cb44f69255fb37594 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:48.838891 2632 raft_consensus.cc:2749] T 26eeef6eebbf4f2cb44f69255fb37594 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:48.839690 2666 raft_consensus.cc:2749] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:48.840204 857 leader_election.cc:304] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
W20260202 02:52:48.839478 999 leader_election.cc:343] T 06b80a1c9251420cbc16fc95ae98aca9 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 06b80a1c9251420cbc16fc95ae98aca9
W20260202 02:52:48.840684 857 raft_consensus.cc:2686] T be993c1512b04df4b92cab3195241d84 P 1521a6d262b7451bbef8debe970dd6b5: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260202 02:52:48.840832 999 leader_election.cc:304] T 06b80a1c9251420cbc16fc95ae98aca9 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
W20260202 02:52:48.841764 857 leader_election.cc:343] T 842e54adae2a452a98521de24326eaff P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:48.841958 999 raft_consensus.cc:2686] T 06b80a1c9251420cbc16fc95ae98aca9 P b1f05820b6034228bdad884cc348716d: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260202 02:52:48.842670 999 leader_election.cc:343] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 874ca4ba4ebc4721bb4b60cba8e01841
I20260202 02:52:48.842847 857 leader_election.cc:304] T 842e54adae2a452a98521de24326eaff P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:48.843178 2666 raft_consensus.cc:2749] T 842e54adae2a452a98521de24326eaff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:48.843597 857 leader_election.cc:343] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:48.844007 999 leader_election.cc:343] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 75569511179b41038e521fa3346a92ae
I20260202 02:52:48.844471 2666 consensus_queue.cc:1059] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260202 02:52:48.844844 2443 raft_consensus.cc:3055] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260202 02:52:48.844949 2443 raft_consensus.cc:740] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:48.845191 2443 consensus_queue.cc:260] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:48.845399 857 leader_election.cc:304] T 70306ed9dec54286885d965f2c012da4 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
W20260202 02:52:48.845724 999 leader_election.cc:343] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: fe53a8bdbd13417f9ce195c54a52a93c
I20260202 02:52:48.845501 2443 raft_consensus.cc:3060] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:48.816553 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d06e48923cd4452bae0538c94c527bb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.846071 2666 raft_consensus.cc:2749] T 70306ed9dec54286885d965f2c012da4 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260202 02:52:48.846308 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 252d16b6b37346b2908862b509c4c8cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.846454 999 leader_election.cc:304] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:48.846897 857 leader_election.cc:304] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
W20260202 02:52:48.847182 999 raft_consensus.cc:2686] T fe53a8bdbd13417f9ce195c54a52a93c P b1f05820b6034228bdad884cc348716d: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260202 02:52:48.847788 2666 raft_consensus.cc:493] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:48.847944 2666 raft_consensus.cc:515] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:48.848461 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14ec41dfdf2648319ba08111c5e7c5e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.848532 2666 leader_election.cc:290] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:48.850096 1053 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7709ea34429e4e91ad4922a3d73ebd14" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:48.850149 2666 raft_consensus.cc:2749] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260202 02:52:48.850540 903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 491784bba44c45bdbe3a762dffe85d9e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:48.850651 859 leader_election.cc:343] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:48.851424 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet be993c1512b04df4b92cab3195241d84 (table default.Table_dd517ae94bf54267bc3a03dee4896708 [id=22f1ff6e13a84a72820c887dd3107f49]) successfully deleted
W20260202 02:52:48.851713 999 leader_election.cc:343] T 02b434fa058b45529ff3938b053263c6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 02b434fa058b45529ff3938b053263c6
I20260202 02:52:48.852675 1175 tablet_bootstrap.cc:654] T 75a24a1bf54d4c799938a5723ac6fa99 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:48.853595 999 leader_election.cc:343] T b7880cb051c24bc998003f0effed3cf8 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: b7880cb051c24bc998003f0effed3cf8
I20260202 02:52:48.853830 999 leader_election.cc:304] T b7880cb051c24bc998003f0effed3cf8 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:48.854104 2632 raft_consensus.cc:2749] T b7880cb051c24bc998003f0effed3cf8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:48.854800 999 consensus_peers.cc:597] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 026e6ee0b3ae4d938fdf1550044ed1fa. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:48.855196 999 leader_election.cc:343] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 874ca4ba4ebc4721bb4b60cba8e01841
I20260202 02:52:48.855968 999 leader_election.cc:304] T c050a2882f764dda8df2a7df6547f149 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:48.856259 2632 raft_consensus.cc:2749] T c050a2882f764dda8df2a7df6547f149 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260202 02:52:48.856724 999 leader_election.cc:304] T 622a113af04a41c6852c29fc8a8acd78 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:48.857122 2632 raft_consensus.cc:2749] T 622a113af04a41c6852c29fc8a8acd78 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260202 02:52:48.857367 1182 ts_tablet_manager.cc:1929] T d8418a9dd0944456b4d4820dde0cde02 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:48.857532 999 leader_election.cc:304] T d06b77ff1a6f4f78b73082b00145fcf6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:48.857887 2632 raft_consensus.cc:2749] T d06b77ff1a6f4f78b73082b00145fcf6 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260202 02:52:48.858271 999 leader_election.cc:304] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:48.858536 2632 raft_consensus.cc:2804] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Leader election won for term 2
I20260202 02:52:48.858632 2632 raft_consensus.cc:697] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d [term 2 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:48.858853 2632 consensus_queue.cc:237] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:48.799892 2633 consensus_queue.cc:1059] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.090s
I20260202 02:52:48.866185 765 raft_consensus.cc:2468] T 79c73f4a92044bafb5b8384229cb3a4d P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
I20260202 02:52:48.866495 778 raft_consensus.cc:3060] T 8bb14bf551f441dcbaaff7057b51b2be P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:48.866703 1182 log.cc:1199] T d8418a9dd0944456b4d4820dde0cde02 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/d8418a9dd0944456b4d4820dde0cde02
I20260202 02:52:48.867002 1182 ts_tablet_manager.cc:1950] T d8418a9dd0944456b4d4820dde0cde02 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:48.867383 2740 consensus_queue.cc:1059] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260202 02:52:48.858898 886 tablet_service.cc:1505] Processing CreateTablet for tablet 4552fa1c83424db2b0c7bc390c3bc2b6 (DEFAULT_TABLE table=default.Table_bd3fed1d460846fe81f7c0f86074b2ca [id=7b113a4090c24abe8376093a5b7a353f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:48.860116 2448 consensus_queue.cc:237] T dada209d982d48f3b74a49905f19c5b6 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:48.867803 2482 raft_consensus.cc:3055] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Stepping down as leader of term 1
I20260202 02:52:48.867877 2482 raft_consensus.cc:740] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:48.868196 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4552fa1c83424db2b0c7bc390c3bc2b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:48.868108 2482 consensus_queue.cc:260] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:48.869452 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 723) took 18236 ms. Trace:
I20260202 02:52:48.869621 1182 rpcz_store.cc:276] 0202 02:52:30.632830 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.682739 (+15049909us) service_pool.cc:225] Handling call
0202 02:52:48.819839 (+3137100us) ts_tablet_manager.cc:1151] Deleting tablet d8418a9dd0944456b4d4820dde0cde02
0202 02:52:48.819850 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.854197 (+ 34347us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.857267 (+ 3070us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.869421 (+ 12154us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3136925}
I20260202 02:52:48.870386 778 raft_consensus.cc:2468] T 8bb14bf551f441dcbaaff7057b51b2be P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
I20260202 02:52:48.870792 1182 tablet_replica.cc:333] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:48.871052 1182 raft_consensus.cc:2243] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:48.871363 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet d8418a9dd0944456b4d4820dde0cde02 (table default.Table_2d42dcf81d2d4b2c90fe09860b9b2318 [id=e620930eae8e4707b93e7ab16369e13d]) successfully deleted
I20260202 02:52:48.871652 2482 raft_consensus.cc:3060] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:48.873333 1182 raft_consensus.cc:2272] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:48.873413 764 raft_consensus.cc:1240] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Rejecting Update request from peer 1521a6d262b7451bbef8debe970dd6b5 for earlier term 1. Current term is 2. Ops: []
I20260202 02:52:48.873728 2633 raft_consensus.cc:3055] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d [term 2 LEADER]: Stepping down as leader of term 2
I20260202 02:52:48.873924 1182 ts_tablet_manager.cc:1916] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:48.874081 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8b0368e03f644f1db28dbb57c95c48c1" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:48.875052 999 leader_election.cc:304] T 79c73f4a92044bafb5b8384229cb3a4d P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:48.875973 2740 raft_consensus.cc:2749] T 79c73f4a92044bafb5b8384229cb3a4d P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:48.876591 999 leader_election.cc:304] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:48.877554 2740 raft_consensus.cc:2749] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260202 02:52:48.878618 999 leader_election.cc:304] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:48.880426 2633 raft_consensus.cc:740] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:48.880441 1173 tablet_bootstrap.cc:492] T ccd6bc8bd0504bf18d108c73685ef9ff P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:48.882086 522 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_48307218be9d434798a0d354f3590805 [id=31abb0facd164caaa15591259eb951c2] alter complete (version 2)
I20260202 02:52:48.883893 1173 tablet_bootstrap.cc:654] T ccd6bc8bd0504bf18d108c73685ef9ff P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:48.884615 1180 ts_tablet_manager.cc:1929] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:48.885398 2740 raft_consensus.cc:2749] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:48.886029 999 leader_election.cc:343] T 550895b4010f4088b2cc34acff50e03f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 550895b4010f4088b2cc34acff50e03f
I20260202 02:52:48.886580 999 leader_election.cc:304] T 550895b4010f4088b2cc34acff50e03f P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:48.893494 2740 raft_consensus.cc:2749] T 550895b4010f4088b2cc34acff50e03f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:48.894482 999 leader_election.cc:304] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:48.894840 2632 raft_consensus.cc:3060] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Advancing to term 3
I20260202 02:52:48.897423 2632 raft_consensus.cc:2749] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260202 02:52:48.898984 1182 ts_tablet_manager.cc:1929] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260202 02:52:48.900537 1882 ts_tablet_manager.cc:1929] T d30557fbe9084cd3860820f69dda8752 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:48.900676 1882 log.cc:1199] T d30557fbe9084cd3860820f69dda8752 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/d30557fbe9084cd3860820f69dda8752
I20260202 02:52:48.900724 2633 consensus_queue.cc:260] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:48.901082 1882 ts_tablet_manager.cc:1950] T d30557fbe9084cd3860820f69dda8752 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:48.901495 2633 raft_consensus.cc:3060] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Advancing to term 3
I20260202 02:52:48.901839 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 644) took 19868 ms. Trace:
I20260202 02:52:48.901922 1882 rpcz_store.cc:276] 0202 02:52:29.033241 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.033850 (+ 609us) service_pool.cc:225] Handling call
0202 02:52:48.685295 (+19651445us) ts_tablet_manager.cc:1151] Deleting tablet d30557fbe9084cd3860820f69dda8752
0202 02:52:48.685302 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.690438 (+ 5136us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.900392 (+209954us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.901816 (+ 1424us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":19651319}
I20260202 02:52:48.902272 1882 tablet_replica.cc:333] T 26eeef6eebbf4f2cb44f69255fb37594 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:48.902439 1882 raft_consensus.cc:2243] T 26eeef6eebbf4f2cb44f69255fb37594 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:48.903527 1182 log.cc:1199] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/91ec01d25f8f441cbe3783cacbee51db
I20260202 02:52:48.903900 1173 tablet_bootstrap.cc:492] T ccd6bc8bd0504bf18d108c73685ef9ff P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:48.903936 1182 ts_tablet_manager.cc:1950] T 91ec01d25f8f441cbe3783cacbee51db P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:48.904038 1173 ts_tablet_manager.cc:1403] T ccd6bc8bd0504bf18d108c73685ef9ff P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.000s
I20260202 02:52:48.904798 1173 raft_consensus.cc:359] T ccd6bc8bd0504bf18d108c73685ef9ff P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:48.905128 1173 raft_consensus.cc:385] T ccd6bc8bd0504bf18d108c73685ef9ff P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:48.905230 1173 raft_consensus.cc:740] T ccd6bc8bd0504bf18d108c73685ef9ff P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:48.905283 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet d30557fbe9084cd3860820f69dda8752 (table default.Table_0bf3a06700844105a02b98edd499e75a [id=966fa513d6e947cdb895edc05d26ff65]) successfully deleted
I20260202 02:52:48.905499 1173 consensus_queue.cc:260] T ccd6bc8bd0504bf18d108c73685ef9ff P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:48.905918 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 724) took 18273 ms. Trace:
I20260202 02:52:48.906177 1173 ts_tablet_manager.cc:1434] T ccd6bc8bd0504bf18d108c73685ef9ff P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:48.907068 1173 tablet_bootstrap.cc:492] T d8418a9dd0944456b4d4820dde0cde02 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:48.906520 1182 rpcz_store.cc:276] 0202 02:52:30.632910 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.682931 (+15050021us) service_pool.cc:225] Handling call
0202 02:52:48.870736 (+3187805us) ts_tablet_manager.cc:1151] Deleting tablet 91ec01d25f8f441cbe3783cacbee51db
0202 02:52:48.870749 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.896060 (+ 25311us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.898871 (+ 2811us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.905896 (+ 7025us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3187684}
I20260202 02:52:48.909147 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 91ec01d25f8f441cbe3783cacbee51db (table default.Table_23afb316ffac47aea4db07bda15a956b [id=255f4d94812f45d0b4423150098460ff]) successfully deleted
I20260202 02:52:48.909631 1180 log.cc:1199] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/761f5a51335a4de5977f5f7fef0d4260
I20260202 02:52:48.909868 2720 tablet.cc:1722] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20260202 02:52:48.910616 1173 tablet_bootstrap.cc:654] T d8418a9dd0944456b4d4820dde0cde02 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:48.911003 1182 tablet_replica.cc:333] T fbcb4a522df648ba9542250450c08f1b P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:48.911537 1182 raft_consensus.cc:2243] T fbcb4a522df648ba9542250450c08f1b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:48.911998 1182 raft_consensus.cc:2272] T fbcb4a522df648ba9542250450c08f1b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:48.912812 1182 ts_tablet_manager.cc:1916] T fbcb4a522df648ba9542250450c08f1b P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:48.916182 1180 ts_tablet_manager.cc:1950] T 761f5a51335a4de5977f5f7fef0d4260 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:48.916638 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 545) took 22480 ms. Trace:
I20260202 02:52:48.916729 1180 rpcz_store.cc:276] 0202 02:52:26.436067 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.436852 (+ 785us) service_pool.cc:225] Handling call
0202 02:52:48.636617 (+22199765us) ts_tablet_manager.cc:1151] Deleting tablet 761f5a51335a4de5977f5f7fef0d4260
0202 02:52:48.636629 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.752432 (+115803us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.872531 (+120099us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.916603 (+ 44072us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":22199466}
I20260202 02:52:48.917115 1180 tablet_replica.cc:333] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:48.917289 1180 raft_consensus.cc:2243] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:48.917588 1180 raft_consensus.cc:2272] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:48.918169 1180 ts_tablet_manager.cc:1916] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:48.923390 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7709ea34429e4e91ad4922a3d73ebd14" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:48.924654 2732 tablet.cc:1722] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20260202 02:52:48.924856 857 leader_election.cc:304] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
W20260202 02:52:48.925666 857 leader_election.cc:343] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:48.925765 857 leader_election.cc:304] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:48.925889 1173 tablet_bootstrap.cc:492] T d8418a9dd0944456b4d4820dde0cde02 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:48.926021 1173 ts_tablet_manager.cc:1403] T d8418a9dd0944456b4d4820dde0cde02 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.000s
I20260202 02:52:48.926476 2443 raft_consensus.cc:2749] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260202 02:52:48.926641 2443 raft_consensus.cc:2749] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:48.926739 1173 raft_consensus.cc:359] T d8418a9dd0944456b4d4820dde0cde02 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:48.926945 1173 raft_consensus.cc:385] T d8418a9dd0944456b4d4820dde0cde02 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:48.927037 1173 raft_consensus.cc:740] T d8418a9dd0944456b4d4820dde0cde02 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:48.927302 1173 consensus_queue.cc:260] T d8418a9dd0944456b4d4820dde0cde02 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:48.927467 764 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "939fbcaf75c7439d9208bbbf83a72717" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:48.928081 857 leader_election.cc:343] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:48.928190 857 leader_election.cc:304] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
W20260202 02:52:48.928644 857 leader_election.cc:343] T 939fbcaf75c7439d9208bbbf83a72717 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:48.928731 857 leader_election.cc:304] T 939fbcaf75c7439d9208bbbf83a72717 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:48.929132 2666 raft_consensus.cc:2749] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:48.929426 2666 raft_consensus.cc:2749] T 939fbcaf75c7439d9208bbbf83a72717 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:48.931408 2732 tablet.cc:1722] T c050a2882f764dda8df2a7df6547f149 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:48.931602 765 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "587cfd0ae62f4cfd937faf6890e78f4a" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:48.932029 999 consensus_peers.cc:597] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: be993c1512b04df4b92cab3195241d84. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:48.932449 857 leader_election.cc:343] T 587cfd0ae62f4cfd937faf6890e78f4a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:48.932566 857 leader_election.cc:304] T 587cfd0ae62f4cfd937faf6890e78f4a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:48.932781 2445 raft_consensus.cc:2749] T 587cfd0ae62f4cfd937faf6890e78f4a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:48.934310 1882 raft_consensus.cc:2272] T 26eeef6eebbf4f2cb44f69255fb37594 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:48.934810 1173 ts_tablet_manager.cc:1434] T d8418a9dd0944456b4d4820dde0cde02 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.009s user 0.002s sys 0.001s
I20260202 02:52:48.935170 1173 tablet_bootstrap.cc:492] T 2c31b45dc8004eae993e972e9faa3ff8 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
W20260202 02:52:48.936259 523 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7709ea34429e4e91ad4922a3d73ebd14 consensus state has no leader
I20260202 02:52:48.937459 2732 tablet.cc:1722] T af5aa68e47bb420eac300754d274fa73 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:48.938867 523 catalog_manager.cc:5654] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d reported cstate change: term changed from 1 to 2, leader changed from 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130) to b1f05820b6034228bdad884cc348716d (127.0.104.131). New cstate: current_term: 2 leader_uuid: "b1f05820b6034228bdad884cc348716d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } }
I20260202 02:52:48.939337 523 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 26b890ed87c3471b9fc0b172c90e0a11 (table default.Table_044ef6a2b1af4e3cab3c8cf00505c6a6 [id=d2e4220a0cbf443e8380433da4f064f8]). Expected version 1 got 0
I20260202 02:52:48.940294 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_e9fb673e50b44eecab9fddc7d896148e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:48.941020 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e9fb673e50b44eecab9fddc7d896148e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:48.943574 2738 tablet.cc:1722] T d06b77ff1a6f4f78b73082b00145fcf6 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260202 02:52:48.944039 1173 tablet_bootstrap.cc:654] T 2c31b45dc8004eae993e972e9faa3ff8 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:48.936765 1882 ts_tablet_manager.cc:1916] T 26eeef6eebbf4f2cb44f69255fb37594 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:48.960270 1182 ts_tablet_manager.cc:1929] T fbcb4a522df648ba9542250450c08f1b P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:48.963861 1882 ts_tablet_manager.cc:1929] T 26eeef6eebbf4f2cb44f69255fb37594 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:48.963996 1882 log.cc:1199] T 26eeef6eebbf4f2cb44f69255fb37594 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/26eeef6eebbf4f2cb44f69255fb37594
I20260202 02:52:48.964262 1882 ts_tablet_manager.cc:1950] T 26eeef6eebbf4f2cb44f69255fb37594 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:48.965598 1182 log.cc:1199] T fbcb4a522df648ba9542250450c08f1b P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/fbcb4a522df648ba9542250450c08f1b
I20260202 02:52:48.966153 1182 ts_tablet_manager.cc:1950] T fbcb4a522df648ba9542250450c08f1b P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:48.966881 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 725) took 18333 ms. Trace:
I20260202 02:52:48.968106 1173 tablet_bootstrap.cc:492] T 2c31b45dc8004eae993e972e9faa3ff8 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:48.968010 1182 rpcz_store.cc:276] 0202 02:52:30.633001 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.683059 (+15050058us) service_pool.cc:225] Handling call
0202 02:52:48.910934 (+3227875us) ts_tablet_manager.cc:1151] Deleting tablet fbcb4a522df648ba9542250450c08f1b
0202 02:52:48.910949 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.923200 (+ 12251us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.960146 (+ 36946us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.966860 (+ 6714us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3227757}
I20260202 02:52:48.968295 1173 ts_tablet_manager.cc:1403] T 2c31b45dc8004eae993e972e9faa3ff8 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.033s user 0.003s sys 0.003s
I20260202 02:52:48.969074 1173 raft_consensus.cc:359] T 2c31b45dc8004eae993e972e9faa3ff8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:48.969305 1173 raft_consensus.cc:385] T 2c31b45dc8004eae993e972e9faa3ff8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:48.969389 1173 raft_consensus.cc:740] T 2c31b45dc8004eae993e972e9faa3ff8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:48.969348 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet fbcb4a522df648ba9542250450c08f1b (table default.Table_0cd8f9947fbd430f8992cc1543aed9f7 [id=163eefc3d38a4b28bce88b948cc5df3c]) successfully deleted
I20260202 02:52:48.969659 1173 consensus_queue.cc:260] T 2c31b45dc8004eae993e972e9faa3ff8 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:48.970345 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 726) took 18337 ms. Trace:
I20260202 02:52:48.971927 1182 rpcz_store.cc:276] 0202 02:52:30.633089 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.683181 (+15050092us) service_pool.cc:225] Handling call
0202 02:52:48.970205 (+3287024us) ts_tablet_manager.cc:1151] Deleting tablet f1fadf2cb5be4ec8980871fc96ee9730
0202 02:52:48.970218 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.970327 (+ 109us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3286919}
W20260202 02:52:48.973232 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet f1fadf2cb5be4ec8980871fc96ee9730 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f1fadf2cb5be4ec8980871fc96ee9730 already in progress: creating tablet
I20260202 02:52:48.973846 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 722) took 18341 ms. Trace:
I20260202 02:52:48.973964 1173 ts_tablet_manager.cc:1434] T 2c31b45dc8004eae993e972e9faa3ff8 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20260202 02:52:48.974221 1182 rpcz_store.cc:276] 0202 02:52:30.632705 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.681622 (+15048917us) service_pool.cc:225] Handling call
0202 02:52:48.973030 (+3291408us) ts_tablet_manager.cc:1151] Deleting tablet 91a604fb877a48479914a5b60144a22d
0202 02:52:48.973041 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.973825 (+ 784us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3225011}
I20260202 02:52:48.975251 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 657) took 19707 ms. Trace:
I20260202 02:52:48.975540 1173 tablet_bootstrap.cc:492] T 474df41e4fe24a818fa45a4ce84d5474 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
W20260202 02:52:48.975790 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 91a604fb877a48479914a5b60144a22d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 91a604fb877a48479914a5b60144a22d already in progress: creating tablet
I20260202 02:52:48.976238 1182 tablet_replica.cc:333] T ac2ab12f5040488aa306cb69b107b485 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:48.976707 1182 raft_consensus.cc:2243] T ac2ab12f5040488aa306cb69b107b485 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:48.977054 1173 tablet_bootstrap.cc:654] T 474df41e4fe24a818fa45a4ce84d5474 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:48.977154 1182 raft_consensus.cc:2272] T ac2ab12f5040488aa306cb69b107b485 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:48.977717 1182 ts_tablet_manager.cc:1916] T ac2ab12f5040488aa306cb69b107b485 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:48.975353 1882 rpcz_store.cc:276] 0202 02:52:29.268236 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.268375 (+ 139us) service_pool.cc:225] Handling call
0202 02:52:48.902225 (+19633850us) ts_tablet_manager.cc:1151] Deleting tablet 26eeef6eebbf4f2cb44f69255fb37594
0202 02:52:48.902235 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.960728 (+ 58493us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.963746 (+ 3018us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.975230 (+ 11484us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":19633639}
I20260202 02:52:48.979310 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 659) took 19709 ms. Trace:
I20260202 02:52:48.979398 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 26eeef6eebbf4f2cb44f69255fb37594 (table default.Table_684386396f2a4be68080349404089bf8 [id=7fbb33c105f149ef98505dcf622930c9]) successfully deleted
I20260202 02:52:48.979452 1882 rpcz_store.cc:276] 0202 02:52:29.269429 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.271671 (+ 2242us) service_pool.cc:225] Handling call
0202 02:52:48.979200 (+19707529us) ts_tablet_manager.cc:1151] Deleting tablet 042d1ebf357b4abe8814a407891ce265
0202 02:52:48.979211 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.979292 (+ 81us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":19707349}
I20260202 02:52:48.979773 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 658) took 19710 ms. Trace:
I20260202 02:52:48.979853 1882 rpcz_store.cc:276] 0202 02:52:29.268981 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.271887 (+ 2906us) service_pool.cc:225] Handling call
0202 02:52:48.979689 (+19707802us) ts_tablet_manager.cc:1151] Deleting tablet 1f54c1dfe0a942b3bafd85522953a032
0202 02:52:48.979698 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.979759 (+ 61us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":19707677}
W20260202 02:52:48.980020 515 catalog_manager.cc:4952] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 042d1ebf357b4abe8814a407891ce265 because the tablet was not found. No further retry: Not found: Tablet not found: 042d1ebf357b4abe8814a407891ce265
I20260202 02:52:48.980183 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 661) took 19676 ms. Trace:
I20260202 02:52:48.980268 1882 rpcz_store.cc:276] 0202 02:52:29.303873 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.305745 (+ 1872us) service_pool.cc:225] Handling call
0202 02:52:48.980058 (+19674313us) ts_tablet_manager.cc:1151] Deleting tablet 842197c689064660b791b40e521d346b
0202 02:52:48.980069 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.980164 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":19674102}
W20260202 02:52:48.980583 515 catalog_manager.cc:4952] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 1f54c1dfe0a942b3bafd85522953a032 because the tablet was not found. No further retry: Not found: Tablet not found: 1f54c1dfe0a942b3bafd85522953a032
I20260202 02:52:48.980634 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 660) took 19680 ms. Trace:
I20260202 02:52:48.980695 1882 rpcz_store.cc:276] 0202 02:52:29.300021 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.306091 (+ 6070us) service_pool.cc:225] Handling call
0202 02:52:48.980550 (+19674459us) ts_tablet_manager.cc:1151] Deleting tablet 7665dbea16d14f619af57ea056639eef
0202 02:52:48.980557 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.980621 (+ 64us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":19674324}
I20260202 02:52:48.980983 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 662) took 19657 ms. Trace:
I20260202 02:52:48.981047 1882 rpcz_store.cc:276] 0202 02:52:29.323207 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.323603 (+ 396us) service_pool.cc:225] Handling call
0202 02:52:48.980900 (+19657297us) ts_tablet_manager.cc:1151] Deleting tablet e7df9a78735547cbbaa006c604b95fec
0202 02:52:48.980908 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.980971 (+ 63us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":19656963}
W20260202 02:52:48.981110 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 842197c689064660b791b40e521d346b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 842197c689064660b791b40e521d346b already in progress: creating tablet
I20260202 02:52:48.981300 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 663) took 19633 ms. Trace:
W20260202 02:52:48.981312 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 7665dbea16d14f619af57ea056639eef because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7665dbea16d14f619af57ea056639eef already in progress: creating tablet
I20260202 02:52:48.981354 1882 rpcz_store.cc:276] 0202 02:52:29.348100 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.348159 (+ 59us) service_pool.cc:225] Handling call
0202 02:52:48.981216 (+19633057us) ts_tablet_manager.cc:1151] Deleting tablet fc3bfd542b8e41d98586645358061261
0202 02:52:48.981223 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:48.981286 (+ 63us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":19632543}
I20260202 02:52:48.981560 1882 tablet_replica.cc:333] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:48.981734 1882 raft_consensus.cc:2243] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
W20260202 02:52:48.981828 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet e7df9a78735547cbbaa006c604b95fec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e7df9a78735547cbbaa006c604b95fec already in progress: creating tablet
I20260202 02:52:48.981987 1882 pending_rounds.cc:70] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d: Trying to abort 1 pending ops.
W20260202 02:52:48.982016 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet fc3bfd542b8e41d98586645358061261 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fc3bfd542b8e41d98586645358061261 already in progress: creating tablet
I20260202 02:52:48.982030 1882 pending_rounds.cc:77] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923146549067776 op_type: NO_OP noop_request { }
I20260202 02:52:48.982115 1882 raft_consensus.cc:2889] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:48.982194 1882 raft_consensus.cc:2272] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:48.982407 1110 heartbeater.cc:344] Connected to a master server at 127.0.104.190:33981
I20260202 02:52:48.988638 1882 ts_tablet_manager.cc:1916] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:48.991770 1173 tablet_bootstrap.cc:492] T 474df41e4fe24a818fa45a4ce84d5474 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:48.992102 1173 ts_tablet_manager.cc:1403] T 474df41e4fe24a818fa45a4ce84d5474 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.000s
I20260202 02:52:48.992973 1173 raft_consensus.cc:359] T 474df41e4fe24a818fa45a4ce84d5474 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:48.993245 1173 raft_consensus.cc:385] T 474df41e4fe24a818fa45a4ce84d5474 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:48.993335 1173 raft_consensus.cc:740] T 474df41e4fe24a818fa45a4ce84d5474 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:48.993587 1173 consensus_queue.cc:260] T 474df41e4fe24a818fa45a4ce84d5474 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:48.994256 1173 ts_tablet_manager.cc:1434] T 474df41e4fe24a818fa45a4ce84d5474 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:48.994658 1173 tablet_bootstrap.cc:492] T f94f25f4059a43b8babedb6b45a08a19 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:49.001194 1173 tablet_bootstrap.cc:654] T f94f25f4059a43b8babedb6b45a08a19 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:49.007829 712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260202 02:52:49.008212 1182 ts_tablet_manager.cc:1929] T ac2ab12f5040488aa306cb69b107b485 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:49.014014 1173 tablet_bootstrap.cc:492] T f94f25f4059a43b8babedb6b45a08a19 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:49.014289 1173 ts_tablet_manager.cc:1403] T f94f25f4059a43b8babedb6b45a08a19 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.003s
I20260202 02:52:49.015051 1173 raft_consensus.cc:359] T f94f25f4059a43b8babedb6b45a08a19 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.015308 1173 raft_consensus.cc:385] T f94f25f4059a43b8babedb6b45a08a19 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:49.015410 1173 raft_consensus.cc:740] T f94f25f4059a43b8babedb6b45a08a19 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:49.015697 1173 consensus_queue.cc:260] T f94f25f4059a43b8babedb6b45a08a19 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.017683 1173 ts_tablet_manager.cc:1434] T f94f25f4059a43b8babedb6b45a08a19 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260202 02:52:49.018044 1173 tablet_bootstrap.cc:492] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:49.020089 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_6eea808cf1ea46998a599819ee2cbd5c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:49.020841 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6eea808cf1ea46998a599819ee2cbd5c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:49.022478 526 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_58d1c16bc0404476afd27abf2f3ce9da" } new_table_name: "default.Table_c45137e043394d51b6c93faf57780d7b" modify_external_catalogs: true
I20260202 02:52:49.022610 520 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 7ac33134670747ee860b6d39dd7db547
I20260202 02:52:49.023345 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_95f6cd1339444962a8731fb7629fce47"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:49.023878 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_95f6cd1339444962a8731fb7629fce47 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:49.024360 525 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 8a206159afa84ae19f8c7763a2fe43d4
I20260202 02:52:49.025115 521 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 9d0212e64d934968bdbdc627c69874b1
I20260202 02:52:49.025031 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_4e00e3aee35945da8f96cd16d666323a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:49.025537 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4e00e3aee35945da8f96cd16d666323a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:49.025655 521 master_service.cc:954] ReplaceTablet: received request to replace tablet 26b890ed87c3471b9fc0b172c90e0a11 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:49.026826 527 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet d1342810762e41588ea00ea02acad3e2
I20260202 02:52:49.027460 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_c826a60439364f9eb3cbcd4a5cd99527" } new_table_name: "default.Table_c74fb92d1efb4b2f8804922d51916f0b" modify_external_catalogs: true
I20260202 02:52:49.028013 1182 log.cc:1199] T ac2ab12f5040488aa306cb69b107b485 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/ac2ab12f5040488aa306cb69b107b485
I20260202 02:52:49.028647 1182 ts_tablet_manager.cc:1950] T ac2ab12f5040488aa306cb69b107b485 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:49.028614 2482 raft_consensus.cc:493] T 2c31b45dc8004eae993e972e9faa3ff8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:49.028779 2482 raft_consensus.cc:515] T 2c31b45dc8004eae993e972e9faa3ff8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:49.029134 1182 tablet_replica.cc:333] T d0ce2121d65b4a228596e0ca9018883d P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:49.029304 1182 raft_consensus.cc:2243] T d0ce2121d65b4a228596e0ca9018883d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:49.029343 2482 leader_election.cc:290] T 2c31b45dc8004eae993e972e9faa3ff8 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:49.030426 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2c31b45dc8004eae993e972e9faa3ff8" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:49.030809 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet 7ac33134670747ee860b6d39dd7db547 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:48 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:49.030864 1182 raft_consensus.cc:2272] T d0ce2121d65b4a228596e0ca9018883d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:49.031009 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 8a206159afa84ae19f8c7763a2fe43d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:48 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:49.031168 1035 tablet_service.cc:1552] Processing DeleteTablet for tablet 9d0212e64d934968bdbdc627c69874b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:48 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:49.031302 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet d1342810762e41588ea00ea02acad3e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:48 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:49.032150 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet ac2ab12f5040488aa306cb69b107b485 (table default.Table_6cf38c67711747acb1b5be35632b7a68 [id=640ad152bb4243cda5a7e7242996e2fb]) successfully deleted
W20260202 02:52:49.032205 998 leader_election.cc:343] T 2c31b45dc8004eae993e972e9faa3ff8 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:49.010281 1175 tablet_bootstrap.cc:492] T 75a24a1bf54d4c799938a5723ac6fa99 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:49.031874 1182 ts_tablet_manager.cc:1916] T d0ce2121d65b4a228596e0ca9018883d P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:49.032825 999 leader_election.cc:343] T 2c31b45dc8004eae993e972e9faa3ff8 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 2c31b45dc8004eae993e972e9faa3ff8
I20260202 02:52:49.032631 893 tablet_service.cc:1552] Processing DeleteTablet for tablet 7ac33134670747ee860b6d39dd7db547 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:48 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:49.032935 999 leader_election.cc:304] T 2c31b45dc8004eae993e972e9faa3ff8 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:49.032827 1175 ts_tablet_manager.cc:1403] T 75a24a1bf54d4c799938a5723ac6fa99 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.200s user 0.006s sys 0.005s
I20260202 02:52:49.033404 2482 raft_consensus.cc:2749] T 2c31b45dc8004eae993e972e9faa3ff8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:49.033713 903 tablet_service.cc:1552] Processing DeleteTablet for tablet 8a206159afa84ae19f8c7763a2fe43d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:48 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:49.034003 895 tablet_service.cc:1552] Processing DeleteTablet for tablet 9d0212e64d934968bdbdc627c69874b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:48 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:49.033689 1175 raft_consensus.cc:359] T 75a24a1bf54d4c799938a5723ac6fa99 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:49.034374 1175 raft_consensus.cc:385] T 75a24a1bf54d4c799938a5723ac6fa99 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:49.034475 1175 raft_consensus.cc:740] T 75a24a1bf54d4c799938a5723ac6fa99 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:49.034734 1175 consensus_queue.cc:260] T 75a24a1bf54d4c799938a5723ac6fa99 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
W20260202 02:52:49.039196 524 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b34dc1cab5874fa6aee55d79c6e09ade consensus state has no leader
I20260202 02:52:49.039661 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_a0f2c4c878d9452a8de3ba0040b72195"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:49.040299 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a0f2c4c878d9452a8de3ba0040b72195 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:49.041033 886 tablet_service.cc:1552] Processing DeleteTablet for tablet d1342810762e41588ea00ea02acad3e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:48 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:49.042002 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_cf40ea51a80d469e9cacf1e520c28248"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:49.042574 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cf40ea51a80d469e9cacf1e520c28248 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:49.042975 1882 ts_tablet_manager.cc:1929] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:49.043135 1882 log.cc:1199] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/75569511179b41038e521fa3346a92ae
I20260202 02:52:49.043519 1882 ts_tablet_manager.cc:1950] T 75569511179b41038e521fa3346a92ae P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:49.044538 1175 ts_tablet_manager.cc:1434] T 75a24a1bf54d4c799938a5723ac6fa99 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20260202 02:52:49.045980 1173 tablet_bootstrap.cc:654] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:49.051385 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_edb640fc9a204021b59ae38f17059e3b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:49.051981 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_edb640fc9a204021b59ae38f17059e3b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:49.052131 520 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_f4a5e710b7a444bbbddef402779ebc8d" } new_table_name: "default.Table_c57473bfb15e41da8627401aafd82bce" modify_external_catalogs: true
W20260202 02:52:49.053339 526 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a44aa53c6b98468a8e36478f2fc9cdda consensus state has no leader
I20260202 02:52:49.054013 525 master_service.cc:954] ReplaceTablet: received request to replace tablet d0ce2121d65b4a228596e0ca9018883d from {username='slave'} at 127.0.0.1:47430
W20260202 02:52:49.054617 527 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5ec7d5c6cf834f8190437948f5ff2b8e consensus state has no leader
I20260202 02:52:49.056515 526 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_911bb209b4554ee8b9bc09edaf488d63" } modify_external_catalogs: true
I20260202 02:52:49.056710 526 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_911bb209b4554ee8b9bc09edaf488d63" } modify_external_catalogs: true
I20260202 02:52:49.061518 1175 tablet_bootstrap.cc:492] T da5435f969074364bc8f8a6ddc5e4f8a P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:49.063793 522 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_150ccd742b6f4b208f42e6d0876494e5 [id=915b211258ad498286448a5b2598a289] alter complete (version 4)
I20260202 02:52:49.064021 523 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_5e9068e8d78849919824ab9796e02397 [id=feb8b9f926434cec8b119843413f8c9b] alter complete (version 2)
I20260202 02:52:49.064810 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 587cfd0ae62f4cfd937faf6890e78f4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:49.049823 2482 raft_consensus.cc:493] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:49.066241 2482 raft_consensus.cc:515] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.066969 2482 leader_election.cc:290] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:49.067829 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1116872cc25a4ab5b1a3a6bbba1f0da7" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:49.068403 999 leader_election.cc:343] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 1116872cc25a4ab5b1a3a6bbba1f0da7
W20260202 02:52:49.068805 998 leader_election.cc:343] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:49.068921 998 leader_election.cc:304] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:49.069437 2482 raft_consensus.cc:2749] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:49.070668 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 665) took 19562 ms. Trace:
I20260202 02:52:49.071751 1882 rpcz_store.cc:276] 0202 02:52:29.508530 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.510681 (+ 2151us) service_pool.cc:225] Handling call
0202 02:52:48.981517 (+19470836us) ts_tablet_manager.cc:1151] Deleting tablet 75569511179b41038e521fa3346a92ae
0202 02:52:48.981525 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.005223 (+ 23698us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.025857 (+ 20634us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.070648 (+ 44791us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":19470607}
I20260202 02:52:49.072398 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 75569511179b41038e521fa3346a92ae (table default.Table_6da292362a6c43738dbeb895750b918a [id=6d50800a3c8642c0992dd1a7f38cfd30]) successfully deleted
I20260202 02:52:49.072911 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 664) took 19564 ms. Trace:
I20260202 02:52:49.073534 1175 tablet_bootstrap.cc:654] T da5435f969074364bc8f8a6ddc5e4f8a P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:49.073648 521 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 26b890ed87c3471b9fc0b172c90e0a11
I20260202 02:52:49.073654 1882 rpcz_store.cc:276] 0202 02:52:29.508380 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.511805 (+ 3425us) service_pool.cc:225] Handling call
0202 02:52:49.072761 (+19560956us) ts_tablet_manager.cc:1151] Deleting tablet ef073880898049e2be93d2cb6dc0c0bb
0202 02:52:49.072777 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.072892 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":19560635}
I20260202 02:52:49.074168 521 catalog_manager.cc:6724] ReplaceTablet: tablet 26b890ed87c3471b9fc0b172c90e0a11 deleted and replaced by tablet e5ab9384afe14e4a875be18ffdb863c3
I20260202 02:52:49.074178 1882 tablet_replica.cc:333] T 3d1a7578e83447669e3c3972a426ad44 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:49.074360 1882 raft_consensus.cc:2243] T 3d1a7578e83447669e3c3972a426ad44 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:49.074633 1882 raft_consensus.cc:2272] T 3d1a7578e83447669e3c3972a426ad44 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:49.075197 1882 ts_tablet_manager.cc:1916] T 3d1a7578e83447669e3c3972a426ad44 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:49.075457 515 catalog_manager.cc:4952] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet ef073880898049e2be93d2cb6dc0c0bb because the tablet was not found. No further retry: Not found: Tablet not found: ef073880898049e2be93d2cb6dc0c0bb
I20260202 02:52:49.076193 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet 26b890ed87c3471b9fc0b172c90e0a11 with delete_type TABLET_DATA_DELETED (replaced by tablet e5ab9384afe14e4a875be18ffdb863c3) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:49.076364 2720 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:42950 (request call id 733) took 1025 ms. Trace:
I20260202 02:52:49.078759 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 26b890ed87c3471b9fc0b172c90e0a11 with delete_type TABLET_DATA_DELETED (replaced by tablet e5ab9384afe14e4a875be18ffdb863c3) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:49.079622 520 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet de76ca9a98c840d0967a97154b940b60 consensus state has no leader
I20260202 02:52:49.080766 526 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 28a900bcaa8b41718a285e595886d805
I20260202 02:52:49.081758 1182 ts_tablet_manager.cc:1929] T d0ce2121d65b4a228596e0ca9018883d P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:49.081964 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet 28a900bcaa8b41718a285e595886d805 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:49.082141 1173 tablet_bootstrap.cc:492] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:49.082178 1182 log.cc:1199] T d0ce2121d65b4a228596e0ca9018883d P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/d0ce2121d65b4a228596e0ca9018883d
I20260202 02:52:49.082594 1173 ts_tablet_manager.cc:1403] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.065s user 0.002s sys 0.002s
I20260202 02:52:49.076483 2720 rpcz_store.cc:276] 0202 02:52:47.941669 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:47.945592 (+ 3923us) service_pool.cc:225] Handling call
0202 02:52:48.967033 (+1021441us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:47.981179 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0202 02:52:47.981301 (+ 122us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0202 02:52:47.981309 (+ 8us) alter_schema_op.cc:66] Acquired schema lock
0202 02:52:47.981317 (+ 8us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0202 02:52:47.981356 (+ 39us) alter_schema_op.cc:128] START. Timestamp: 1770000767981290
0202 02:52:47.981359 (+ 3us) op_driver.cc:348] REPLICATION: starting
0202 02:52:47.981559 (+ 200us) log.cc:844] Serialized 173 byte log entry
0202 02:52:48.767053 (+785494us) op_driver.cc:464] REPLICATION: finished
0202 02:52:48.881929 (+114876us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0202 02:52:48.910304 (+ 28375us) log.cc:844] Serialized 5 byte log entry
0202 02:52:48.966632 (+ 56328us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:48.966947 (+ 315us) log.cc:844] Serialized 14 byte log entry
0202 02:52:48.966987 (+ 40us) alter_schema_op.cc:73] Released schema lock
0202 02:52:48.966992 (+ 5us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":114826,"prepare.queue_time_us":35294,"prepare.run_cpu_time_us":640,"prepare.run_wall_time_us":4211,"replication_time_us":785659,"thread_start_us":268380,"threads_started":3,"wal-append.queue_time_us":3235}]]}
I20260202 02:52:49.084424 1173 raft_consensus.cc:359] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.084707 1173 raft_consensus.cc:385] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:49.084826 1173 raft_consensus.cc:740] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:49.085131 1173 consensus_queue.cc:260] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.085989 523 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_4e4d461d49574c68a7d93ac891db67c6 [id=316b8a276d9b415fafe8511aa7c32fc5] alter complete (version 4)
I20260202 02:52:49.082562 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 28a900bcaa8b41718a285e595886d805 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:49.086781 1180 ts_tablet_manager.cc:1929] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:49.086923 1180 log.cc:1199] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/d0671586e45e45ffb7fd9e1be6e2bbf8
I20260202 02:52:49.087212 1182 ts_tablet_manager.cc:1950] T d0ce2121d65b4a228596e0ca9018883d P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:49.087275 1180 ts_tablet_manager.cc:1950] T d0671586e45e45ffb7fd9e1be6e2bbf8 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
W20260202 02:52:49.087608 1180 rpcz_store.cc:267] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 546) took 22647 ms (client timeout 29999 ms). Trace:
I20260202 02:52:49.087898 1173 ts_tablet_manager.cc:1434] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260202 02:52:49.088078 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet d0ce2121d65b4a228596e0ca9018883d (table default.Table_bde65e061a274734a7df7a428bd8ce53 [id=ca632e6d14ca41e2ab9444dc362a4265]) successfully deleted
W20260202 02:52:49.092201 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 3dcf4f794ec14b2f9d181d015fc437fb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3dcf4f794ec14b2f9d181d015fc437fb already in progress: creating tablet
I20260202 02:52:49.092489 1182 tablet_replica.cc:333] T 75a24a1bf54d4c799938a5723ac6fa99 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
W20260202 02:52:49.092543 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 09a8aa4dd70b4fe2806293d2620e521c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 09a8aa4dd70b4fe2806293d2620e521c already in progress: creating tablet
I20260202 02:52:49.092865 1182 raft_consensus.cc:2243] T 75a24a1bf54d4c799938a5723ac6fa99 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260202 02:52:49.093766 1180 rpcz_store.cc:269] 0202 02:52:26.439932 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.442560 (+ 2628us) service_pool.cc:225] Handling call
0202 02:52:48.917049 (+22474489us) ts_tablet_manager.cc:1151] Deleting tablet d0671586e45e45ffb7fd9e1be6e2bbf8
0202 02:52:48.917058 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.078906 (+161848us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.086641 (+ 7735us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.087572 (+ 931us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":22474026}
W20260202 02:52:49.094336 1180 rpcz_store.cc:267] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 544) took 22658 ms (client timeout 29999 ms). Trace:
W20260202 02:52:49.094470 1180 rpcz_store.cc:269] 0202 02:52:26.435909 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.442634 (+ 6725us) service_pool.cc:225] Handling call
0202 02:52:49.094178 (+22651544us) ts_tablet_manager.cc:1151] Deleting tablet 2f664f6119ba4d28b5ef1c212e611866
0202 02:52:49.094190 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.094304 (+ 114us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":22648464}
W20260202 02:52:49.094869 1180 rpcz_store.cc:267] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 548) took 22571 ms (client timeout 29999 ms). Trace:
W20260202 02:52:49.095019 1180 rpcz_store.cc:269] 0202 02:52:26.523538 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.523595 (+ 57us) service_pool.cc:225] Handling call
0202 02:52:49.094740 (+22571145us) ts_tablet_manager.cc:1151] Deleting tablet dc90eeb79ded4739883ad98928476680
0202 02:52:49.094748 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.094842 (+ 94us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":22570897}
I20260202 02:52:49.095352 1173 tablet_bootstrap.cc:492] T fbcb4a522df648ba9542250450c08f1b P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
W20260202 02:52:49.095405 1180 rpcz_store.cc:267] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 549) took 22562 ms (client timeout 29999 ms). Trace:
W20260202 02:52:49.095505 1180 rpcz_store.cc:269] 0202 02:52:26.532964 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.533147 (+ 183us) service_pool.cc:225] Handling call
0202 02:52:49.095286 (+22562139us) ts_tablet_manager.cc:1151] Deleting tablet 0a5bbb994cfa40f0add3163816b65269
0202 02:52:49.095293 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.095368 (+ 75us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":22561774}
W20260202 02:52:49.095790 1180 rpcz_store.cc:267] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 551) took 22549 ms (client timeout 29999 ms). Trace:
W20260202 02:52:49.095886 1180 rpcz_store.cc:269] 0202 02:52:26.546754 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.547074 (+ 320us) service_pool.cc:225] Handling call
0202 02:52:49.095688 (+22548614us) ts_tablet_manager.cc:1151] Deleting tablet 8ba28ab7d7ae44059006179067dff3c0
0202 02:52:49.095696 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.095764 (+ 68us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":22548330}
W20260202 02:52:49.096165 1180 rpcz_store.cc:267] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 547) took 22575 ms (client timeout 29999 ms). Trace:
W20260202 02:52:49.096237 1180 rpcz_store.cc:269] 0202 02:52:26.520456 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.550921 (+ 30465us) service_pool.cc:225] Handling call
0202 02:52:49.096065 (+22545144us) ts_tablet_manager.cc:1151] Deleting tablet 23671408fcbc454cbd918328015cf6e6
0202 02:52:49.096073 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.096137 (+ 64us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":22543510}
I20260202 02:52:49.096491 1180 tablet_replica.cc:333] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:49.096684 1180 raft_consensus.cc:2243] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:49.096993 1180 raft_consensus.cc:2272] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:49.097613 1180 ts_tablet_manager.cc:1916] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:49.097892 714 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 308656us
I20260202 02:52:49.098526 741 tablet_service.cc:1505] Processing CreateTablet for tablet d06e48923cd4452bae0538c94c527bb2 (DEFAULT_TABLE table=default.Table_b1e92e2bd13d4f12906c13ff7e91f109 [id=a1653b0201104a06afd16cd2ae2fb0b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.098618 753 tablet_service.cc:1505] Processing CreateTablet for tablet 4552fa1c83424db2b0c7bc390c3bc2b6 (DEFAULT_TABLE table=default.Table_bd3fed1d460846fe81f7c0f86074b2ca [id=7b113a4090c24abe8376093a5b7a353f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.098937 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d06e48923cd4452bae0538c94c527bb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.099126 757 tablet_service.cc:1505] Processing CreateTablet for tablet 14ec41dfdf2648319ba08111c5e7c5e8 (DEFAULT_TABLE table=default.Table_ab48976710ec44e1a6b293a951fab152 [id=d2f8f193d18047a4924a90fc2d4ae1de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.099431 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14ec41dfdf2648319ba08111c5e7c5e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.099922 1174 tablet_bootstrap.cc:492] T b7880cb051c24bc998003f0effed3cf8 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:49.100062 1174 ts_tablet_manager.cc:1403] T b7880cb051c24bc998003f0effed3cf8 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.466s user 0.005s sys 0.017s
I20260202 02:52:49.100950 1174 raft_consensus.cc:359] T b7880cb051c24bc998003f0effed3cf8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.101125 1174 raft_consensus.cc:385] T b7880cb051c24bc998003f0effed3cf8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:49.101202 1174 raft_consensus.cc:740] T b7880cb051c24bc998003f0effed3cf8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:49.102515 750 tablet_service.cc:1505] Processing CreateTablet for tablet 491784bba44c45bdbe3a762dffe85d9e (DEFAULT_TABLE table=default.Table_d5520a9602ad4f69bd95b3eeef903589 [id=1cf5b0b923754af08ba5b1eb9cbe908d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.102757 758 tablet_service.cc:1552] Processing DeleteTablet for tablet 7ac33134670747ee860b6d39dd7db547 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:48 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:49.103008 744 tablet_service.cc:1505] Processing CreateTablet for tablet 252d16b6b37346b2908862b509c4c8cc (DEFAULT_TABLE table=default.Table_22e604b386714cefbe344e7f6d6c3d6e [id=fa167d2a101641ffbb21593606f5a711]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.104049 742 tablet_service.cc:1552] Processing DeleteTablet for tablet d1342810762e41588ea00ea02acad3e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:48 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:49.104256 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 9d0212e64d934968bdbdc627c69874b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:48 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:49.104414 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 8a206159afa84ae19f8c7763a2fe43d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:48 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:49.105345 758 tablet_service.cc:1552] Processing DeleteTablet for tablet 26b890ed87c3471b9fc0b172c90e0a11 with delete_type TABLET_DATA_DELETED (replaced by tablet e5ab9384afe14e4a875be18ffdb863c3) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:49.105619 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4552fa1c83424db2b0c7bc390c3bc2b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.108125 524 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_a77859a40f12489a8578b7eb1a66f0d4 [id=fa95e04db2d443b5972de98c7c0e6457] alter complete (version 5)
I20260202 02:52:49.110080 1182 raft_consensus.cc:2272] T 75a24a1bf54d4c799938a5723ac6fa99 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:49.110424 758 tablet_service.cc:1552] Processing DeleteTablet for tablet 28a900bcaa8b41718a285e595886d805 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:49.112493 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 491784bba44c45bdbe3a762dffe85d9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.113080 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 252d16b6b37346b2908862b509c4c8cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.116724 523 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_874748aec6a14c11b3787312a348ef91 [id=5f8923fc0fb84598a1fb7c3a8abbb8cb] alter complete (version 2)
I20260202 02:52:49.118064 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet 1116872cc25a4ab5b1a3a6bbba1f0da7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:25 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:49.118283 2720 tablet.cc:1722] T 80eb3013f5354022a2dd78bb294a8497 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260202 02:52:49.119040 1173 tablet_bootstrap.cc:654] T fbcb4a522df648ba9542250450c08f1b P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:49.127699 1174 consensus_queue.cc:260] T b7880cb051c24bc998003f0effed3cf8 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.128803 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 65bd1c1a382e42e4a68077264563c55b with delete_type TABLET_DATA_DELETED (replaced by tablet 3706562a22634eb6b72e36bb6a2d1ae5) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:49.130359 1882 ts_tablet_manager.cc:1929] T 3d1a7578e83447669e3c3972a426ad44 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:49.130558 1882 log.cc:1199] T 3d1a7578e83447669e3c3972a426ad44 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/3d1a7578e83447669e3c3972a426ad44
I20260202 02:52:49.131072 1882 ts_tablet_manager.cc:1950] T 3d1a7578e83447669e3c3972a426ad44 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:49.131264 1182 ts_tablet_manager.cc:1916] T 75a24a1bf54d4c799938a5723ac6fa99 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:49.131954 1175 tablet_bootstrap.cc:492] T da5435f969074364bc8f8a6ddc5e4f8a P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:49.132005 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 673) took 19497 ms. Trace:
I20260202 02:52:49.132194 1882 rpcz_store.cc:276] 0202 02:52:29.634858 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.636095 (+ 1237us) service_pool.cc:225] Handling call
0202 02:52:49.074126 (+19438031us) ts_tablet_manager.cc:1151] Deleting tablet 3d1a7578e83447669e3c3972a426ad44
0202 02:52:49.074137 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.107328 (+ 33191us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.130157 (+ 22829us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.131982 (+ 1825us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":19437851}
I20260202 02:52:49.132722 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 674) took 19449 ms. Trace:
I20260202 02:52:49.132860 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 3d1a7578e83447669e3c3972a426ad44 (table default.Table_7b5f2de229cb4b2f8bd174ccfd6b9553 [id=97a0437f66864a7291b966c9b72c0eb7]) successfully deleted
I20260202 02:52:49.133952 1882 rpcz_store.cc:276] 0202 02:52:29.682809 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.682861 (+ 52us) service_pool.cc:225] Handling call
0202 02:52:49.132593 (+19449732us) ts_tablet_manager.cc:1151] Deleting tablet 67495e8e525542fb976f675b71512ed2
0202 02:52:49.132604 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.132707 (+ 103us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":19449532}
I20260202 02:52:49.134639 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 675) took 19409 ms. Trace:
W20260202 02:52:49.134778 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 67495e8e525542fb976f675b71512ed2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 67495e8e525542fb976f675b71512ed2 already in progress: creating tablet
I20260202 02:52:49.135197 1882 rpcz_store.cc:276] 0202 02:52:29.725184 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.725233 (+ 49us) service_pool.cc:225] Handling call
0202 02:52:49.134514 (+19409281us) ts_tablet_manager.cc:1151] Deleting tablet 8dd62de8e8234d6c9ce380b1aadc097d
0202 02:52:49.134525 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.134623 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":19404248}
I20260202 02:52:49.136034 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 676) took 19401 ms. Trace:
I20260202 02:52:49.136039 714 leader_election.cc:304] T 25765fb687294e9db1289f76530dd877 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
W20260202 02:52:49.136125 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 8dd62de8e8234d6c9ce380b1aadc097d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8dd62de8e8234d6c9ce380b1aadc097d already in progress: creating tablet
I20260202 02:52:49.136560 1882 rpcz_store.cc:276] 0202 02:52:29.734593 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.734646 (+ 53us) service_pool.cc:225] Handling call
0202 02:52:49.135910 (+19401264us) ts_tablet_manager.cc:1151] Deleting tablet 1d98c7a46139413b8f0d3697b74b1e52
0202 02:52:49.135920 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.136017 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":19401062}
I20260202 02:52:49.137156 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 677) took 19388 ms. Trace:
I20260202 02:52:49.137269 1174 ts_tablet_manager.cc:1434] T b7880cb051c24bc998003f0effed3cf8 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.037s user 0.002s sys 0.000s
W20260202 02:52:49.137264 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 1d98c7a46139413b8f0d3697b74b1e52 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1d98c7a46139413b8f0d3697b74b1e52 already in progress: creating tablet
I20260202 02:52:49.137601 1882 rpcz_store.cc:276] 0202 02:52:29.749195 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.749251 (+ 56us) service_pool.cc:225] Handling call
0202 02:52:49.137057 (+19387806us) ts_tablet_manager.cc:1151] Deleting tablet 78a9a483b95c4bb5a5ab69abc899837b
0202 02:52:49.137066 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.137142 (+ 76us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":19387611}
I20260202 02:52:49.138104 1882 tablet_replica.cc:333] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
W20260202 02:52:49.138268 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 78a9a483b95c4bb5a5ab69abc899837b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 78a9a483b95c4bb5a5ab69abc899837b already in progress: creating tablet
I20260202 02:52:49.138507 1882 raft_consensus.cc:2243] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:49.138875 1882 raft_consensus.cc:2272] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:49.138939 1175 ts_tablet_manager.cc:1403] T da5435f969074364bc8f8a6ddc5e4f8a P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.071s user 0.003s sys 0.002s
I20260202 02:52:49.139444 1882 ts_tablet_manager.cc:1916] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:49.143194 1182 ts_tablet_manager.cc:1929] T 75a24a1bf54d4c799938a5723ac6fa99 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:49.143424 1182 log.cc:1199] T 75a24a1bf54d4c799938a5723ac6fa99 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/75a24a1bf54d4c799938a5723ac6fa99
I20260202 02:52:49.143803 1182 ts_tablet_manager.cc:1950] T 75a24a1bf54d4c799938a5723ac6fa99 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:49.144452 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 75a24a1bf54d4c799938a5723ac6fa99 (table default.Table_52a2f389faba40fea93e2bca972e81e1 [id=70ac373200604907b00132eb4727ea5e]) successfully deleted
I20260202 02:52:49.145299 1882 ts_tablet_manager.cc:1929] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:49.145418 1882 log.cc:1199] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/be993c1512b04df4b92cab3195241d84
I20260202 02:52:49.145516 1175 raft_consensus.cc:359] T da5435f969074364bc8f8a6ddc5e4f8a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:49.145728 1882 ts_tablet_manager.cc:1950] T be993c1512b04df4b92cab3195241d84 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:49.145772 1175 raft_consensus.cc:385] T da5435f969074364bc8f8a6ddc5e4f8a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:49.145905 1175 raft_consensus.cc:740] T da5435f969074364bc8f8a6ddc5e4f8a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:49.146202 1175 consensus_queue.cc:260] T da5435f969074364bc8f8a6ddc5e4f8a P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:49.146914 1175 ts_tablet_manager.cc:1434] T da5435f969074364bc8f8a6ddc5e4f8a P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
W20260202 02:52:49.147943 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet b8f2b34b4f1e4ff2b2be4ff40aeb0c66 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b8f2b34b4f1e4ff2b2be4ff40aeb0c66 already in progress: creating tablet
I20260202 02:52:49.154242 1175 tablet_bootstrap.cc:492] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:49.146517 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 678) took 19229 ms. Trace:
I20260202 02:52:49.156769 1882 rpcz_store.cc:276] 0202 02:52:29.916732 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.916796 (+ 64us) service_pool.cc:225] Handling call
0202 02:52:49.138052 (+19221256us) ts_tablet_manager.cc:1151] Deleting tablet be993c1512b04df4b92cab3195241d84
0202 02:52:49.138061 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.142439 (+ 4378us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.145204 (+ 2765us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.146494 (+ 1290us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":18897278}
I20260202 02:52:49.157298 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 707) took 18556 ms. Trace:
I20260202 02:52:49.157301 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet be993c1512b04df4b92cab3195241d84 (table default.Table_dd517ae94bf54267bc3a03dee4896708 [id=22f1ff6e13a84a72820c887dd3107f49]) successfully deleted
I20260202 02:52:49.157382 1882 rpcz_store.cc:276] 0202 02:52:30.607996 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.711412 (+103416us) service_pool.cc:225] Handling call
0202 02:52:49.157112 (+18445700us) ts_tablet_manager.cc:1151] Deleting tablet 91a604fb877a48479914a5b60144a22d
0202 02:52:49.157125 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.157281 (+ 156us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":13387012}
I20260202 02:52:49.157670 1882 tablet_replica.cc:333] T d8418a9dd0944456b4d4820dde0cde02 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:49.157858 1882 raft_consensus.cc:2243] T d8418a9dd0944456b4d4820dde0cde02 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
W20260202 02:52:49.157899 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 91a604fb877a48479914a5b60144a22d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 91a604fb877a48479914a5b60144a22d already in progress: creating tablet
I20260202 02:52:49.158012 1882 raft_consensus.cc:2272] T d8418a9dd0944456b4d4820dde0cde02 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:49.160362 1882 ts_tablet_manager.cc:1916] T d8418a9dd0944456b4d4820dde0cde02 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:49.160518 886 tablet_service.cc:1552] Processing DeleteTablet for tablet da5435f969074364bc8f8a6ddc5e4f8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:49.162645 1182 tablet_replica.cc:333] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:49.163148 1182 raft_consensus.cc:2243] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:49.163517 1182 pending_rounds.cc:70] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5: Trying to abort 1 pending ops.
I20260202 02:52:49.163641 1182 pending_rounds.cc:77] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923146911883264 op_type: NO_OP noop_request { }
I20260202 02:52:49.163816 1182 raft_consensus.cc:2889] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:49.163959 1182 raft_consensus.cc:2272] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:49.164721 1182 ts_tablet_manager.cc:1916] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:49.166090 1180 ts_tablet_manager.cc:1929] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:49.166219 1180 log.cc:1199] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/757492288d7c44fb91b25b7e6e21e01b
I20260202 02:52:49.166599 1180 ts_tablet_manager.cc:1950] T 757492288d7c44fb91b25b7e6e21e01b P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:49.170275 2448 raft_consensus.cc:2749] T 25765fb687294e9db1289f76530dd877 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260202 02:52:49.170584 714 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 58541us
W20260202 02:52:49.170960 714 leader_election.cc:343] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 94bada52db6a47179415d302b541d01f
I20260202 02:52:49.171066 714 leader_election.cc:304] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
W20260202 02:52:49.171241 714 raft_consensus.cc:2686] T 94bada52db6a47179415d302b541d01f P 1f442ef85029461091de847118c34c0d: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260202 02:52:49.172221 714 leader_election.cc:343] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 8774c6bc0fe9414ea2f17ed859d9392a
I20260202 02:52:49.172489 714 leader_election.cc:304] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:49.172684 2667 consensus_queue.cc:1059] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260202 02:52:49.172987 2667 raft_consensus.cc:2749] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260202 02:52:49.172988 1882 ts_tablet_manager.cc:1929] T d8418a9dd0944456b4d4820dde0cde02 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:49.173130 2667 raft_consensus.cc:3055] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Stepping down as leader of term 1
I20260202 02:52:49.173141 1882 log.cc:1199] T d8418a9dd0944456b4d4820dde0cde02 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/d8418a9dd0944456b4d4820dde0cde02
I20260202 02:52:49.173185 2667 raft_consensus.cc:740] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:49.173457 1882 ts_tablet_manager.cc:1950] T d8418a9dd0944456b4d4820dde0cde02 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:49.173414 2667 consensus_queue.cc:260] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:49.173691 2667 raft_consensus.cc:3060] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:49.174230 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 708) took 18566 ms. Trace:
I20260202 02:52:49.174333 1882 rpcz_store.cc:276] 0202 02:52:30.608156 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:35.770130 (+5161974us) service_pool.cc:225] Handling call
0202 02:52:49.157622 (+13387492us) ts_tablet_manager.cc:1151] Deleting tablet d8418a9dd0944456b4d4820dde0cde02
0202 02:52:49.157630 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.163396 (+ 5766us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.166184 (+ 2788us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.174209 (+ 8025us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":13387310}
I20260202 02:52:49.174665 1882 tablet_replica.cc:333] T 91ec01d25f8f441cbe3783cacbee51db P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:49.174877 1882 raft_consensus.cc:2243] T 91ec01d25f8f441cbe3783cacbee51db P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:49.175055 1882 raft_consensus.cc:2272] T 91ec01d25f8f441cbe3783cacbee51db P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:49.175586 1882 ts_tablet_manager.cc:1916] T 91ec01d25f8f441cbe3783cacbee51db P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:49.176769 714 leader_election.cc:343] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 38fdbc6df7ec40b9afe4e0dfe0119e3e
I20260202 02:52:49.176903 714 leader_election.cc:304] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:49.177404 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet d8418a9dd0944456b4d4820dde0cde02 (table default.Table_2d42dcf81d2d4b2c90fe09860b9b2318 [id=e620930eae8e4707b93e7ab16369e13d]) successfully deleted
I20260202 02:52:49.178793 2633 raft_consensus.cc:2749] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:49.179220 714 leader_election.cc:304] T 622a113af04a41c6852c29fc8a8acd78 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:49.179509 2633 raft_consensus.cc:2749] T 622a113af04a41c6852c29fc8a8acd78 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260202 02:52:49.179603 714 leader_election.cc:343] T ef073880898049e2be93d2cb6dc0c0bb P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: ef073880898049e2be93d2cb6dc0c0bb
I20260202 02:52:49.179692 714 leader_election.cc:304] T ef073880898049e2be93d2cb6dc0c0bb P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
W20260202 02:52:49.179850 714 raft_consensus.cc:2686] T ef073880898049e2be93d2cb6dc0c0bb P 1f442ef85029461091de847118c34c0d: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260202 02:52:49.180444 714 leader_election.cc:304] T f6746ccbf37e477196a9706396d1e3fd P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:49.180709 2633 raft_consensus.cc:2804] T f6746ccbf37e477196a9706396d1e3fd P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260202 02:52:49.180799 2633 raft_consensus.cc:493] T f6746ccbf37e477196a9706396d1e3fd P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election (detected failure of leader b1f05820b6034228bdad884cc348716d)
W20260202 02:52:49.180836 714 consensus_peers.cc:597] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 06b80a1c9251420cbc16fc95ae98aca9. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:49.180867 2633 raft_consensus.cc:3060] T f6746ccbf37e477196a9706396d1e3fd P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:49.181107 714 leader_election.cc:304] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
W20260202 02:52:49.181399 714 leader_election.cc:343] T 3d1a7578e83447669e3c3972a426ad44 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Illegal state: Tablet not RUNNING: STOPPED
I20260202 02:52:49.181481 714 leader_election.cc:304] T 3d1a7578e83447669e3c3972a426ad44 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:49.181958 1174 tablet_bootstrap.cc:492] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:49.183704 2633 raft_consensus.cc:515] T f6746ccbf37e477196a9706396d1e3fd P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.185081 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 558) took 22338 ms. Trace:
I20260202 02:52:49.185201 1180 rpcz_store.cc:276] 0202 02:52:26.846411 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.847285 (+ 874us) service_pool.cc:225] Handling call
0202 02:52:49.096437 (+22249152us) ts_tablet_manager.cc:1151] Deleting tablet 757492288d7c44fb91b25b7e6e21e01b
0202 02:52:49.096445 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.127761 (+ 31316us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.165954 (+ 38193us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.185066 (+ 19112us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":22248855}
I20260202 02:52:49.185516 1053 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f6746ccbf37e477196a9706396d1e3fd" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:49.185616 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 557) took 22339 ms. Trace:
I20260202 02:52:49.185676 1180 rpcz_store.cc:276] 0202 02:52:26.846292 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.847516 (+ 1224us) service_pool.cc:225] Handling call
0202 02:52:49.185470 (+22337954us) ts_tablet_manager.cc:1151] Deleting tablet f1c8f7e1dc9644a6b6dd2bc264cce8ef
0202 02:52:49.185480 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.185600 (+ 120us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":22337754}
I20260202 02:52:49.185971 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 556) took 22339 ms. Trace:
I20260202 02:52:49.186034 1180 rpcz_store.cc:276] 0202 02:52:26.846165 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.886610 (+ 40445us) service_pool.cc:225] Handling call
0202 02:52:49.185869 (+22299259us) ts_tablet_manager.cc:1151] Deleting tablet 36db4ee7f5404cffaa3c14ad58e71701
0202 02:52:49.185878 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.185956 (+ 78us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":22299010}
I20260202 02:52:49.186247 1180 tablet_replica.cc:333] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:49.186342 2633 leader_election.cc:290] T f6746ccbf37e477196a9706396d1e3fd P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:49.186425 1180 raft_consensus.cc:2243] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:49.187211 2667 raft_consensus.cc:2804] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260202 02:52:49.187247 2448 raft_consensus.cc:2749] T 3d1a7578e83447669e3c3972a426ad44 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260202 02:52:49.187297 2667 raft_consensus.cc:493] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election (detected failure of leader b1f05820b6034228bdad884cc348716d)
I20260202 02:52:49.187361 2667 raft_consensus.cc:3060] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:49.187579 1180 raft_consensus.cc:2272] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:49.188230 1180 ts_tablet_manager.cc:1916] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:49.189755 1882 ts_tablet_manager.cc:1929] T 91ec01d25f8f441cbe3783cacbee51db P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260202 02:52:49.189885 1882 log.cc:1199] T 91ec01d25f8f441cbe3783cacbee51db P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/91ec01d25f8f441cbe3783cacbee51db
I20260202 02:52:49.190191 1882 ts_tablet_manager.cc:1950] T 91ec01d25f8f441cbe3783cacbee51db P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:49.190552 2667 raft_consensus.cc:515] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:49.190990 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 709) took 18582 ms. Trace:
I20260202 02:52:49.191074 1882 rpcz_store.cc:276] 0202 02:52:30.608257 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:35.770302 (+5162045us) service_pool.cc:225] Handling call
0202 02:52:49.174619 (+13404317us) ts_tablet_manager.cc:1151] Deleting tablet 91ec01d25f8f441cbe3783cacbee51db
0202 02:52:49.174629 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.178688 (+ 4059us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.189641 (+ 10953us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.190964 (+ 1323us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":13404213}
I20260202 02:52:49.191408 1882 tablet_replica.cc:333] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:49.191591 1882 raft_consensus.cc:2243] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:49.191771 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 91ec01d25f8f441cbe3783cacbee51db (table default.Table_23afb316ffac47aea4db07bda15a956b [id=255f4d94812f45d0b4423150098460ff]) successfully deleted
I20260202 02:52:49.192152 1053 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "20ea39bc5d3d46f5a4ba427277d6c6a3" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:49.195580 714 leader_election.cc:304] T be993c1512b04df4b92cab3195241d84 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:49.195890 2633 raft_consensus.cc:2804] T be993c1512b04df4b92cab3195241d84 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260202 02:52:49.195963 2633 raft_consensus.cc:493] T be993c1512b04df4b92cab3195241d84 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election (detected failure of leader b1f05820b6034228bdad884cc348716d)
I20260202 02:52:49.196033 2633 raft_consensus.cc:3060] T be993c1512b04df4b92cab3195241d84 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:49.196638 1180 ts_tablet_manager.cc:1929] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:49.196869 1180 log.cc:1199] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/4e79cd48fefb4ce29ef48beacfc4d26f
I20260202 02:52:49.198901 2633 raft_consensus.cc:515] T be993c1512b04df4b92cab3195241d84 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:49.199503 2633 leader_election.cc:290] T be993c1512b04df4b92cab3195241d84 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:49.199648 2667 leader_election.cc:290] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:49.199963 714 leader_election.cc:304] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:49.200311 714 leader_election.cc:304] T 73f8db8f30f14d369d2227c9be7c4647 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:49.200737 714 leader_election.cc:304] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
W20260202 02:52:49.201009 714 leader_election.cc:343] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 26eeef6eebbf4f2cb44f69255fb37594
W20260202 02:52:49.201201 714 leader_election.cc:343] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 026e6ee0b3ae4d938fdf1550044ed1fa
I20260202 02:52:49.201263 714 leader_election.cc:304] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:49.201473 2619 raft_consensus.cc:2804] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260202 02:52:49.201530 2619 raft_consensus.cc:493] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election (detected failure of leader b1f05820b6034228bdad884cc348716d)
I20260202 02:52:49.201582 2619 raft_consensus.cc:3060] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Advancing to term 2
W20260202 02:52:49.203994 715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260202 02:52:49.204203 2619 raft_consensus.cc:515] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.204422 2669 raft_consensus.cc:2749] T 73f8db8f30f14d369d2227c9be7c4647 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260202 02:52:49.204598 2669 raft_consensus.cc:2804] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260202 02:52:49.204653 2669 raft_consensus.cc:493] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election (detected failure of leader b1f05820b6034228bdad884cc348716d)
I20260202 02:52:49.204715 2669 raft_consensus.cc:3060] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:49.204891 2619 leader_election.cc:290] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:49.205034 2619 raft_consensus.cc:2749] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:49.205926 714 leader_election.cc:304] T 16b47a2831f54d809ed0b16deaec7e3d P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:49.206214 2619 raft_consensus.cc:3060] T 16b47a2831f54d809ed0b16deaec7e3d P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:49.207268 2669 raft_consensus.cc:515] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.207834 2669 leader_election.cc:290] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:49.209352 2619 raft_consensus.cc:2749] T 16b47a2831f54d809ed0b16deaec7e3d P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260202 02:52:49.210080 714 leader_election.cc:400] T 26b890ed87c3471b9fc0b172c90e0a11 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Vote denied by peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) with higher term. Message: Invalid argument: T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f442ef85029461091de847118c34c0d for earlier term 1. Current term is 2.
I20260202 02:52:49.210137 714 leader_election.cc:403] T 26b890ed87c3471b9fc0b172c90e0a11 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Cancelling election due to peer responding with higher term
W20260202 02:52:49.210453 714 leader_election.cc:343] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 874ca4ba4ebc4721bb4b60cba8e01841
W20260202 02:52:49.210891 714 leader_election.cc:343] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 4e79cd48fefb4ce29ef48beacfc4d26f
I20260202 02:52:49.210983 714 leader_election.cc:304] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
W20260202 02:52:49.211134 714 raft_consensus.cc:2686] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1f442ef85029461091de847118c34c0d: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260202 02:52:49.211377 714 consensus_peers.cc:597] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fe53a8bdbd13417f9ce195c54a52a93c. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:49.211485 714 leader_election.cc:343] T 33150e1456144b8e9c64cc2a2ac6dcca P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 33150e1456144b8e9c64cc2a2ac6dcca
W20260202 02:52:49.211719 714 leader_election.cc:343] T 02b434fa058b45529ff3938b053263c6 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 02b434fa058b45529ff3938b053263c6
I20260202 02:52:49.211797 714 leader_election.cc:304] T 02b434fa058b45529ff3938b053263c6 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:49.197371 1180 ts_tablet_manager.cc:1950] T 4e79cd48fefb4ce29ef48beacfc4d26f P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:49.219902 1174 tablet_bootstrap.cc:654] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:49.222625 1053 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cb634d4bae554ef3b39eeaa9958adc6c" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:49.223752 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e1284fde67bc4d65b067a54c416ccc33" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
W20260202 02:52:49.223963 715 leader_election.cc:343] T be993c1512b04df4b92cab3195241d84 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Not found: Tablet not found: be993c1512b04df4b92cab3195241d84
I20260202 02:52:49.228708 1882 raft_consensus.cc:2272] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:49.229664 1882 ts_tablet_manager.cc:1916] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:49.233289 1174 tablet_bootstrap.cc:492] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:49.233438 1174 ts_tablet_manager.cc:1403] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.052s user 0.001s sys 0.002s
W20260202 02:52:49.245738 999 consensus_peers.cc:597] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 874ca4ba4ebc4721bb4b60cba8e01841. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:49.245947 514 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet eea6bb04ddf24276b46270d009ed78b4 (table default.Table_1366b68811c14d28b1785266d6f65a3a [id=b7139f5062824d66af2dbd62469a9c03], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet eea6bb04ddf24276b46270d009ed78b4 consensus state has no leader
I20260202 02:52:49.246469 2619 raft_consensus.cc:3060] T 26b890ed87c3471b9fc0b172c90e0a11 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:49.247808 1175 tablet_bootstrap.cc:654] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:49.249357 2619 raft_consensus.cc:2749] T 26b890ed87c3471b9fc0b172c90e0a11 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Leader pre-election lost for term 1. Reason: Vote denied by peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) with higher term. Message: Invalid argument: T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f442ef85029461091de847118c34c0d for earlier term 1. Current term is 2.
I20260202 02:52:49.250167 2619 raft_consensus.cc:2749] T 02b434fa058b45529ff3938b053263c6 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:49.250686 714 leader_election.cc:343] T 75569511179b41038e521fa3346a92ae P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 75569511179b41038e521fa3346a92ae
I20260202 02:52:49.250859 714 leader_election.cc:304] T 75569511179b41038e521fa3346a92ae P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:49.251287 714 leader_election.cc:304] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:49.251746 714 leader_election.cc:304] T b76d1837813c46e8a0925b49939d8884 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:49.251911 1174 raft_consensus.cc:359] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.252070 1174 raft_consensus.cc:385] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:49.252122 1174 raft_consensus.cc:740] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
W20260202 02:52:49.252300 714 leader_election.cc:343] T 33150e1456144b8e9c64cc2a2ac6dcca P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 33150e1456144b8e9c64cc2a2ac6dcca
I20260202 02:52:49.252298 1174 consensus_queue.cc:260] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.252923 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 560) took 22343 ms. Trace:
I20260202 02:52:49.253032 1174 ts_tablet_manager.cc:1434] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.020s user 0.000s sys 0.002s
I20260202 02:52:49.253026 1180 rpcz_store.cc:276] 0202 02:52:26.909644 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.909812 (+ 168us) service_pool.cc:225] Handling call
0202 02:52:49.186210 (+22276398us) ts_tablet_manager.cc:1151] Deleting tablet 4e79cd48fefb4ce29ef48beacfc4d26f
0202 02:52:49.186218 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.191488 (+ 5270us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.196507 (+ 5019us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.252894 (+ 56387us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":36357,"tablet-delete.queue_time_us":22276185}
I20260202 02:52:49.253441 1174 tablet_bootstrap.cc:492] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:49.253491 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 559) took 22344 ms. Trace:
I20260202 02:52:49.253572 1180 rpcz_store.cc:276] 0202 02:52:26.909491 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.914784 (+ 5293us) service_pool.cc:225] Handling call
0202 02:52:49.253345 (+22338561us) ts_tablet_manager.cc:1151] Deleting tablet 1c8317adc47a4c399f2db650ed3d2e53
0202 02:52:49.253358 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.253475 (+ 117us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":22314294}
I20260202 02:52:49.253868 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 561) took 22293 ms. Trace:
I20260202 02:52:49.253939 1180 rpcz_store.cc:276] 0202 02:52:26.960397 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:26.961297 (+ 900us) service_pool.cc:225] Handling call
0202 02:52:49.253768 (+22292471us) ts_tablet_manager.cc:1151] Deleting tablet 0138ba7326674741b0729324735f1de7
0202 02:52:49.253777 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.253854 (+ 77us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":22287203}
I20260202 02:52:49.254227 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 572) took 22101 ms. Trace:
I20260202 02:52:49.254287 1180 rpcz_store.cc:276] 0202 02:52:27.152924 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:27.152971 (+ 47us) service_pool.cc:225] Handling call
0202 02:52:49.254130 (+22101159us) ts_tablet_manager.cc:1151] Deleting tablet c977cd5aa66a4f6c8eeb37bf67f18fa7
0202 02:52:49.254140 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.254213 (+ 73us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":22100926}
I20260202 02:52:49.254590 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 573) took 22101 ms. Trace:
I20260202 02:52:49.254658 1180 rpcz_store.cc:276] 0202 02:52:27.153060 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:27.153111 (+ 51us) service_pool.cc:225] Handling call
0202 02:52:49.254462 (+22101351us) ts_tablet_manager.cc:1151] Deleting tablet b9fc33e8efc04734a951f7fcf9048142
0202 02:52:49.254470 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.254574 (+ 104us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":22101214}
I20260202 02:52:49.254885 1174 tablet_bootstrap.cc:654] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:49.254961 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 571) took 22102 ms. Trace:
I20260202 02:52:49.255025 1180 rpcz_store.cc:276] 0202 02:52:27.152832 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:27.161993 (+ 9161us) service_pool.cc:225] Handling call
0202 02:52:49.254861 (+22092868us) ts_tablet_manager.cc:1151] Deleting tablet 550895b4010f4088b2cc34acff50e03f
0202 02:52:49.254870 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.254947 (+ 77us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":22092562}
I20260202 02:52:49.255273 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 574) took 22092 ms. Trace:
I20260202 02:52:49.255330 1180 rpcz_store.cc:276] 0202 02:52:27.162378 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:27.164908 (+ 2530us) service_pool.cc:225] Handling call
0202 02:52:49.255189 (+22090281us) ts_tablet_manager.cc:1151] Deleting tablet 940e656cd34e4e53896fda6d38132b6e
0202 02:52:49.255197 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.255261 (+ 64us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":22089989}
I20260202 02:52:49.255591 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 575) took 22043 ms. Trace:
I20260202 02:52:49.252543 2667 raft_consensus.cc:2749] T 75569511179b41038e521fa3346a92ae P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:49.252594 2633 raft_consensus.cc:2749] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260202 02:52:49.258679 2445 consensus_queue.cc:1048] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:49.252624 2619 raft_consensus.cc:2749] T b76d1837813c46e8a0925b49939d8884 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260202 02:52:49.255664 1180 rpcz_store.cc:276] 0202 02:52:27.212596 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:27.212658 (+ 62us) service_pool.cc:225] Handling call
0202 02:52:49.255494 (+22042836us) ts_tablet_manager.cc:1151] Deleting tablet 85f754f006d6432bb1cba87043903ac2
0202 02:52:49.255501 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.255578 (+ 77us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":22042589}
I20260202 02:52:49.261404 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 576) took 22043 ms. Trace:
I20260202 02:52:49.261507 1180 rpcz_store.cc:276] 0202 02:52:27.218341 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:27.218401 (+ 60us) service_pool.cc:225] Handling call
0202 02:52:49.261247 (+22042846us) ts_tablet_manager.cc:1151] Deleting tablet 7b45a2e6b4b14d3fa96a7131afb16208
0202 02:52:49.261258 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.261383 (+ 125us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":22042651}
I20260202 02:52:49.265189 1174 tablet_bootstrap.cc:492] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:49.265439 1174 ts_tablet_manager.cc:1403] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.002s
I20260202 02:52:49.265990 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 577) took 22045 ms. Trace:
I20260202 02:52:49.266181 1175 tablet_bootstrap.cc:492] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:49.266321 1175 ts_tablet_manager.cc:1403] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.113s user 0.009s sys 0.001s
I20260202 02:52:49.267077 1175 raft_consensus.cc:359] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:49.267237 1175 raft_consensus.cc:385] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:49.267289 1175 raft_consensus.cc:740] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:49.266752 1180 rpcz_store.cc:276] 0202 02:52:27.220495 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:27.220544 (+ 49us) service_pool.cc:225] Handling call
0202 02:52:49.261752 (+22041208us) ts_tablet_manager.cc:1151] Deleting tablet 25f1d698c4ee4be7a33380184b5e2b4d
0202 02:52:49.265439 (+ 3687us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.265964 (+ 525us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":22041013}
I20260202 02:52:49.267519 1175 consensus_queue.cc:260] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:49.268113 1175 ts_tablet_manager.cc:1434] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:49.268744 1174 raft_consensus.cc:359] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.268896 1174 raft_consensus.cc:385] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:49.268939 1174 raft_consensus.cc:740] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:49.269343 1174 consensus_queue.cc:260] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.269796 1174 ts_tablet_manager.cc:1434] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260202 02:52:49.271375 1182 ts_tablet_manager.cc:1929] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:49.273350 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 589) took 21832 ms. Trace:
I20260202 02:52:49.275386 1180 rpcz_store.cc:276] 0202 02:52:27.440665 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:27.442499 (+ 1834us) service_pool.cc:225] Handling call
0202 02:52:49.269695 (+21827196us) ts_tablet_manager.cc:1151] Deleting tablet 4126bf5218644ca583c82190ce952267
0202 02:52:49.271517 (+ 1822us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.273322 (+ 1805us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":21826980}
I20260202 02:52:49.276722 528 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_52a2f389faba40fea93e2bca972e81e1" } modify_external_catalogs: true
I20260202 02:52:49.276948 528 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_52a2f389faba40fea93e2bca972e81e1" } modify_external_catalogs: true
I20260202 02:52:49.277459 526 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_d2f92c0892494da18afb7e3336faec7a" } modify_external_catalogs: true
I20260202 02:52:49.277617 526 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_d2f92c0892494da18afb7e3336faec7a" } modify_external_catalogs: true
I20260202 02:52:49.278244 520 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_4e4d461d49574c68a7d93ac891db67c6" } modify_external_catalogs: true
I20260202 02:52:49.278386 520 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_4e4d461d49574c68a7d93ac891db67c6" } modify_external_catalogs: true
I20260202 02:52:49.279978 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_a486c34ee08d4e55978c0e884529329b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:49.281536 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a486c34ee08d4e55978c0e884529329b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:49.282642 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_645e1697bb644d629ba1c4b4f6c5ae33"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:49.283227 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_645e1697bb644d629ba1c4b4f6c5ae33 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:49.288587 520 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 80eb3013f5354022a2dd78bb294a8497
I20260202 02:52:49.290169 524 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_0276ce9d8aaa44fa832bb88a8b1f9446" } modify_external_catalogs: true
I20260202 02:52:49.290400 524 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_0276ce9d8aaa44fa832bb88a8b1f9446" } modify_external_catalogs: true
I20260202 02:52:49.294730 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_cda21f6dd8d34e47abae4a73c38b597c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:49.295349 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cda21f6dd8d34e47abae4a73c38b597c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:49.295962 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_b40b893658b34e4593e3eb6f6af5b829"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:49.296533 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b40b893658b34e4593e3eb6f6af5b829 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:49.297266 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_190bbbb8a6a6454cb95f513c1c72b8fd" } new_table_name: "default.Table_fdd03abf2a0e4d3a8f621f28cc062904" modify_external_catalogs: true
I20260202 02:52:49.271770 1175 tablet_bootstrap.cc:492] T 59ad016e15ec4a488921eb564029b689 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:49.299517 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_f61ca1b0d7424868993ccd4657d50429"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:49.299794 1175 tablet_bootstrap.cc:654] T 59ad016e15ec4a488921eb564029b689 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:49.300102 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f61ca1b0d7424868993ccd4657d50429 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:49.289623 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 80eb3013f5354022a2dd78bb294a8497 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:49.300725 519 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_9ecd3c559b81444ba6573e92172911ee" } new_table_name: "default.Table_62b7624afec14fb6a860d51a1a09913d" modify_external_catalogs: true
I20260202 02:52:49.302764 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet 80eb3013f5354022a2dd78bb294a8497 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:49.302721 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_9d1b173cf04349108d4b0b4d810e9cb0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:49.303272 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9d1b173cf04349108d4b0b4d810e9cb0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:49.304672 715 consensus_peers.cc:597] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 26eeef6eebbf4f2cb44f69255fb37594. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:49.304843 1182 log.cc:1199] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/79c73f4a92044bafb5b8384229cb3a4d
I20260202 02:52:49.305194 1182 ts_tablet_manager.cc:1950] T 79c73f4a92044bafb5b8384229cb3a4d P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:49.305712 1175 tablet_bootstrap.cc:492] T 59ad016e15ec4a488921eb564029b689 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:49.305842 1175 ts_tablet_manager.cc:1403] T 59ad016e15ec4a488921eb564029b689 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.034s user 0.002s sys 0.002s
I20260202 02:52:49.306337 1182 tablet_replica.cc:333] T 842e54adae2a452a98521de24326eaff P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:49.306319 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 79c73f4a92044bafb5b8384229cb3a4d (table default.Table_a15c97bb32284dcfb36cdc47921b2346 [id=dc77705cb5704fd58af1eed3edd45675]) successfully deleted
I20260202 02:52:49.306511 1182 raft_consensus.cc:2243] T 842e54adae2a452a98521de24326eaff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:49.306558 1175 raft_consensus.cc:359] T 59ad016e15ec4a488921eb564029b689 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:49.306655 1182 raft_consensus.cc:2272] T 842e54adae2a452a98521de24326eaff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:49.306705 1175 raft_consensus.cc:385] T 59ad016e15ec4a488921eb564029b689 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:49.306757 1175 raft_consensus.cc:740] T 59ad016e15ec4a488921eb564029b689 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:49.307013 524 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 73f8db8f30f14d369d2227c9be7c4647
I20260202 02:52:49.306962 1175 consensus_queue.cc:260] T 59ad016e15ec4a488921eb564029b689 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:49.307170 1182 ts_tablet_manager.cc:1916] T 842e54adae2a452a98521de24326eaff P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:49.307869 1173 tablet_bootstrap.cc:492] T fbcb4a522df648ba9542250450c08f1b P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:49.308034 1173 ts_tablet_manager.cc:1403] T fbcb4a522df648ba9542250450c08f1b P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.213s user 0.015s sys 0.025s
I20260202 02:52:49.308866 1173 raft_consensus.cc:359] T fbcb4a522df648ba9542250450c08f1b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.309041 1173 raft_consensus.cc:385] T fbcb4a522df648ba9542250450c08f1b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:49.309100 1173 raft_consensus.cc:740] T fbcb4a522df648ba9542250450c08f1b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:49.309312 1173 consensus_queue.cc:260] T fbcb4a522df648ba9542250450c08f1b P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.309844 1173 ts_tablet_manager.cc:1434] T fbcb4a522df648ba9542250450c08f1b P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:49.310163 1173 tablet_bootstrap.cc:492] T 072bb7f0e4ab4935be49461bc8a3c163 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:49.312109 1173 tablet_bootstrap.cc:654] T 072bb7f0e4ab4935be49461bc8a3c163 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:49.314152 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 590) took 21871 ms. Trace:
I20260202 02:52:49.314253 1180 rpcz_store.cc:276] 0202 02:52:27.442718 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:27.443050 (+ 332us) service_pool.cc:225] Handling call
0202 02:52:49.275853 (+21832803us) ts_tablet_manager.cc:1151] Deleting tablet 2c2a535de2294cc697d75561d0463642
0202 02:52:49.313983 (+ 38130us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.314123 (+ 140us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":21832561}
I20260202 02:52:49.314916 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet 73f8db8f30f14d369d2227c9be7c4647 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:49.316642 522 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_7b5f2de229cb4b2f8bd174ccfd6b9553" } new_table_name: "default.Table_a5c327c53c1141b79b4274480bbc8f11" modify_external_catalogs: true
I20260202 02:52:49.318035 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_c88baa3501324c49a705daab8b694d0c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:49.322360 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c88baa3501324c49a705daab8b694d0c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:49.324642 1173 tablet_bootstrap.cc:492] T 072bb7f0e4ab4935be49461bc8a3c163 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:49.324841 1173 ts_tablet_manager.cc:1403] T 072bb7f0e4ab4935be49461bc8a3c163 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.005s
I20260202 02:52:49.324919 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 591) took 21790 ms. Trace:
I20260202 02:52:49.325076 1180 rpcz_store.cc:276] 0202 02:52:27.534655 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:27.535087 (+ 432us) service_pool.cc:225] Handling call
0202 02:52:49.314496 (+21779409us) ts_tablet_manager.cc:1151] Deleting tablet 8cf6a52eea054ab688410e2dd029abf3
0202 02:52:49.324711 (+ 10215us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.324888 (+ 177us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":21779203}
I20260202 02:52:49.326138 1174 tablet_bootstrap.cc:492] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:49.326223 1173 raft_consensus.cc:359] T 072bb7f0e4ab4935be49461bc8a3c163 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.326429 1173 raft_consensus.cc:385] T 072bb7f0e4ab4935be49461bc8a3c163 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:49.326516 1173 raft_consensus.cc:740] T 072bb7f0e4ab4935be49461bc8a3c163 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:49.326787 1173 consensus_queue.cc:260] T 072bb7f0e4ab4935be49461bc8a3c163 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:49.327032 714 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 74731.5us
W20260202 02:52:49.327275 714 leader_election.cc:343] T fe9ea4c0502f4976b440465992f2ea2c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: fe9ea4c0502f4976b440465992f2ea2c
I20260202 02:52:49.327378 714 leader_election.cc:304] T fe9ea4c0502f4976b440465992f2ea2c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:49.327412 1173 ts_tablet_manager.cc:1434] T 072bb7f0e4ab4935be49461bc8a3c163 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260202 02:52:49.327595 2667 raft_consensus.cc:2749] T fe9ea4c0502f4976b440465992f2ea2c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:49.327715 1173 tablet_bootstrap.cc:492] T 842e54adae2a452a98521de24326eaff P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:49.327751 714 leader_election.cc:304] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:49.327989 2633 raft_consensus.cc:2749] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260202 02:52:49.328121 714 leader_election.cc:304] T d06b77ff1a6f4f78b73082b00145fcf6 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:49.328161 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 596) took 21774 ms. Trace:
I20260202 02:52:49.328297 2633 raft_consensus.cc:2749] T d06b77ff1a6f4f78b73082b00145fcf6 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260202 02:52:49.328490 714 leader_election.cc:304] T 79c73f4a92044bafb5b8384229cb3a4d P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:49.328365 1180 rpcz_store.cc:276] 0202 02:52:27.554112 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:27.556398 (+ 2286us) service_pool.cc:225] Handling call
0202 02:52:49.325554 (+21769156us) ts_tablet_manager.cc:1151] Deleting tablet ce757ddc5c144d1fbaec84c9723cdbf2
0202 02:52:49.327992 (+ 2438us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.328143 (+ 151us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":21768972}
W20260202 02:52:49.328883 714 leader_election.cc:343] T dada209d982d48f3b74a49905f19c5b6 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: dada209d982d48f3b74a49905f19c5b6
I20260202 02:52:49.329046 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 640) took 20950 ms. Trace:
W20260202 02:52:49.329128 714 leader_election.cc:343] T dada209d982d48f3b74a49905f19c5b6 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: dada209d982d48f3b74a49905f19c5b6
I20260202 02:52:49.329155 1180 rpcz_store.cc:276] 0202 02:52:28.378123 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:28.378203 (+ 80us) service_pool.cc:225] Handling call
0202 02:52:49.328860 (+20950657us) ts_tablet_manager.cc:1151] Deleting tablet 425dab1cf74c4f379f10e202613093dd
0202 02:52:49.328871 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.329026 (+ 155us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":20950440}
I20260202 02:52:49.329360 1174 tablet_bootstrap.cc:654] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:49.330183 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 639) took 20952 ms. Trace:
I20260202 02:52:49.330295 1180 rpcz_store.cc:276] 0202 02:52:28.377987 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:28.380279 (+ 2292us) service_pool.cc:225] Handling call
0202 02:52:49.330072 (+20949793us) ts_tablet_manager.cc:1151] Deleting tablet 2c31b45dc8004eae993e972e9faa3ff8
0202 02:52:49.330082 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.330163 (+ 81us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":20941100}
I20260202 02:52:49.330595 1180 tablet_replica.cc:333] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:49.330888 1180 raft_consensus.cc:2243] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:49.331120 1180 raft_consensus.cc:2272] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:49.331743 1180 ts_tablet_manager.cc:1916] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:49.332414 2619 raft_consensus.cc:2749] T 79c73f4a92044bafb5b8384229cb3a4d P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:49.307886 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 73f8db8f30f14d369d2227c9be7c4647 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:49.313252 1175 ts_tablet_manager.cc:1434] T 59ad016e15ec4a488921eb564029b689 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260202 02:52:49.334080 1175 tablet_bootstrap.cc:492] T e2edc6660b9a41e39e9529627fafcb03 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
W20260202 02:52:49.335000 527 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ec3511e5c58b4dba9e48e2b73df833a4 consensus state has no leader
I20260202 02:52:49.335418 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_20a23d2f97dc42fcbff9071d1b935941" } new_table_name: "default.Table_1fa8930369e844ca8384126bf8f22e66" modify_external_catalogs: true
I20260202 02:52:49.335649 1175 tablet_bootstrap.cc:654] T e2edc6660b9a41e39e9529627fafcb03 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:49.336268 519 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ee04563d3b8e4cb2b989bb8d6ee9b20e consensus state has no leader
I20260202 02:52:49.336603 519 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ed899eb11436405fa74f3b38d08552f7" } new_table_name: "default.Table_dfa22e5a08544023b64919ccf651c07a" modify_external_catalogs: true
I20260202 02:52:49.336637 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_a3b4c2b185b143b49e036ffd0b4d33af"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:49.337185 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a3b4c2b185b143b49e036ffd0b4d33af in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:49.329398 1173 tablet_bootstrap.cc:654] T 842e54adae2a452a98521de24326eaff P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:49.329372 714 leader_election.cc:304] T 80eb3013f5354022a2dd78bb294a8497 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:49.341171 2619 raft_consensus.cc:2749] T 80eb3013f5354022a2dd78bb294a8497 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260202 02:52:49.343698 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 761f5a51335a4de5977f5f7fef0d4260 (table default.Table_88cb19d3d7684dc1a8aa41ed28fed4b8 [id=b40664207c364307ba6f5ade490a21d6]) successfully deleted
I20260202 02:52:49.344027 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet d0671586e45e45ffb7fd9e1be6e2bbf8 (table default.Table_53cc5731c3e24222a97702bdde84a954 [id=b2fe4f8ae50642e9a6ea05759313b31b]) successfully deleted
W20260202 02:52:49.344251 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 2f664f6119ba4d28b5ef1c212e611866 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2f664f6119ba4d28b5ef1c212e611866 already in progress: creating tablet
W20260202 02:52:49.344511 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet dc90eeb79ded4739883ad98928476680 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dc90eeb79ded4739883ad98928476680 already in progress: creating tablet
W20260202 02:52:49.344730 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 0a5bbb994cfa40f0add3163816b65269 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0a5bbb994cfa40f0add3163816b65269 already in progress: creating tablet
I20260202 02:52:49.344828 915 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "20ea39bc5d3d46f5a4ba427277d6c6a3" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
W20260202 02:52:49.344950 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 8ba28ab7d7ae44059006179067dff3c0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8ba28ab7d7ae44059006179067dff3c0 already in progress: creating tablet
W20260202 02:52:49.345136 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 23671408fcbc454cbd918328015cf6e6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 23671408fcbc454cbd918328015cf6e6 already in progress: creating tablet
I20260202 02:52:49.345198 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cb634d4bae554ef3b39eeaa9958adc6c" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:49.345585 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 757492288d7c44fb91b25b7e6e21e01b (table default.Table_7c01ea0d668a490099dbd42db17162ec [id=02e9e243f710428091de2f9497532c37]) successfully deleted
I20260202 02:52:49.345538 919 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e1284fde67bc4d65b067a54c416ccc33" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
W20260202 02:52:49.345784 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet f1c8f7e1dc9644a6b6dd2bc264cce8ef because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f1c8f7e1dc9644a6b6dd2bc264cce8ef already in progress: creating tablet
I20260202 02:52:49.345875 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f6746ccbf37e477196a9706396d1e3fd" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
W20260202 02:52:49.345945 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 36db4ee7f5404cffaa3c14ad58e71701 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 36db4ee7f5404cffaa3c14ad58e71701 already in progress: creating tablet
I20260202 02:52:49.346194 1175 tablet_bootstrap.cc:492] T e2edc6660b9a41e39e9529627fafcb03 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:49.346320 1175 ts_tablet_manager.cc:1403] T e2edc6660b9a41e39e9529627fafcb03 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.004s
I20260202 02:52:49.347018 1175 raft_consensus.cc:359] T e2edc6660b9a41e39e9529627fafcb03 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:49.347177 1175 raft_consensus.cc:385] T e2edc6660b9a41e39e9529627fafcb03 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:49.347227 1175 raft_consensus.cc:740] T e2edc6660b9a41e39e9529627fafcb03 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:49.347541 1175 consensus_queue.cc:260] T e2edc6660b9a41e39e9529627fafcb03 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:49.348130 1175 ts_tablet_manager.cc:1434] T e2edc6660b9a41e39e9529627fafcb03 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:49.348445 1175 tablet_bootstrap.cc:492] T abfe7f76b2424c4d9ab07b29b66ec103 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:49.348719 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 4e79cd48fefb4ce29ef48beacfc4d26f (table default.Table_bed5f0c218b24f529e48138901d1afa0 [id=2a625eeef55b43f19cd4bbeb61e85bca]) successfully deleted
W20260202 02:52:49.349076 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 1c8317adc47a4c399f2db650ed3d2e53 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1c8317adc47a4c399f2db650ed3d2e53 already in progress: creating tablet
W20260202 02:52:49.349306 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 0138ba7326674741b0729324735f1de7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0138ba7326674741b0729324735f1de7 already in progress: creating tablet
W20260202 02:52:49.349499 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet c977cd5aa66a4f6c8eeb37bf67f18fa7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c977cd5aa66a4f6c8eeb37bf67f18fa7 already in progress: creating tablet
W20260202 02:52:49.349697 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet b9fc33e8efc04734a951f7fcf9048142 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b9fc33e8efc04734a951f7fcf9048142 already in progress: creating tablet
I20260202 02:52:49.349803 1175 tablet_bootstrap.cc:654] T abfe7f76b2424c4d9ab07b29b66ec103 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:49.349860 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 550895b4010f4088b2cc34acff50e03f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 550895b4010f4088b2cc34acff50e03f already in progress: creating tablet
W20260202 02:52:49.350064 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 940e656cd34e4e53896fda6d38132b6e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 940e656cd34e4e53896fda6d38132b6e already in progress: creating tablet
W20260202 02:52:49.350546 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 85f754f006d6432bb1cba87043903ac2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 85f754f006d6432bb1cba87043903ac2 already in progress: creating tablet
W20260202 02:52:49.350801 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 7b45a2e6b4b14d3fa96a7131afb16208 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7b45a2e6b4b14d3fa96a7131afb16208 already in progress: creating tablet
W20260202 02:52:49.350996 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 25f1d698c4ee4be7a33380184b5e2b4d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 25f1d698c4ee4be7a33380184b5e2b4d already in progress: creating tablet
W20260202 02:52:49.351209 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 4126bf5218644ca583c82190ce952267 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4126bf5218644ca583c82190ce952267 already in progress: creating tablet
I20260202 02:52:49.352074 1174 tablet_bootstrap.cc:492] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:49.352223 1174 ts_tablet_manager.cc:1403] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.004s
I20260202 02:52:49.352972 1174 raft_consensus.cc:359] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.353155 1174 raft_consensus.cc:385] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:49.353206 1174 raft_consensus.cc:740] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:49.353555 1174 consensus_queue.cc:260] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.354421 1173 tablet_bootstrap.cc:492] T 842e54adae2a452a98521de24326eaff P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:49.354593 1173 ts_tablet_manager.cc:1403] T 842e54adae2a452a98521de24326eaff P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.027s user 0.001s sys 0.003s
W20260202 02:52:49.354876 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 2c2a535de2294cc697d75561d0463642 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2c2a535de2294cc697d75561d0463642 already in progress: creating tablet
I20260202 02:52:49.355197 1174 ts_tablet_manager.cc:1434] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260202 02:52:49.355407 1173 raft_consensus.cc:359] T 842e54adae2a452a98521de24326eaff P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.355561 1174 tablet_bootstrap.cc:492] T 072bb7f0e4ab4935be49461bc8a3c163 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:49.355584 1173 raft_consensus.cc:385] T 842e54adae2a452a98521de24326eaff P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:49.355652 1173 raft_consensus.cc:740] T 842e54adae2a452a98521de24326eaff P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
W20260202 02:52:49.355867 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 8cf6a52eea054ab688410e2dd029abf3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8cf6a52eea054ab688410e2dd029abf3 already in progress: creating tablet
I20260202 02:52:49.355897 1173 consensus_queue.cc:260] T 842e54adae2a452a98521de24326eaff P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:49.356110 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet ce757ddc5c144d1fbaec84c9723cdbf2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ce757ddc5c144d1fbaec84c9723cdbf2 already in progress: creating tablet
W20260202 02:52:49.356304 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 425dab1cf74c4f379f10e202613093dd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 425dab1cf74c4f379f10e202613093dd already in progress: creating tablet
W20260202 02:52:49.356531 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 2c31b45dc8004eae993e972e9faa3ff8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2c31b45dc8004eae993e972e9faa3ff8 already in progress: creating tablet
I20260202 02:52:49.356599 1173 ts_tablet_manager.cc:1434] T 842e54adae2a452a98521de24326eaff P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260202 02:52:49.356890 1173 tablet_bootstrap.cc:492] T 6df3d11cf578404ca41807f0777b6c53 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:49.357012 1174 tablet_bootstrap.cc:654] T 072bb7f0e4ab4935be49461bc8a3c163 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:49.358645 1173 tablet_bootstrap.cc:654] T 6df3d11cf578404ca41807f0777b6c53 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:49.359227 1882 ts_tablet_manager.cc:1929] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:49.359375 1882 log.cc:1199] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/026e6ee0b3ae4d938fdf1550044ed1fa
W20260202 02:52:49.359659 715 consensus_peers.cc:597] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d30557fbe9084cd3860820f69dda8752. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:49.359740 1882 ts_tablet_manager.cc:1950] T 026e6ee0b3ae4d938fdf1550044ed1fa P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:49.360054 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 710) took 18751 ms. Trace:
I20260202 02:52:49.360162 1882 rpcz_store.cc:276] 0202 02:52:30.616378 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:35.770412 (+5154034us) service_pool.cc:225] Handling call
0202 02:52:49.191359 (+13420947us) ts_tablet_manager.cc:1151] Deleting tablet 026e6ee0b3ae4d938fdf1550044ed1fa
0202 02:52:49.191369 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.233255 (+ 41886us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.359068 (+125813us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.360013 (+ 945us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":13420849}
I20260202 02:52:49.364770 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 80eb3013f5354022a2dd78bb294a8497 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:49.365206 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 73f8db8f30f14d369d2227c9be7c4647 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:42950
W20260202 02:52:49.367247 714 leader_election.cc:343] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 91ec01d25f8f441cbe3783cacbee51db
I20260202 02:52:49.367630 714 leader_election.cc:304] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:49.368534 2667 raft_consensus.cc:2749] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260202 02:52:49.368978 714 leader_election.cc:343] T be993c1512b04df4b92cab3195241d84 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: be993c1512b04df4b92cab3195241d84
I20260202 02:52:49.369087 714 leader_election.cc:304] T be993c1512b04df4b92cab3195241d84 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:49.369403 2667 raft_consensus.cc:2749] T be993c1512b04df4b92cab3195241d84 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260202 02:52:49.369722 714 leader_election.cc:304] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:49.370689 2667 raft_consensus.cc:2749] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260202 02:52:49.371064 714 leader_election.cc:304] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:49.372495 2667 raft_consensus.cc:2749] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260202 02:52:49.373066 1174 tablet_bootstrap.cc:492] T 072bb7f0e4ab4935be49461bc8a3c163 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:49.373184 1174 ts_tablet_manager.cc:1403] T 072bb7f0e4ab4935be49461bc8a3c163 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.000s
I20260202 02:52:49.373875 1174 raft_consensus.cc:359] T 072bb7f0e4ab4935be49461bc8a3c163 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.374051 1174 raft_consensus.cc:385] T 072bb7f0e4ab4935be49461bc8a3c163 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:49.374115 1174 raft_consensus.cc:740] T 072bb7f0e4ab4935be49461bc8a3c163 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:49.374523 714 leader_election.cc:304] T f6746ccbf37e477196a9706396d1e3fd P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:49.375185 2667 raft_consensus.cc:2749] T f6746ccbf37e477196a9706396d1e3fd P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260202 02:52:49.375635 714 consensus_peers.cc:597] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 26eeef6eebbf4f2cb44f69255fb37594. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:49.375877 714 consensus_peers.cc:597] T 33150e1456144b8e9c64cc2a2ac6dcca P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 33150e1456144b8e9c64cc2a2ac6dcca. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:49.377266 1174 consensus_queue.cc:260] T 072bb7f0e4ab4935be49461bc8a3c163 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.379983 1174 ts_tablet_manager.cc:1434] T 072bb7f0e4ab4935be49461bc8a3c163 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260202 02:52:49.381417 1174 tablet_bootstrap.cc:492] T 6df3d11cf578404ca41807f0777b6c53 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:49.387414 1174 tablet_bootstrap.cc:654] T 6df3d11cf578404ca41807f0777b6c53 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:49.369843 2482 consensus_queue.cc:1048] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:49.390424 2753 tablet.cc:1722] T 70306ed9dec54286885d965f2c012da4 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260202 02:52:49.390960 2445 raft_consensus.cc:493] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:49.391083 2445 raft_consensus.cc:515] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:49.391155 773 raft_consensus.cc:3060] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:49.391338 1882 tablet_replica.cc:333] T dada209d982d48f3b74a49905f19c5b6 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:49.391538 1882 raft_consensus.cc:2243] T dada209d982d48f3b74a49905f19c5b6 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:49.391664 2445 leader_election.cc:290] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:49.391714 1882 raft_consensus.cc:2272] T dada209d982d48f3b74a49905f19c5b6 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:49.392370 1882 ts_tablet_manager.cc:1916] T dada209d982d48f3b74a49905f19c5b6 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:49.392827 765 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c8a273ac8cd344719a48fd81945bed0a" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:49.393385 857 leader_election.cc:343] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:49.394287 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 026e6ee0b3ae4d938fdf1550044ed1fa (table default.Table_f93d759396c04d25ac602a609ecc4962 [id=7482d135c01241b4b3752b88fb1a0b2c]) successfully deleted
I20260202 02:52:49.394780 1053 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c8a273ac8cd344719a48fd81945bed0a" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:49.395290 859 leader_election.cc:343] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:49.395393 859 leader_election.cc:304] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:49.395764 2443 consensus_queue.cc:1048] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:49.396183 2443 raft_consensus.cc:2749] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:49.399766 1182 ts_tablet_manager.cc:1929] T 842e54adae2a452a98521de24326eaff P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260202 02:52:49.402500 522 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bdd72f995a704520a5fb1c24a53ed252 consensus state has no leader
I20260202 02:52:49.404142 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_ee8db454beae4d16a36f06dab8f41364"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:49.404939 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ee8db454beae4d16a36f06dab8f41364 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:49.409467 523 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_7a09e5be1e824c82bfd2d252cbd1043e" } modify_external_catalogs: true
I20260202 02:52:49.409754 523 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_7a09e5be1e824c82bfd2d252cbd1043e" } modify_external_catalogs: true
I20260202 02:52:49.410565 524 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet c8a273ac8cd344719a48fd81945bed0a (table default.Table_5a703dc84eb34674bdc5793ac7295098 [id=62c3c39fb73249349cee49b86d535612]). Expected version 2 got 0
I20260202 02:52:49.411437 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_a8324bdbd82941c8b8e035d01c6997a6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:49.412102 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a8324bdbd82941c8b8e035d01c6997a6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:49.415009 520 master_service.cc:954] ReplaceTablet: received request to replace tablet 00f2d3bafb2d472aaffc0a7ba926c825 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:49.416476 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet ccd6bc8bd0504bf18d108c73685ef9ff with delete_type TABLET_DATA_DELETED (replaced by tablet 8ba28ab7d7ae44059006179067dff3c0) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:49.416755 1035 tablet_service.cc:1552] Processing DeleteTablet for tablet 2c31b45dc8004eae993e972e9faa3ff8 with delete_type TABLET_DATA_DELETED (replaced by tablet aed1081136f7431b9b14784c21112d1c) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:49.416929 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 474df41e4fe24a818fa45a4ce84d5474 with delete_type TABLET_DATA_DELETED (replaced by tablet f30b698c177c48ee9a4e577552bf7eee) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:49.417080 1034 tablet_service.cc:1552] Processing DeleteTablet for tablet d8418a9dd0944456b4d4820dde0cde02 with delete_type TABLET_DATA_DELETED (replaced by tablet 1b2b4f429b6a484ea6ef7c19124b236b) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:49.417459 1041 tablet_service.cc:1505] Processing CreateTablet for tablet 477c599f8c9b4eb0b435ea50abd5d36b (DEFAULT_TABLE table=default.Table_f7636b54f7f34f2a9487c03e57a621a9 [id=68bb32ad1fd34ab28681b80cccea3f3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.417917 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 477c599f8c9b4eb0b435ea50abd5d36b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.419204 2633 raft_consensus.cc:493] T 072bb7f0e4ab4935be49461bc8a3c163 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:49.419355 2633 raft_consensus.cc:515] T 072bb7f0e4ab4935be49461bc8a3c163 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.419982 2633 leader_election.cc:290] T 072bb7f0e4ab4935be49461bc8a3c163 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:49.421267 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "072bb7f0e4ab4935be49461bc8a3c163" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:49.421501 1045 raft_consensus.cc:2468] T 072bb7f0e4ab4935be49461bc8a3c163 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:49.421655 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 41f2a25a3ca84d84af18fcd8c58b6603 (DEFAULT_TABLE table=default.Table_274e129a05bd42a39cbee9731d492dac [id=d059b7a1992a48de9b259e4d753263bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.421985 715 leader_election.cc:304] T 072bb7f0e4ab4935be49461bc8a3c163 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:49.422497 2633 raft_consensus.cc:2804] T 072bb7f0e4ab4935be49461bc8a3c163 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:49.422591 2633 raft_consensus.cc:493] T 072bb7f0e4ab4935be49461bc8a3c163 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:49.422657 2633 raft_consensus.cc:3060] T 072bb7f0e4ab4935be49461bc8a3c163 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:49.425189 1034 tablet_service.cc:1505] Processing CreateTablet for tablet 7612d648eb6546afb945d0ebfaa2f7ce (DEFAULT_TABLE table=default.Table_a0b796c134ec461ba052a12b3cc914d1 [id=981525cb58af4671ac28f451e77b1125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.425608 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7612d648eb6546afb945d0ebfaa2f7ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.426174 2633 raft_consensus.cc:515] T 072bb7f0e4ab4935be49461bc8a3c163 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.427426 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "072bb7f0e4ab4935be49461bc8a3c163" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:49.427608 1045 raft_consensus.cc:3060] T 072bb7f0e4ab4935be49461bc8a3c163 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:49.428628 2633 leader_election.cc:290] T 072bb7f0e4ab4935be49461bc8a3c163 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:49.428628 1027 tablet_service.cc:1505] Processing CreateTablet for tablet a5a7f612cde040df956bf6741f7f2760 (DEFAULT_TABLE table=default.Table_339b1866686f40e689308903d8ab370e [id=2b604f51980d4bc89ac378cb4054ac36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.429032 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5a7f612cde040df956bf6741f7f2760. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.432456 1045 raft_consensus.cc:2468] T 072bb7f0e4ab4935be49461bc8a3c163 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:49.432770 1035 tablet_service.cc:1505] Processing CreateTablet for tablet b955baff0bec484d90a1b43e117a6e20 (DEFAULT_TABLE table=default.Table_5d3d1a79e21c432bbe40c8639f375fb1 [id=eef70043f3454fdfbd2983e0efc5c9c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.433086 715 leader_election.cc:304] T 072bb7f0e4ab4935be49461bc8a3c163 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:49.433167 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b955baff0bec484d90a1b43e117a6e20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.433396 2633 raft_consensus.cc:2804] T 072bb7f0e4ab4935be49461bc8a3c163 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:49.433852 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 1eb0c7bd3ca34700aa8289a3775b7479 (DEFAULT_TABLE table=default.Table_99b0aec996a8431fba8dcd993bc5c846 [id=5df3be7b34be46978613d685b652d54b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.434194 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1eb0c7bd3ca34700aa8289a3775b7479. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:49.436147 423 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 52916us
I20260202 02:52:49.436641 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 3ee6153db90b43c5866a8bb09bcebcb3 (DEFAULT_TABLE table=default.Table_063766871b1745af8bf479effa86c7f1 [id=e31a5be325444a16af618dd5b9f99521]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.437023 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ee6153db90b43c5866a8bb09bcebcb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.437815 1036 tablet_service.cc:1505] Processing CreateTablet for tablet 0291c3e873bb4266ab040356fe35f4e9 (DEFAULT_TABLE table=default.Table_37ff6365170a49efb41f9a4d5db68e6a [id=4ba0056d876142b894e9fd0569636753]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.438182 1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0291c3e873bb4266ab040356fe35f4e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.441614 2633 raft_consensus.cc:697] T 072bb7f0e4ab4935be49461bc8a3c163 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:49.443992 2633 consensus_queue.cc:237] T 072bb7f0e4ab4935be49461bc8a3c163 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.445927 1036 tablet_service.cc:1505] Processing CreateTablet for tablet 08566164eec34cf792ea9281cfa3bb31 (DEFAULT_TABLE table=default.Table_fc9e04c1d0564a7896286c1a2d4f87a6 [id=0038f9c6f2d942d889c0feec728ec436]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.446537 1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08566164eec34cf792ea9281cfa3bb31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.450806 2619 raft_consensus.cc:493] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:49.450968 2619 raft_consensus.cc:515] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.451587 2619 leader_election.cc:290] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:49.453516 2482 raft_consensus.cc:493] T fbcb4a522df648ba9542250450c08f1b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:49.453663 2482 raft_consensus.cc:515] T fbcb4a522df648ba9542250450c08f1b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.454267 2482 leader_election.cc:290] T fbcb4a522df648ba9542250450c08f1b P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:49.454844 765 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fbcb4a522df648ba9542250450c08f1b" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:49.455150 999 leader_election.cc:343] T fbcb4a522df648ba9542250450c08f1b P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: fbcb4a522df648ba9542250450c08f1b
W20260202 02:52:49.455354 998 leader_election.cc:343] T fbcb4a522df648ba9542250450c08f1b P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:49.455453 998 leader_election.cc:304] T fbcb4a522df648ba9542250450c08f1b P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:49.455878 2482 raft_consensus.cc:2749] T fbcb4a522df648ba9542250450c08f1b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:49.456049 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 00a8c927e0344033a774a4db2cb97d5b (DEFAULT_TABLE table=default.Table_c8fe5bcb63164a1980d102e0374c1d0e [id=72d18bef7ba346ebb0ffabeeda66317d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.457952 2619 raft_consensus.cc:493] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:49.458097 2619 raft_consensus.cc:515] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.458755 2619 leader_election.cc:290] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:49.462402 1041 tablet_service.cc:1505] Processing CreateTablet for tablet 451bbf57e03a4562b1ae3b16ab5bfb4b (DEFAULT_TABLE table=default.Table_c8269ebb04c6479fa3643887f6f51f8d [id=f6a48bf5237a43d794d5038323a35bd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.466121 1033 tablet_service.cc:1505] Processing CreateTablet for tablet ad7c98afaa3447d1aac8a243c6a98ed6 (DEFAULT_TABLE table=default.Table_502b1ec344f64ad78b8613e843e3f73b [id=baeb590c153e4a79928e5e33161f4f19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.404433 2443 raft_consensus.cc:493] T 93f958924885457ab15f2aac5509e0aa P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:49.404752 2666 raft_consensus.cc:493] T 0138ba7326674741b0729324735f1de7 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:49.466734 1182 log.cc:1199] T 842e54adae2a452a98521de24326eaff P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/842e54adae2a452a98521de24326eaff
I20260202 02:52:49.414969 895 tablet_service.cc:1505] Processing CreateTablet for tablet 7612d648eb6546afb945d0ebfaa2f7ce (DEFAULT_TABLE table=default.Table_a0b796c134ec461ba052a12b3cc914d1 [id=981525cb58af4671ac28f451e77b1125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.467986 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7612d648eb6546afb945d0ebfaa2f7ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.468353 1182 ts_tablet_manager.cc:1950] T 842e54adae2a452a98521de24326eaff P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
W20260202 02:52:49.469102 519 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bb1005103c25423783736ace1866c328 consensus state has no leader
I20260202 02:52:49.471093 519 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet c8a273ac8cd344719a48fd81945bed0a (table default.Table_5a703dc84eb34674bdc5793ac7295098 [id=62c3c39fb73249349cee49b86d535612]). Expected version 2 got 0
I20260202 02:52:49.472143 519 master_service.cc:954] ReplaceTablet: received request to replace tablet c707180fc9c5457292a8f43bf676f4f9 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:49.418514 903 tablet_service.cc:1505] Processing CreateTablet for tablet 41f2a25a3ca84d84af18fcd8c58b6603 (DEFAULT_TABLE table=default.Table_274e129a05bd42a39cbee9731d492dac [id=d059b7a1992a48de9b259e4d753263bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.420935 893 tablet_service.cc:1505] Processing CreateTablet for tablet b955baff0bec484d90a1b43e117a6e20 (DEFAULT_TABLE table=default.Table_5d3d1a79e21c432bbe40c8639f375fb1 [id=eef70043f3454fdfbd2983e0efc5c9c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.424723 894 tablet_service.cc:1505] Processing CreateTablet for tablet a5a7f612cde040df956bf6741f7f2760 (DEFAULT_TABLE table=default.Table_339b1866686f40e689308903d8ab370e [id=2b604f51980d4bc89ac378cb4054ac36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.426918 892 tablet_service.cc:1505] Processing CreateTablet for tablet 1eb0c7bd3ca34700aa8289a3775b7479 (DEFAULT_TABLE table=default.Table_99b0aec996a8431fba8dcd993bc5c846 [id=5df3be7b34be46978613d685b652d54b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.434921 901 tablet_service.cc:1505] Processing CreateTablet for tablet 3ee6153db90b43c5866a8bb09bcebcb3 (DEFAULT_TABLE table=default.Table_063766871b1745af8bf479effa86c7f1 [id=e31a5be325444a16af618dd5b9f99521]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:49.472594 527 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 67a8c9a1998b47b3b214a06ce62381e9 consensus state has no leader
I20260202 02:52:49.472688 903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41f2a25a3ca84d84af18fcd8c58b6603. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.436618 889 tablet_service.cc:1505] Processing CreateTablet for tablet 0291c3e873bb4266ab040356fe35f4e9 (DEFAULT_TABLE table=default.Table_37ff6365170a49efb41f9a4d5db68e6a [id=4ba0056d876142b894e9fd0569636753]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.445426 887 tablet_service.cc:1505] Processing CreateTablet for tablet 08566164eec34cf792ea9281cfa3bb31 (DEFAULT_TABLE table=default.Table_fc9e04c1d0564a7896286c1a2d4f87a6 [id=0038f9c6f2d942d889c0feec728ec436]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.453037 899 tablet_service.cc:1505] Processing CreateTablet for tablet 00a8c927e0344033a774a4db2cb97d5b (DEFAULT_TABLE table=default.Table_c8fe5bcb63164a1980d102e0374c1d0e [id=72d18bef7ba346ebb0ffabeeda66317d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.473263 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 842e54adae2a452a98521de24326eaff (table default.Table_c6e682a233904983978e7abb23565e41 [id=0d7624b55c1a4bf0a449f651ac419d77]) successfully deleted
I20260202 02:52:49.459425 898 tablet_service.cc:1505] Processing CreateTablet for tablet 451bbf57e03a4562b1ae3b16ab5bfb4b (DEFAULT_TABLE table=default.Table_c8269ebb04c6479fa3643887f6f51f8d [id=f6a48bf5237a43d794d5038323a35bd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.461357 896 tablet_service.cc:1505] Processing CreateTablet for tablet ad7c98afaa3447d1aac8a243c6a98ed6 (DEFAULT_TABLE table=default.Table_502b1ec344f64ad78b8613e843e3f73b [id=baeb590c153e4a79928e5e33161f4f19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.473613 891 tablet_service.cc:1505] Processing CreateTablet for tablet 35c1f0e54f39426885effbaae428cce7 (DEFAULT_TABLE table=default.Table_c00e5644e8f444a696c11cdb71e4e698 [id=f946125d3edf4c03ad0e6dc275ca00ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.473848 1180 ts_tablet_manager.cc:1929] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:49.473861 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 59ad016e15ec4a488921eb564029b689 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:21 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:49.473975 1180 log.cc:1199] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/8774c6bc0fe9414ea2f17ed859d9392a
I20260202 02:52:49.474257 1180 ts_tablet_manager.cc:1950] T 8774c6bc0fe9414ea2f17ed859d9392a P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:49.474735 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1eb0c7bd3ca34700aa8289a3775b7479. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.474961 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5a7f612cde040df956bf6741f7f2760. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.475203 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ee6153db90b43c5866a8bb09bcebcb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.478202 1040 tablet_service.cc:1552] Processing DeleteTablet for tablet 072bb7f0e4ab4935be49461bc8a3c163 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:49.478967 1038 tablet_service.cc:1505] Processing CreateTablet for tablet 35c1f0e54f39426885effbaae428cce7 (DEFAULT_TABLE table=default.Table_c00e5644e8f444a696c11cdb71e4e698 [id=f946125d3edf4c03ad0e6dc275ca00ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.485548 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1116872cc25a4ab5b1a3a6bbba1f0da7" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:49.485778 1048 raft_consensus.cc:2468] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:49.486289 715 leader_election.cc:304] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:49.486657 2619 raft_consensus.cc:2804] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:49.486742 2619 raft_consensus.cc:493] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:49.486804 2619 raft_consensus.cc:3060] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:49.479570 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0291c3e873bb4266ab040356fe35f4e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.487214 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35c1f0e54f39426885effbaae428cce7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.487219 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad7c98afaa3447d1aac8a243c6a98ed6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.488461 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08566164eec34cf792ea9281cfa3bb31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.489324 2619 raft_consensus.cc:515] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.490207 2619 leader_election.cc:290] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:49.490875 1053 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1116872cc25a4ab5b1a3a6bbba1f0da7" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:49.491070 1053 raft_consensus.cc:3060] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:49.491078 2619 consensus_queue.cc:1048] T 33150e1456144b8e9c64cc2a2ac6dcca P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:49.493307 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 451bbf57e03a4562b1ae3b16ab5bfb4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.493551 1053 raft_consensus.cc:2468] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:49.494026 715 leader_election.cc:304] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:49.494119 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad7c98afaa3447d1aac8a243c6a98ed6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.494309 2633 raft_consensus.cc:2804] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:49.494418 2633 raft_consensus.cc:697] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:49.494803 2633 consensus_queue.cc:237] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.497351 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00a8c927e0344033a774a4db2cb97d5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.499159 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00a8c927e0344033a774a4db2cb97d5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.500597 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b955baff0bec484d90a1b43e117a6e20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.500698 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 451bbf57e03a4562b1ae3b16ab5bfb4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.506242 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f94f25f4059a43b8babedb6b45a08a19" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:49.510890 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41f2a25a3ca84d84af18fcd8c58b6603. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.512462 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35c1f0e54f39426885effbaae428cce7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.517210 2789 consensus_queue.cc:1048] T 6b8f924aecf44e959e47d2f32e6b0bf5 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:49.475270 1182 tablet_replica.cc:333] T f94f25f4059a43b8babedb6b45a08a19 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:49.519129 923 raft_consensus.cc:2468] T f94f25f4059a43b8babedb6b45a08a19 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:49.519464 1182 raft_consensus.cc:2243] T f94f25f4059a43b8babedb6b45a08a19 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:49.519595 1182 raft_consensus.cc:2272] T f94f25f4059a43b8babedb6b45a08a19 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:49.520081 1182 ts_tablet_manager.cc:1916] T f94f25f4059a43b8babedb6b45a08a19 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:49.520577 527 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_7a314585590647ca8fd0f51e125b0622" } modify_external_catalogs: true
I20260202 02:52:49.520784 527 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_7a314585590647ca8fd0f51e125b0622" } modify_external_catalogs: true
I20260202 02:52:49.522413 1882 ts_tablet_manager.cc:1929] T dada209d982d48f3b74a49905f19c5b6 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:49.522526 1882 log.cc:1199] T dada209d982d48f3b74a49905f19c5b6 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/dada209d982d48f3b74a49905f19c5b6
I20260202 02:52:49.522818 1882 ts_tablet_manager.cc:1950] T dada209d982d48f3b74a49905f19c5b6 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:49.523531 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 711) took 18907 ms. Trace:
I20260202 02:52:49.523609 1882 rpcz_store.cc:276] 0202 02:52:30.616558 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:35.770517 (+5153959us) service_pool.cc:225] Handling call
0202 02:52:49.391251 (+13620734us) ts_tablet_manager.cc:1151] Deleting tablet dada209d982d48f3b74a49905f19c5b6
0202 02:52:49.391264 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.395228 (+ 3964us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.522267 (+127039us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.523512 (+ 1245us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":13620630}
I20260202 02:52:49.523959 1882 tablet_replica.cc:333] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:49.528718 741 tablet_service.cc:1505] Processing CreateTablet for tablet 477c599f8c9b4eb0b435ea50abd5d36b (DEFAULT_TABLE table=default.Table_f7636b54f7f34f2a9487c03e57a621a9 [id=68bb32ad1fd34ab28681b80cccea3f3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.529160 764 raft_consensus.cc:1240] T f6746ccbf37e477196a9706396d1e3fd P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Rejecting Update request from peer b1f05820b6034228bdad884cc348716d for earlier term 1. Current term is 2. Ops: []
I20260202 02:52:49.529518 744 tablet_service.cc:1505] Processing CreateTablet for tablet 7612d648eb6546afb945d0ebfaa2f7ce (DEFAULT_TABLE table=default.Table_a0b796c134ec461ba052a12b3cc914d1 [id=981525cb58af4671ac28f451e77b1125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.529874 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 477c599f8c9b4eb0b435ea50abd5d36b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.530113 2789 raft_consensus.cc:493] T 65bd1c1a382e42e4a68077264563c55b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:49.530234 2789 raft_consensus.cc:515] T 65bd1c1a382e42e4a68077264563c55b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:49.531072 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 643) took 21041 ms. Trace:
I20260202 02:52:49.531296 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet dada209d982d48f3b74a49905f19c5b6 (table default.Table_1fa8930369e844ca8384126bf8f22e66 [id=116247212fca4c0a95ad1bdbbac4787f]) successfully deleted
I20260202 02:52:49.533136 2632 consensus_queue.cc:1059] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260202 02:52:49.534299 1058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "65bd1c1a382e42e4a68077264563c55b" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:49.534839 859 leader_election.cc:343] T 65bd1c1a382e42e4a68077264563c55b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:49.532585 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7612d648eb6546afb945d0ebfaa2f7ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.536203 1180 rpcz_store.cc:276] 0202 02:52:28.489654 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:28.490732 (+ 1078us) service_pool.cc:225] Handling call
0202 02:52:49.330552 (+20839820us) ts_tablet_manager.cc:1151] Deleting tablet 8774c6bc0fe9414ea2f17ed859d9392a
0202 02:52:49.330560 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.378401 (+ 47841us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.473707 (+ 95306us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.531051 (+ 57344us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":20835483}
I20260202 02:52:49.534757 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_dd3beb3a0d7147db9f5e8739810746df"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:49.537192 1180 tablet_replica.cc:333] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d: stopping tablet replica
W20260202 02:52:49.537287 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dd3beb3a0d7147db9f5e8739810746df in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:49.537324 2789 leader_election.cc:290] T 65bd1c1a382e42e4a68077264563c55b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:49.538201 1180 raft_consensus.cc:2243] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:49.539805 1180 raft_consensus.cc:2272] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:49.539330 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f94f25f4059a43b8babedb6b45a08a19" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:49.541088 1045 raft_consensus.cc:2468] T f94f25f4059a43b8babedb6b45a08a19 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:49.542143 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_988dd0b38bd141bf8cba853c4d6685b0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:49.542774 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_988dd0b38bd141bf8cba853c4d6685b0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:49.544016 528 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 6750a1212ceb4e6e82e4ffa93caab590
I20260202 02:52:49.544840 528 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ae634747dc6e4a789c36df9b29d861c0" } modify_external_catalogs: true
I20260202 02:52:49.545015 528 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ae634747dc6e4a789c36df9b29d861c0" } modify_external_catalogs: true
I20260202 02:52:49.545513 526 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet c03a12f911084222be15ea6f482ebe32
I20260202 02:52:49.546157 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_9e2f89fa77414b56a2ebd040c0e80085"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:49.546686 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9e2f89fa77414b56a2ebd040c0e80085 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:49.547701 765 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "65bd1c1a382e42e4a68077264563c55b" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:49.547878 523 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 4482b24826884d0da0d34f43f69e1646
I20260202 02:52:49.548517 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_5d897c15ea4046ac9869bce1c15a1445"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:49.548628 1180 ts_tablet_manager.cc:1916] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:49.548617 857 leader_election.cc:343] T 65bd1c1a382e42e4a68077264563c55b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:49.548736 857 leader_election.cc:304] T 65bd1c1a382e42e4a68077264563c55b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:49.548743 715 leader_election.cc:304] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:49.548978 2667 raft_consensus.cc:2804] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:49.549044 2667 raft_consensus.cc:493] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:49.549103 2667 raft_consensus.cc:3060] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:49.549093 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5d897c15ea4046ac9869bce1c15a1445 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:49.550263 520 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 00f2d3bafb2d472aaffc0a7ba926c825
I20260202 02:52:49.550675 520 catalog_manager.cc:6724] ReplaceTablet: tablet 00f2d3bafb2d472aaffc0a7ba926c825 deleted and replaced by tablet 40bb2baffa53450683e26e816cf3e11e
I20260202 02:52:49.551035 520 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_a77859a40f12489a8578b7eb1a66f0d4" } new_table_name: "default.Table_4062ed5efe774bc2b380c3ae7fa53abe" modify_external_catalogs: true
I20260202 02:52:49.551602 2667 raft_consensus.cc:515] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.552232 2667 leader_election.cc:290] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:49.552547 757 tablet_service.cc:1505] Processing CreateTablet for tablet a5a7f612cde040df956bf6741f7f2760 (DEFAULT_TABLE table=default.Table_339b1866686f40e689308903d8ab370e [id=2b604f51980d4bc89ac378cb4054ac36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.552990 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5a7f612cde040df956bf6741f7f2760. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.553658 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet c03a12f911084222be15ea6f482ebe32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:49.554255 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet 4482b24826884d0da0d34f43f69e1646 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:49.554453 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet 00f2d3bafb2d472aaffc0a7ba926c825 with delete_type TABLET_DATA_DELETED (replaced by tablet 40bb2baffa53450683e26e816cf3e11e) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:49.559540 744 tablet_service.cc:1505] Processing CreateTablet for tablet b955baff0bec484d90a1b43e117a6e20 (DEFAULT_TABLE table=default.Table_5d3d1a79e21c432bbe40c8639f375fb1 [id=eef70043f3454fdfbd2983e0efc5c9c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.559979 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b955baff0bec484d90a1b43e117a6e20. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:49.561312 2650 lock_manager.cc:585] Waited 1 seconds to obtain row lock on key '<redacted>' (key hash 9839659925192154851) tablet 00000000000000000000000000000000 cur holder 0x617000052b80
I20260202 02:52:49.562373 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_e28fec940ad64825a9e8eeece80cce05"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:49.563287 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e28fec940ad64825a9e8eeece80cce05 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:49.564281 1182 ts_tablet_manager.cc:1929] T f94f25f4059a43b8babedb6b45a08a19 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:49.564425 1182 log.cc:1199] T f94f25f4059a43b8babedb6b45a08a19 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/f94f25f4059a43b8babedb6b45a08a19
I20260202 02:52:49.564594 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f94f25f4059a43b8babedb6b45a08a19" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:49.564739 1182 ts_tablet_manager.cc:1950] T f94f25f4059a43b8babedb6b45a08a19 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:49.564719 741 tablet_service.cc:1505] Processing CreateTablet for tablet 41f2a25a3ca84d84af18fcd8c58b6603 (DEFAULT_TABLE table=default.Table_274e129a05bd42a39cbee9731d492dac [id=d059b7a1992a48de9b259e4d753263bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.564781 1045 raft_consensus.cc:3060] T f94f25f4059a43b8babedb6b45a08a19 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:49.564867 524 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_5538c7d2137648efb81b8057751d9972" } new_table_name: "default.Table_f4ed15aa08c44cb29cf820bbbac9c6df" modify_external_catalogs: true
I20260202 02:52:49.565092 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41f2a25a3ca84d84af18fcd8c58b6603. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.565764 1182 tablet_replica.cc:333] T 0138ba7326674741b0729324735f1de7 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:49.566069 519 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet c707180fc9c5457292a8f43bf676f4f9
I20260202 02:52:49.566506 519 catalog_manager.cc:6724] ReplaceTablet: tablet c707180fc9c5457292a8f43bf676f4f9 deleted and replaced by tablet 881abac44f1c47d2a4193555a22be250
I20260202 02:52:49.567468 1045 raft_consensus.cc:2468] T f94f25f4059a43b8babedb6b45a08a19 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:49.568660 527 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 8bb14bf551f441dcbaaff7057b51b2be
I20260202 02:52:49.569689 757 tablet_service.cc:1505] Processing CreateTablet for tablet 1eb0c7bd3ca34700aa8289a3775b7479 (DEFAULT_TABLE table=default.Table_99b0aec996a8431fba8dcd993bc5c846 [id=5df3be7b34be46978613d685b652d54b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.569913 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet c707180fc9c5457292a8f43bf676f4f9 with delete_type TABLET_DATA_DELETED (replaced by tablet 881abac44f1c47d2a4193555a22be250) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:49.570201 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1eb0c7bd3ca34700aa8289a3775b7479. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.570374 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 8bb14bf551f441dcbaaff7057b51b2be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:49.571167 715 leader_election.cc:304] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:49.571465 2667 raft_consensus.cc:2804] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:49.571579 2667 raft_consensus.cc:697] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:49.571833 2667 consensus_queue.cc:237] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.572924 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet f94f25f4059a43b8babedb6b45a08a19 (table default.Table_22445ccfd7b64298b92a0f70733f3625 [id=300ea019d2724bee8211d7462710dc26]) successfully deleted
W20260202 02:52:49.573189 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet abfe7f76b2424c4d9ab07b29b66ec103 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet abfe7f76b2424c4d9ab07b29b66ec103 already in progress: creating tablet
W20260202 02:52:49.573916 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet dd8f36d7a82f47adae1bd3c1f8628b6e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dd8f36d7a82f47adae1bd3c1f8628b6e already in progress: creating tablet
W20260202 02:52:49.574085 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 3338964dfed548f980f58147f0283691 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3338964dfed548f980f58147f0283691 already in progress: creating tablet
W20260202 02:52:49.574285 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet d594295561ff4b01bf13451533a5743c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d594295561ff4b01bf13451533a5743c already in progress: creating tablet
I20260202 02:52:49.574954 899 tablet_service.cc:1552] Processing DeleteTablet for tablet c03a12f911084222be15ea6f482ebe32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:49.575856 899 tablet_service.cc:1552] Processing DeleteTablet for tablet 4482b24826884d0da0d34f43f69e1646 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:49.576689 899 tablet_service.cc:1552] Processing DeleteTablet for tablet 00f2d3bafb2d472aaffc0a7ba926c825 with delete_type TABLET_DATA_DELETED (replaced by tablet 40bb2baffa53450683e26e816cf3e11e) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:49.578482 1173 tablet_bootstrap.cc:492] T 6df3d11cf578404ca41807f0777b6c53 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:49.578625 1173 ts_tablet_manager.cc:1403] T 6df3d11cf578404ca41807f0777b6c53 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.222s user 0.004s sys 0.025s
I20260202 02:52:49.579263 1033 tablet_service.cc:1552] Processing DeleteTablet for tablet 6750a1212ceb4e6e82e4ffa93caab590 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:49.581753 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 6750a1212ceb4e6e82e4ffa93caab590 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:49.583801 1180 ts_tablet_manager.cc:1929] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:49.583940 1180 log.cc:1199] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/68823eff1bc04108ba83487605aec0ea
I20260202 02:52:49.584237 1180 ts_tablet_manager.cc:1950] T 68823eff1bc04108ba83487605aec0ea P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:49.586319 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 644) took 21096 ms. Trace:
I20260202 02:52:49.588272 750 tablet_service.cc:1505] Processing CreateTablet for tablet 3ee6153db90b43c5866a8bb09bcebcb3 (DEFAULT_TABLE table=default.Table_063766871b1745af8bf479effa86c7f1 [id=e31a5be325444a16af618dd5b9f99521]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.588981 753 tablet_service.cc:1505] Processing CreateTablet for tablet 0291c3e873bb4266ab040356fe35f4e9 (DEFAULT_TABLE table=default.Table_37ff6365170a49efb41f9a4d5db68e6a [id=4ba0056d876142b894e9fd0569636753]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.592567 1180 rpcz_store.cc:276] 0202 02:52:28.489747 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:28.491305 (+ 1558us) service_pool.cc:225] Handling call
0202 02:52:49.537138 (+21045833us) ts_tablet_manager.cc:1151] Deleting tablet 68823eff1bc04108ba83487605aec0ea
0202 02:52:49.537149 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.551681 (+ 14532us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.583669 (+ 31988us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.586292 (+ 2623us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":21037595}
I20260202 02:52:49.589471 754 tablet_service.cc:1505] Processing CreateTablet for tablet 00a8c927e0344033a774a4db2cb97d5b (DEFAULT_TABLE table=default.Table_c8fe5bcb63164a1980d102e0374c1d0e [id=72d18bef7ba346ebb0ffabeeda66317d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.589941 756 tablet_service.cc:1505] Processing CreateTablet for tablet 451bbf57e03a4562b1ae3b16ab5bfb4b (DEFAULT_TABLE table=default.Table_c8269ebb04c6479fa3643887f6f51f8d [id=f6a48bf5237a43d794d5038323a35bd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.590392 742 tablet_service.cc:1505] Processing CreateTablet for tablet ad7c98afaa3447d1aac8a243c6a98ed6 (DEFAULT_TABLE table=default.Table_502b1ec344f64ad78b8613e843e3f73b [id=baeb590c153e4a79928e5e33161f4f19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.590811 751 tablet_service.cc:1505] Processing CreateTablet for tablet 35c1f0e54f39426885effbaae428cce7 (DEFAULT_TABLE table=default.Table_c00e5644e8f444a696c11cdb71e4e698 [id=f946125d3edf4c03ad0e6dc275ca00ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.593542 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 647) took 21010 ms. Trace:
I20260202 02:52:49.591284 740 tablet_service.cc:1505] Processing CreateTablet for tablet 137dae39f8f54be5af6354e2b6226adc (DEFAULT_TABLE table=default.Table_0c357b5eb1e14dbba252f4edfc630705 [id=64eea8b3e2d34698ad2dbe65d87d29f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.591604 755 tablet_service.cc:1552] Processing DeleteTablet for tablet c03a12f911084222be15ea6f482ebe32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:49.591697 747 tablet_service.cc:1552] Processing DeleteTablet for tablet 00f2d3bafb2d472aaffc0a7ba926c825 with delete_type TABLET_DATA_DELETED (replaced by tablet 40bb2baffa53450683e26e816cf3e11e) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:49.591727 745 tablet_service.cc:1552] Processing DeleteTablet for tablet 4482b24826884d0da0d34f43f69e1646 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:49.591976 759 tablet_service.cc:1552] Processing DeleteTablet for tablet b7880cb051c24bc998003f0effed3cf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:49.593837 1180 rpcz_store.cc:276] 0202 02:52:28.582958 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:28.583029 (+ 71us) service_pool.cc:225] Handling call
0202 02:52:49.593217 (+21010188us) ts_tablet_manager.cc:1151] Deleting tablet 47edd9f65ef343b9a3be74c30708daca
0202 02:52:49.593223 (+ 6us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.593524 (+ 301us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":21009982}
I20260202 02:52:49.592213 746 tablet_service.cc:1552] Processing DeleteTablet for tablet 1116872cc25a4ab5b1a3a6bbba1f0da7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:25 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:49.592504 749 tablet_service.cc:1552] Processing DeleteTablet for tablet c707180fc9c5457292a8f43bf676f4f9 with delete_type TABLET_DATA_DELETED (replaced by tablet 881abac44f1c47d2a4193555a22be250) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:49.592908 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ee6153db90b43c5866a8bb09bcebcb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.588999 758 tablet_service.cc:1505] Processing CreateTablet for tablet 08566164eec34cf792ea9281cfa3bb31 (DEFAULT_TABLE table=default.Table_fc9e04c1d0564a7896286c1a2d4f87a6 [id=0038f9c6f2d942d889c0feec728ec436]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.591236 748 tablet_service.cc:1552] Processing DeleteTablet for tablet 6750a1212ceb4e6e82e4ffa93caab590 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:49.594694 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00a8c927e0344033a774a4db2cb97d5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.594712 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 646) took 21011 ms. Trace:
I20260202 02:52:49.595014 1180 rpcz_store.cc:276] 0202 02:52:28.582873 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:28.583105 (+ 232us) service_pool.cc:225] Handling call
0202 02:52:49.594522 (+21011417us) ts_tablet_manager.cc:1151] Deleting tablet 83d198d90a204c6396b569fc1b65abef
0202 02:52:49.594534 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.594695 (+ 161us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":21011155}
I20260202 02:52:49.598760 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad7c98afaa3447d1aac8a243c6a98ed6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.598810 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 648) took 21015 ms. Trace:
I20260202 02:52:49.598939 1180 rpcz_store.cc:276] 0202 02:52:28.583050 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:28.583097 (+ 47us) service_pool.cc:225] Handling call
0202 02:52:49.595437 (+21012340us) ts_tablet_manager.cc:1151] Deleting tablet b05008bde58d47f7adaaf35ac5949bdc
0202 02:52:49.598654 (+ 3217us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.598786 (+ 132us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":21012082}
I20260202 02:52:49.599301 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 137dae39f8f54be5af6354e2b6226adc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.599454 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 645) took 21016 ms. Trace:
I20260202 02:52:49.599541 1180 rpcz_store.cc:276] 0202 02:52:28.582759 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:28.583988 (+ 1229us) service_pool.cc:225] Handling call
0202 02:52:49.599304 (+21015316us) ts_tablet_manager.cc:1151] Deleting tablet 4ec86e245f8c4d9e87c3ee3409a164b3
0202 02:52:49.599314 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.599437 (+ 123us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":21015097}
I20260202 02:52:49.599884 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 653) took 20757 ms. Trace:
I20260202 02:52:49.600061 1180 rpcz_store.cc:276] 0202 02:52:28.842301 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:28.843925 (+ 1624us) service_pool.cc:225] Handling call
0202 02:52:49.599786 (+20755861us) ts_tablet_manager.cc:1151] Deleting tablet 6a3dac5cd1ce47488a50b4c32eca67c7
0202 02:52:49.599795 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.599871 (+ 76us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":20743605}
I20260202 02:52:49.600360 1180 tablet_replica.cc:333] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:49.600593 1180 raft_consensus.cc:2243] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:49.600888 1180 pending_rounds.cc:70] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d: Trying to abort 1 pending ops.
I20260202 02:52:49.600960 1180 pending_rounds.cc:77] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923067651256320 op_type: NO_OP noop_request { }
I20260202 02:52:49.601060 1180 raft_consensus.cc:2889] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:49.601147 1180 raft_consensus.cc:2272] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:49.601699 1180 ts_tablet_manager.cc:1916] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:49.602262 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08566164eec34cf792ea9281cfa3bb31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.604642 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 451bbf57e03a4562b1ae3b16ab5bfb4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.604930 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0291c3e873bb4266ab040356fe35f4e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.605103 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35c1f0e54f39426885effbaae428cce7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.609433 523 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_2679d310b29046beb6169593bb96e42f" } new_table_name: "default.Table_de1ca8a99f3d4948b4cb7081e782ebd7" modify_external_catalogs: true
I20260202 02:52:49.611517 1180 ts_tablet_manager.cc:1929] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:49.612349 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_ebd39aa54af54c0e9b4efe9b5d986034"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:49.613049 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ebd39aa54af54c0e9b4efe9b5d986034 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:49.613907 1180 log.cc:1199] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/9c1271ce0f194021b588b82ac9d3db79
I20260202 02:52:49.614408 1180 ts_tablet_manager.cc:1950] T 9c1271ce0f194021b588b82ac9d3db79 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:49.620097 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_2fb99cc8383241959a5a1d845f684a98"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:49.620870 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2fb99cc8383241959a5a1d845f684a98 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:49.622480 520 master_service.cc:954] ReplaceTablet: received request to replace tablet 42321f4e77484027a485fdd584de3ae3 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:49.629271 2784 raft_consensus.cc:2749] T 65bd1c1a382e42e4a68077264563c55b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:49.579262 1173 raft_consensus.cc:359] T 6df3d11cf578404ca41807f0777b6c53 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:49.634295 423 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 57935.3us
I20260202 02:52:49.634760 899 tablet_service.cc:1552] Processing DeleteTablet for tablet c707180fc9c5457292a8f43bf676f4f9 with delete_type TABLET_DATA_DELETED (replaced by tablet 881abac44f1c47d2a4193555a22be250) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:49.464430 2786 consensus_queue.cc:1048] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:49.414969 886 tablet_service.cc:1505] Processing CreateTablet for tablet 477c599f8c9b4eb0b435ea50abd5d36b (DEFAULT_TABLE table=default.Table_f7636b54f7f34f2a9487c03e57a621a9 [id=68bb32ad1fd34ab28681b80cccea3f3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.466857 2666 raft_consensus.cc:515] T 0138ba7326674741b0729324735f1de7 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:49.635915 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 477c599f8c9b4eb0b435ea50abd5d36b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.636102 2666 leader_election.cc:290] T 0138ba7326674741b0729324735f1de7 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:49.466756 2443 raft_consensus.cc:515] T 93f958924885457ab15f2aac5509e0aa P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:49.636642 2443 leader_election.cc:290] T 93f958924885457ab15f2aac5509e0aa P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:49.636669 1182 raft_consensus.cc:2243] T 0138ba7326674741b0729324735f1de7 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:49.637045 1182 raft_consensus.cc:2272] T 0138ba7326674741b0729324735f1de7 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:49.635463 899 tablet_service.cc:1552] Processing DeleteTablet for tablet 8bb14bf551f441dcbaaff7057b51b2be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:49.637594 1182 ts_tablet_manager.cc:1916] T 0138ba7326674741b0729324735f1de7 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:49.638118 765 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0138ba7326674741b0729324735f1de7" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:49.638510 2443 raft_consensus.cc:493] T da5435f969074364bc8f8a6ddc5e4f8a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:49.638622 2443 raft_consensus.cc:515] T da5435f969074364bc8f8a6ddc5e4f8a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:49.639127 2443 leader_election.cc:290] T da5435f969074364bc8f8a6ddc5e4f8a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:49.640170 765 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "93f958924885457ab15f2aac5509e0aa" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:49.641019 765 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "da5435f969074364bc8f8a6ddc5e4f8a" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:49.641516 857 leader_election.cc:343] T 0138ba7326674741b0729324735f1de7 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:49.641724 857 leader_election.cc:343] T 93f958924885457ab15f2aac5509e0aa P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:49.641875 857 leader_election.cc:343] T da5435f969074364bc8f8a6ddc5e4f8a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:49.640534 1054 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "da5435f969074364bc8f8a6ddc5e4f8a" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:49.641549 1058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "93f958924885457ab15f2aac5509e0aa" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:49.645741 1173 raft_consensus.cc:385] T 6df3d11cf578404ca41807f0777b6c53 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:49.645843 1173 raft_consensus.cc:740] T 6df3d11cf578404ca41807f0777b6c53 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
W20260202 02:52:49.646126 859 leader_election.cc:343] T da5435f969074364bc8f8a6ddc5e4f8a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:49.646237 859 leader_election.cc:304] T da5435f969074364bc8f8a6ddc5e4f8a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
W20260202 02:52:49.646771 859 leader_election.cc:343] T 93f958924885457ab15f2aac5509e0aa P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:49.646842 1182 ts_tablet_manager.cc:1929] T 0138ba7326674741b0729324735f1de7 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:49.646886 859 leader_election.cc:304] T 93f958924885457ab15f2aac5509e0aa P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:49.646935 1182 log.cc:1199] T 0138ba7326674741b0729324735f1de7 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/0138ba7326674741b0729324735f1de7
I20260202 02:52:49.647250 1182 ts_tablet_manager.cc:1950] T 0138ba7326674741b0729324735f1de7 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:49.647325 2443 raft_consensus.cc:2749] T da5435f969074364bc8f8a6ddc5e4f8a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:49.647467 2443 raft_consensus.cc:2749] T 93f958924885457ab15f2aac5509e0aa P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:49.647832 1182 tablet_replica.cc:333] T 93f958924885457ab15f2aac5509e0aa P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:49.648008 1182 raft_consensus.cc:2243] T 93f958924885457ab15f2aac5509e0aa P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:49.648139 1182 raft_consensus.cc:2272] T 93f958924885457ab15f2aac5509e0aa P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:49.648653 1182 ts_tablet_manager.cc:1916] T 93f958924885457ab15f2aac5509e0aa P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:49.642004 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0138ba7326674741b0729324735f1de7" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:49.650064 859 leader_election.cc:343] T 0138ba7326674741b0729324735f1de7 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:49.650177 859 leader_election.cc:304] T 0138ba7326674741b0729324735f1de7 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
W20260202 02:52:49.650333 859 raft_consensus.cc:2686] T 0138ba7326674741b0729324735f1de7 P 1521a6d262b7451bbef8debe970dd6b5: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260202 02:52:49.654961 524 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 455e7d1197c144eda1cf1ef405aeb262 consensus state has no leader
I20260202 02:52:49.655428 528 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet 881abac44f1c47d2a4193555a22be250
I20260202 02:52:49.655426 1182 ts_tablet_manager.cc:1929] T 93f958924885457ab15f2aac5509e0aa P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260202 02:52:49.654963 523 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 258e0afdba894f3cb2a3c7238e5634b7 consensus state has no leader
I20260202 02:52:49.657526 2797 raft_consensus.cc:493] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:49.657694 2797 raft_consensus.cc:515] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.658293 2797 leader_election.cc:290] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:49.658425 2797 raft_consensus.cc:493] T ccd6bc8bd0504bf18d108c73685ef9ff P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:49.659035 1182 log.cc:1199] T 93f958924885457ab15f2aac5509e0aa P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/93f958924885457ab15f2aac5509e0aa
I20260202 02:52:49.659673 1182 ts_tablet_manager.cc:1950] T 93f958924885457ab15f2aac5509e0aa P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:49.661188 1182 tablet_replica.cc:333] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:49.662068 1173 consensus_queue.cc:260] T 6df3d11cf578404ca41807f0777b6c53 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.662704 1173 ts_tablet_manager.cc:1434] T 6df3d11cf578404ca41807f0777b6c53 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.084s user 0.001s sys 0.001s
I20260202 02:52:49.662720 1182 raft_consensus.cc:2243] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:49.663113 1173 tablet_bootstrap.cc:492] T a0b5d82e5fb94b60b4aa324b978a5378 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:49.663584 1182 raft_consensus.cc:2272] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:49.664428 1182 ts_tablet_manager.cc:1916] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:49.665009 1173 tablet_bootstrap.cc:654] T a0b5d82e5fb94b60b4aa324b978a5378 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:49.682199 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_c0f4096c1e384c37b990a33d78d9cfc0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:49.682842 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c0f4096c1e384c37b990a33d78d9cfc0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:49.685106 526 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_a8324bdbd82941c8b8e035d01c6997a6" } new_table_name: "default.Table_619bb68d71834b3a9f5e0b2b0fdaed09" modify_external_catalogs: true
I20260202 02:52:49.688587 743 tablet_service.cc:1552] Processing DeleteTablet for tablet 8bb14bf551f441dcbaaff7057b51b2be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:49.688810 523 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_4d0600d5e4824d48ae2a8aabcb4b486b" } new_table_name: "default.Table_756b4deb63c645e79a6fbb8f8f131ee0" modify_external_catalogs: true
I20260202 02:52:49.694024 743 tablet_service.cc:1505] Processing CreateTablet for tablet f90a66147b1e417e8739dfd8faca034e (DEFAULT_TABLE table=default.Table_2d8e70c77d2e4bbe994ff7493c108625 [id=99d756cd7e01458b8869b01866861f55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.694881 743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f90a66147b1e417e8739dfd8faca034e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.702849 520 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 42321f4e77484027a485fdd584de3ae3
I20260202 02:52:49.703341 520 catalog_manager.cc:6724] ReplaceTablet: tablet 42321f4e77484027a485fdd584de3ae3 deleted and replaced by tablet fea0fcb8854c4fc6acda80c8ffb8d341
I20260202 02:52:49.703874 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 0138ba7326674741b0729324735f1de7 (table default.Table_516d610640654ea4b49f7de3b04cbb0c [id=5c8367fbb78840d18a26a026d75c3d6b]) successfully deleted
W20260202 02:52:49.704106 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 118753e8cb344b47a35a745aa8fed265 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 118753e8cb344b47a35a745aa8fed265 already in progress: creating tablet
I20260202 02:52:49.704253 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 93f958924885457ab15f2aac5509e0aa (table default.Table_2e2049e67b8943acab5a0db3deb1535a [id=0db38635f6af4f168f221f4749adf0dd]) successfully deleted
W20260202 02:52:49.704466 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 2d18485f5a4a4b11916c64a487cd28b4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2d18485f5a4a4b11916c64a487cd28b4 already in progress: creating tablet
W20260202 02:52:49.704649 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 48863e2d60c84460858daabb9a08dcd2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 48863e2d60c84460858daabb9a08dcd2 already in progress: creating tablet
I20260202 02:52:49.704633 886 tablet_service.cc:1505] Processing CreateTablet for tablet 137dae39f8f54be5af6354e2b6226adc (DEFAULT_TABLE table=default.Table_0c357b5eb1e14dbba252f4edfc630705 [id=64eea8b3e2d34698ad2dbe65d87d29f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.704634 899 tablet_service.cc:1505] Processing CreateTablet for tablet f90a66147b1e417e8739dfd8faca034e (DEFAULT_TABLE table=default.Table_2d8e70c77d2e4bbe994ff7493c108625 [id=99d756cd7e01458b8869b01866861f55]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:49.704854 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet e07e87c0182b46a49b8ca64c072f4210 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e07e87c0182b46a49b8ca64c072f4210 already in progress: creating tablet
W20260202 02:52:49.705068 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet f87ee8d6ff9042c08ba2e32724bea720 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f87ee8d6ff9042c08ba2e32724bea720 already in progress: creating tablet
I20260202 02:52:49.705137 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 137dae39f8f54be5af6354e2b6226adc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.705331 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f90a66147b1e417e8739dfd8faca034e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.706931 891 tablet_service.cc:1505] Processing CreateTablet for tablet 6c4f969e31214b6ab61328ac2e8413b6 (DEFAULT_TABLE table=default.Table_82c4120b31ef4f7fbb290c51dc0f7866 [id=ef32a2eeb5f94bbc8e3ae63aabbc232f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.707321 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c4f969e31214b6ab61328ac2e8413b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.709911 1182 ts_tablet_manager.cc:1929] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:49.713796 1182 log.cc:1199] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/d2d27ceea1bb4122b18f0fcf4d2d1bf4
I20260202 02:52:49.714818 1182 ts_tablet_manager.cc:1950] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:49.715325 2619 raft_consensus.cc:493] T b7880cb051c24bc998003f0effed3cf8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:49.715503 2619 raft_consensus.cc:515] T b7880cb051c24bc998003f0effed3cf8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.716154 2619 leader_election.cc:290] T b7880cb051c24bc998003f0effed3cf8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:49.717430 1182 tablet_replica.cc:333] T 587cfd0ae62f4cfd937faf6890e78f4a P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:49.717626 748 tablet_service.cc:1505] Processing CreateTablet for tablet 6c4f969e31214b6ab61328ac2e8413b6 (DEFAULT_TABLE table=default.Table_82c4120b31ef4f7fbb290c51dc0f7866 [id=ef32a2eeb5f94bbc8e3ae63aabbc232f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.718066 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c4f969e31214b6ab61328ac2e8413b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.718325 1182 raft_consensus.cc:2243] T 587cfd0ae62f4cfd937faf6890e78f4a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:49.718767 1182 raft_consensus.cc:2272] T 587cfd0ae62f4cfd937faf6890e78f4a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:49.719538 1182 ts_tablet_manager.cc:1916] T 587cfd0ae62f4cfd937faf6890e78f4a P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:49.723922 526 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b00b4a0b85374bfe83928162c4da6479 has no consensus state
I20260202 02:52:49.724117 749 tablet_service.cc:1505] Processing CreateTablet for tablet 910f7b96740c44b6af683109f5732824 (DEFAULT_TABLE table=default.Table_7d923e5f4b2f4f5993cf00febdffa326 [id=3afdf689960a47638cb9a8273d4d8dc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.724524 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 910f7b96740c44b6af683109f5732824. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:49.727087 523 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 604b3b863c9540d3b55381dbfe1afdc3 consensus state has no leader
I20260202 02:52:49.729261 523 master_service.cc:954] ReplaceTablet: received request to replace tablet 745fc215251e4ccb9a2abed02573dd84 from {username='slave'} at 127.0.0.1:47430
W20260202 02:52:49.735478 2753 tablet_replica.cc:1410] Time spent flushing the Log queue took a long time: real 0.345s user 0.001s sys 0.000s
I20260202 02:52:49.741989 746 tablet_service.cc:1552] Processing DeleteTablet for tablet 42321f4e77484027a485fdd584de3ae3 with delete_type TABLET_DATA_DELETED (replaced by tablet fea0fcb8854c4fc6acda80c8ffb8d341) from {username='slave'} at 127.0.0.1:42950
W20260202 02:52:49.742429 714 leader_election.cc:343] T 072bb7f0e4ab4935be49461bc8a3c163 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 072bb7f0e4ab4935be49461bc8a3c163
W20260202 02:52:49.742712 714 leader_election.cc:343] T 072bb7f0e4ab4935be49461bc8a3c163 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 072bb7f0e4ab4935be49461bc8a3c163
W20260202 02:52:49.743077 714 leader_election.cc:343] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 1116872cc25a4ab5b1a3a6bbba1f0da7
W20260202 02:52:49.743320 714 leader_election.cc:343] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 1116872cc25a4ab5b1a3a6bbba1f0da7
I20260202 02:52:49.743705 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 8774c6bc0fe9414ea2f17ed859d9392a (table default.Table_0eb450e3edbb4a4ba0906567f831830c [id=8918ec9e2cdf4f2995b2b4baf21297d7]) successfully deleted
I20260202 02:52:49.744897 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 68823eff1bc04108ba83487605aec0ea (table default.Table_d4ad0ff5676e4af2a187497eda81d7a2 [id=0769ae34ed3d479a9f7010b34c35a110]) successfully deleted
W20260202 02:52:49.745301 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 47edd9f65ef343b9a3be74c30708daca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 47edd9f65ef343b9a3be74c30708daca already in progress: creating tablet
W20260202 02:52:49.745656 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 83d198d90a204c6396b569fc1b65abef because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 83d198d90a204c6396b569fc1b65abef already in progress: creating tablet
I20260202 02:52:49.745757 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_eaefe83cde2b48418b7a3407edb48f5e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:49.746038 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet b05008bde58d47f7adaaf35ac5949bdc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b05008bde58d47f7adaaf35ac5949bdc already in progress: creating tablet
W20260202 02:52:49.746380 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 4ec86e245f8c4d9e87c3ee3409a164b3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4ec86e245f8c4d9e87c3ee3409a164b3 already in progress: creating tablet
W20260202 02:52:49.746730 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 6a3dac5cd1ce47488a50b4c32eca67c7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6a3dac5cd1ce47488a50b4c32eca67c7 already in progress: creating tablet
W20260202 02:52:49.746388 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eaefe83cde2b48418b7a3407edb48f5e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:49.751099 523 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 745fc215251e4ccb9a2abed02573dd84
I20260202 02:52:49.751579 523 catalog_manager.cc:6724] ReplaceTablet: tablet 745fc215251e4ccb9a2abed02573dd84 deleted and replaced by tablet 8457781dd88e4a9c8e343797cc781e22
I20260202 02:52:49.753527 521 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_e71a7347855d447cabe6ea2cc64b293a" } modify_external_catalogs: true
I20260202 02:52:49.753775 521 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_e71a7347855d447cabe6ea2cc64b293a" } modify_external_catalogs: true
I20260202 02:52:49.753914 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 654) took 20876 ms. Trace:
I20260202 02:52:49.754009 1180 rpcz_store.cc:276] 0202 02:52:28.877812 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:28.877866 (+ 54us) service_pool.cc:225] Handling call
0202 02:52:49.600282 (+20722416us) ts_tablet_manager.cc:1151] Deleting tablet 9c1271ce0f194021b588b82ac9d3db79
0202 02:52:49.600289 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.606225 (+ 5936us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.611405 (+ 5180us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.753898 (+142493us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":138786,"tablet-delete.queue_time_us":20719694}
I20260202 02:52:49.754302 1180 tablet_replica.cc:333] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:49.754482 1180 raft_consensus.cc:2243] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:49.754806 1180 pending_rounds.cc:70] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d: Trying to abort 1 pending ops.
I20260202 02:52:49.754856 1180 pending_rounds.cc:77] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923066236026880 op_type: NO_OP noop_request { }
I20260202 02:52:49.754966 1180 raft_consensus.cc:2889] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:49.755043 1180 raft_consensus.cc:2272] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:49.755587 1180 ts_tablet_manager.cc:1916] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:49.760221 520 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_a3b4c2b185b143b49e036ffd0b4d33af" } modify_external_catalogs: true
I20260202 02:52:49.760459 520 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_a3b4c2b185b143b49e036ffd0b4d33af" } modify_external_catalogs: true
I20260202 02:52:49.763669 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet d2d27ceea1bb4122b18f0fcf4d2d1bf4 (table default.Table_10aa0dafec0547069eb92ecf12b3c780 [id=1f9c48c8e2484856a66a0719268f22ae]) successfully deleted
W20260202 02:52:49.763921 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 7ac33134670747ee860b6d39dd7db547 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7ac33134670747ee860b6d39dd7db547 already in progress: creating tablet
W20260202 02:52:49.764139 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 8a206159afa84ae19f8c7763a2fe43d4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8a206159afa84ae19f8c7763a2fe43d4 already in progress: creating tablet
W20260202 02:52:49.764314 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 9d0212e64d934968bdbdc627c69874b1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9d0212e64d934968bdbdc627c69874b1 already in progress: creating tablet
W20260202 02:52:49.764518 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet d1342810762e41588ea00ea02acad3e2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d1342810762e41588ea00ea02acad3e2 already in progress: creating tablet
I20260202 02:52:49.766388 2807 tablet.cc:1722] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 6
I20260202 02:52:49.658519 2797 raft_consensus.cc:515] T ccd6bc8bd0504bf18d108c73685ef9ff P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.769348 2797 leader_election.cc:290] T ccd6bc8bd0504bf18d108c73685ef9ff P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:49.682442 2794 raft_consensus.cc:493] T 474df41e4fe24a818fa45a4ce84d5474 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:49.685562 2740 raft_consensus.cc:3055] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Stepping down as leader of term 1
I20260202 02:52:49.769580 2740 raft_consensus.cc:740] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:49.769843 2740 consensus_queue.cc:260] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.770159 2740 raft_consensus.cc:3060] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:49.771790 1174 tablet_bootstrap.cc:492] T 6df3d11cf578404ca41807f0777b6c53 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:49.771930 1174 ts_tablet_manager.cc:1403] T 6df3d11cf578404ca41807f0777b6c53 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.391s user 0.009s sys 0.008s
I20260202 02:52:49.772742 1174 raft_consensus.cc:359] T 6df3d11cf578404ca41807f0777b6c53 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.772920 1174 raft_consensus.cc:385] T 6df3d11cf578404ca41807f0777b6c53 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:49.772974 1174 raft_consensus.cc:740] T 6df3d11cf578404ca41807f0777b6c53 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:49.773195 1174 consensus_queue.cc:260] T 6df3d11cf578404ca41807f0777b6c53 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.773413 2794 raft_consensus.cc:515] T 474df41e4fe24a818fa45a4ce84d5474 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.690929 1033 tablet_service.cc:1505] Processing CreateTablet for tablet 137dae39f8f54be5af6354e2b6226adc (DEFAULT_TABLE table=default.Table_0c357b5eb1e14dbba252f4edfc630705 [id=64eea8b3e2d34698ad2dbe65d87d29f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.773732 1174 ts_tablet_manager.cc:1434] T 6df3d11cf578404ca41807f0777b6c53 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:49.773949 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 137dae39f8f54be5af6354e2b6226adc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.774040 1174 tablet_bootstrap.cc:492] T a0b5d82e5fb94b60b4aa324b978a5378 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:49.775983 524 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_4062ed5efe774bc2b380c3ae7fa53abe" } modify_external_catalogs: true
I20260202 02:52:49.691552 1038 tablet_service.cc:1505] Processing CreateTablet for tablet 6c4f969e31214b6ab61328ac2e8413b6 (DEFAULT_TABLE table=default.Table_82c4120b31ef4f7fbb290c51dc0f7866 [id=ef32a2eeb5f94bbc8e3ae63aabbc232f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.691812 1026 tablet_service.cc:1505] Processing CreateTablet for tablet f90a66147b1e417e8739dfd8faca034e (DEFAULT_TABLE table=default.Table_2d8e70c77d2e4bbe994ff7493c108625 [id=99d756cd7e01458b8869b01866861f55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.693235 1041 tablet_service.cc:1505] Processing CreateTablet for tablet 910f7b96740c44b6af683109f5732824 (DEFAULT_TABLE table=default.Table_7d923e5f4b2f4f5993cf00febdffa326 [id=3afdf689960a47638cb9a8273d4d8dc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.776679 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c4f969e31214b6ab61328ac2e8413b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.776700 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_672611622d8d492f97aa14af71206575"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:49.777663 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_672611622d8d492f97aa14af71206575 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:49.777942 523 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_458f570486154241ad8c7f10a8135ace" } modify_external_catalogs: true
I20260202 02:52:49.778234 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 910f7b96740c44b6af683109f5732824. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.724689 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b7880cb051c24bc998003f0effed3cf8" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:49.782212 1045 raft_consensus.cc:2468] T b7880cb051c24bc998003f0effed3cf8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:49.743062 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet 42321f4e77484027a485fdd584de3ae3 with delete_type TABLET_DATA_DELETED (replaced by tablet fea0fcb8854c4fc6acda80c8ffb8d341) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:49.752858 1035 tablet_service.cc:1552] Processing DeleteTablet for tablet 745fc215251e4ccb9a2abed02573dd84 with delete_type TABLET_DATA_DELETED (replaced by tablet 8457781dd88e4a9c8e343797cc781e22) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:49.785655 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f90a66147b1e417e8739dfd8faca034e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.786788 2794 leader_election.cc:290] T 474df41e4fe24a818fa45a4ce84d5474 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:49.787233 715 leader_election.cc:304] T b7880cb051c24bc998003f0effed3cf8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:49.789292 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_de3de7c394c64794af3b8bffdc413699"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:49.789923 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_de3de7c394c64794af3b8bffdc413699 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:49.790043 2633 raft_consensus.cc:2804] T b7880cb051c24bc998003f0effed3cf8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:49.790135 2633 raft_consensus.cc:493] T b7880cb051c24bc998003f0effed3cf8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:49.790187 2633 raft_consensus.cc:3060] T b7880cb051c24bc998003f0effed3cf8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:49.792824 2633 raft_consensus.cc:515] T b7880cb051c24bc998003f0effed3cf8 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.793768 525 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_063766871b1745af8bf479effa86c7f1" } new_table_name: "default.Table_1676352d51154567a520a2e75b86277f" modify_external_catalogs: true
I20260202 02:52:49.793732 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_b8aab442fd004224a0e08473c4f2b075"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:49.794396 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b8aab442fd004224a0e08473c4f2b075 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:49.794397 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b7880cb051c24bc998003f0effed3cf8" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:49.794598 1045 raft_consensus.cc:3060] T b7880cb051c24bc998003f0effed3cf8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:49.794947 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_7beba71f16ad4551b219e1a1e8b99a62"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:49.795519 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7beba71f16ad4551b219e1a1e8b99a62 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:49.798663 1045 raft_consensus.cc:2468] T b7880cb051c24bc998003f0effed3cf8 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:49.799055 1180 ts_tablet_manager.cc:1929] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:49.799206 1180 log.cc:1199] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/08d8d781bb4e4acf915f11afb4fb2e22
I20260202 02:52:49.799181 715 leader_election.cc:304] T b7880cb051c24bc998003f0effed3cf8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
W20260202 02:52:49.757635 998 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 108754us
I20260202 02:52:49.758097 1882 raft_consensus.cc:2243] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:49.799844 2667 raft_consensus.cc:2804] T b7880cb051c24bc998003f0effed3cf8 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:49.800048 2667 raft_consensus.cc:697] T b7880cb051c24bc998003f0effed3cf8 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:49.800225 1882 raft_consensus.cc:2272] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:49.800266 2667 consensus_queue.cc:237] T b7880cb051c24bc998003f0effed3cf8 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.690411 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet 6df3d11cf578404ca41807f0777b6c53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:49.802628 1882 ts_tablet_manager.cc:1916] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:49.805316 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "29c996781cb947f3a1aee5dd0a9c4828" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:49.806874 2764 raft_consensus.cc:493] T 6df3d11cf578404ca41807f0777b6c53 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:49.807021 2764 raft_consensus.cc:515] T 6df3d11cf578404ca41807f0777b6c53 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.807667 2764 leader_election.cc:290] T 6df3d11cf578404ca41807f0777b6c53 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:49.810241 1882 ts_tablet_manager.cc:1929] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:49.810729 1882 log.cc:1199] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/874ca4ba4ebc4721bb4b60cba8e01841
I20260202 02:52:49.811174 1882 ts_tablet_manager.cc:1950] T 874ca4ba4ebc4721bb4b60cba8e01841 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:49.811336 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_a4b8204d7f474e539edf79a5e07fd706"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:49.811532 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 712) took 19194 ms. Trace:
I20260202 02:52:49.811606 1882 rpcz_store.cc:276] 0202 02:52:30.616668 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:35.770623 (+5153955us) service_pool.cc:225] Handling call
0202 02:52:49.523914 (+13753291us) ts_tablet_manager.cc:1151] Deleting tablet 874ca4ba4ebc4721bb4b60cba8e01841
0202 02:52:49.523924 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.799963 (+276039us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x6160000a9654. stack: 00007fec53205a86 00007fec53205840 00007fec521e03ad 0000000000984f94 0000000000984f5d 000000000098492a 00007fec5f0f31e0 00007fec604410b3 00007fec63001757 00007fec63034744 00007fec63015240 00007fec63014fcd 0000000000a8970c 00007fec5324bdaf 00007fec5324f0cb 00007fec5324ef48
0202 02:52:49.805564 (+ 5601us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.810126 (+ 4562us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.811513 (+ 1387us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":538081664,"tablet-delete.queue_time_us":13753179}
I20260202 02:52:49.811969 1882 tablet_replica.cc:333] T 33150e1456144b8e9c64cc2a2ac6dcca P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:49.812139 1882 raft_consensus.cc:2243] T 33150e1456144b8e9c64cc2a2ac6dcca P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:49.812296 1882 raft_consensus.cc:2272] T 33150e1456144b8e9c64cc2a2ac6dcca P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:49.753572 1175 tablet_bootstrap.cc:492] T abfe7f76b2424c4d9ab07b29b66ec103 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:49.812633 1175 ts_tablet_manager.cc:1403] T abfe7f76b2424c4d9ab07b29b66ec103 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.464s user 0.022s sys 0.038s
I20260202 02:52:49.812609 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6df3d11cf578404ca41807f0777b6c53" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:49.812672 524 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_4062ed5efe774bc2b380c3ae7fa53abe" } modify_external_catalogs: true
I20260202 02:52:49.812805 1045 raft_consensus.cc:2468] T 6df3d11cf578404ca41807f0777b6c53 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:49.813259 715 leader_election.cc:304] T 6df3d11cf578404ca41807f0777b6c53 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:49.813377 1175 raft_consensus.cc:359] T abfe7f76b2424c4d9ab07b29b66ec103 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:49.813529 1175 raft_consensus.cc:385] T abfe7f76b2424c4d9ab07b29b66ec103 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:49.762991 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "29c996781cb947f3a1aee5dd0a9c4828" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:49.813793 1174 tablet_bootstrap.cc:654] T a0b5d82e5fb94b60b4aa324b978a5378 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:49.813784 923 raft_consensus.cc:2468] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:49.813880 2619 raft_consensus.cc:2804] T 6df3d11cf578404ca41807f0777b6c53 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:49.813968 2619 raft_consensus.cc:493] T 6df3d11cf578404ca41807f0777b6c53 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:49.765261 893 tablet_service.cc:1505] Processing CreateTablet for tablet 0aa023e74e74480498b721bbb5d15052 (DEFAULT_TABLE table=default.Table_88cb19d3d7684dc1a8aa41ed28fed4b8 [id=b40664207c364307ba6f5ade490a21d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.814054 2619 raft_consensus.cc:3060] T 6df3d11cf578404ca41807f0777b6c53 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:49.814263 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a4b8204d7f474e539edf79a5e07fd706 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:49.814373 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0aa023e74e74480498b721bbb5d15052. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.816552 523 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_458f570486154241ad8c7f10a8135ace" } modify_external_catalogs: true
I20260202 02:52:49.816704 2619 raft_consensus.cc:515] T 6df3d11cf578404ca41807f0777b6c53 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.818030 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6df3d11cf578404ca41807f0777b6c53" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:49.818145 2619 leader_election.cc:290] T 6df3d11cf578404ca41807f0777b6c53 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:49.818207 1045 raft_consensus.cc:3060] T 6df3d11cf578404ca41807f0777b6c53 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:49.806542 899 tablet_service.cc:1552] Processing DeleteTablet for tablet 745fc215251e4ccb9a2abed02573dd84 with delete_type TABLET_DATA_DELETED (replaced by tablet 8457781dd88e4a9c8e343797cc781e22) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:49.819664 2633 leader_election.cc:290] T b7880cb051c24bc998003f0effed3cf8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:49.813593 1175 raft_consensus.cc:740] T abfe7f76b2424c4d9ab07b29b66ec103 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:49.820242 1175 consensus_queue.cc:260] T abfe7f76b2424c4d9ab07b29b66ec103 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:49.762802 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 42321f4e77484027a485fdd584de3ae3 with delete_type TABLET_DATA_DELETED (replaced by tablet fea0fcb8854c4fc6acda80c8ffb8d341) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:49.820823 1175 ts_tablet_manager.cc:1434] T abfe7f76b2424c4d9ab07b29b66ec103 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20260202 02:52:49.821130 1175 tablet_bootstrap.cc:492] T 3284213e4b0447b49a7718ba896d3fa8 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:49.822052 1180 ts_tablet_manager.cc:1950] T 08d8d781bb4e4acf915f11afb4fb2e22 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:49.822976 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 655) took 20945 ms. Trace:
W20260202 02:52:49.823168 714 leader_election.cc:343] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: f94f25f4059a43b8babedb6b45a08a19
I20260202 02:52:49.823143 1180 rpcz_store.cc:276] 0202 02:52:28.877948 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:28.880505 (+ 2557us) service_pool.cc:225] Handling call
0202 02:52:49.754260 (+20873755us) ts_tablet_manager.cc:1151] Deleting tablet 08d8d781bb4e4acf915f11afb4fb2e22
0202 02:52:49.754267 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.760916 (+ 6649us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.798929 (+ 38013us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.822956 (+ 24027us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":20873552}
W20260202 02:52:49.823556 714 leader_election.cc:343] T b7880cb051c24bc998003f0effed3cf8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: b7880cb051c24bc998003f0effed3cf8
I20260202 02:52:49.823650 1045 raft_consensus.cc:2468] T 6df3d11cf578404ca41807f0777b6c53 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:49.823817 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 656) took 20877 ms. Trace:
I20260202 02:52:49.823916 1180 rpcz_store.cc:276] 0202 02:52:28.945954 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:28.946035 (+ 81us) service_pool.cc:225] Handling call
0202 02:52:49.823597 (+20877562us) ts_tablet_manager.cc:1151] Deleting tablet 79e65a55f29449a0b13349bc152c651b
0202 02:52:49.823608 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.823800 (+ 192us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":20877362}
I20260202 02:52:49.824131 715 leader_election.cc:304] T 6df3d11cf578404ca41807f0777b6c53 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:49.824255 1180 tablet_replica.cc:333] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:49.824455 1180 raft_consensus.cc:2243] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:49.824640 744 tablet_service.cc:1552] Processing DeleteTablet for tablet 745fc215251e4ccb9a2abed02573dd84 with delete_type TABLET_DATA_DELETED (replaced by tablet 8457781dd88e4a9c8e343797cc781e22) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:49.824820 1180 pending_rounds.cc:70] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d: Trying to abort 1 pending ops.
I20260202 02:52:49.824767 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 9c1271ce0f194021b588b82ac9d3db79 (table default.Table_733044f900d44218a06c976c8701f122 [id=9f912bc02bec49a8b79e4e7e921595b9]) successfully deleted
I20260202 02:52:49.824955 1180 pending_rounds.cc:77] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923070173491200 op_type: NO_OP noop_request { }
I20260202 02:52:49.825119 1180 raft_consensus.cc:2889] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:49.825259 1180 raft_consensus.cc:2272] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:49.825232 744 tablet_service.cc:1505] Processing CreateTablet for tablet 0aa023e74e74480498b721bbb5d15052 (DEFAULT_TABLE table=default.Table_88cb19d3d7684dc1a8aa41ed28fed4b8 [id=b40664207c364307ba6f5ade490a21d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.825641 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0aa023e74e74480498b721bbb5d15052. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.825989 1180 ts_tablet_manager.cc:1916] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:49.828833 1882 ts_tablet_manager.cc:1916] T 33150e1456144b8e9c64cc2a2ac6dcca P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:49.829545 2633 raft_consensus.cc:2804] T 6df3d11cf578404ca41807f0777b6c53 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:49.829813 2633 raft_consensus.cc:697] T 6df3d11cf578404ca41807f0777b6c53 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:49.830251 2633 consensus_queue.cc:237] T 6df3d11cf578404ca41807f0777b6c53 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.835448 1882 ts_tablet_manager.cc:1929] T 33150e1456144b8e9c64cc2a2ac6dcca P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:49.835595 1882 log.cc:1199] T 33150e1456144b8e9c64cc2a2ac6dcca P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/33150e1456144b8e9c64cc2a2ac6dcca
I20260202 02:52:49.835974 1882 ts_tablet_manager.cc:1950] T 33150e1456144b8e9c64cc2a2ac6dcca P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:49.837707 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "474df41e4fe24a818fa45a4ce84d5474" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:49.837749 998 leader_election.cc:343] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:49.839228 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ccd6bc8bd0504bf18d108c73685ef9ff" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:49.839465 999 leader_election.cc:304] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:49.840041 2794 raft_consensus.cc:2804] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:49.840155 2794 raft_consensus.cc:493] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:49.840235 2794 raft_consensus.cc:3060] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:49.843039 2794 raft_consensus.cc:515] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.843955 2794 leader_election.cc:290] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:49.844396 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 874ca4ba4ebc4721bb4b60cba8e01841 (table default.Table_9913ba1a0b0c4f30bc3dd302ee61b92d [id=6777a441613b40cbb21d431744fca23f]) successfully deleted
I20260202 02:52:49.845786 1036 tablet_service.cc:1505] Processing CreateTablet for tablet 0aa023e74e74480498b721bbb5d15052 (DEFAULT_TABLE table=default.Table_88cb19d3d7684dc1a8aa41ed28fed4b8 [id=b40664207c364307ba6f5ade490a21d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.846215 1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0aa023e74e74480498b721bbb5d15052. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.850394 1174 tablet_bootstrap.cc:492] T a0b5d82e5fb94b60b4aa324b978a5378 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:49.850554 1174 ts_tablet_manager.cc:1403] T a0b5d82e5fb94b60b4aa324b978a5378 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.077s user 0.002s sys 0.001s
W20260202 02:52:49.850912 999 leader_election.cc:343] T ccd6bc8bd0504bf18d108c73685ef9ff P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: ccd6bc8bd0504bf18d108c73685ef9ff
I20260202 02:52:49.851280 1174 raft_consensus.cc:359] T a0b5d82e5fb94b60b4aa324b978a5378 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.851465 1174 raft_consensus.cc:385] T a0b5d82e5fb94b60b4aa324b978a5378 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:49.851840 1174 raft_consensus.cc:740] T a0b5d82e5fb94b60b4aa324b978a5378 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:49.852044 1174 consensus_queue.cc:260] T a0b5d82e5fb94b60b4aa324b978a5378 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.852555 1174 ts_tablet_manager.cc:1434] T a0b5d82e5fb94b60b4aa324b978a5378 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:49.853037 1174 tablet_bootstrap.cc:492] T 474df41e4fe24a818fa45a4ce84d5474 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:49.854702 1174 tablet_bootstrap.cc:654] T 474df41e4fe24a818fa45a4ce84d5474 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:49.855559 521 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 3284213e4b0447b49a7718ba896d3fa8
I20260202 02:52:49.856252 520 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet 1a0bcff4269e4e03bc0dcd48f6967671
I20260202 02:52:49.856688 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_fdd92a256f0648e2b27bd79fa4435d9f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:49.857903 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fdd92a256f0648e2b27bd79fa4435d9f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:49.858281 998 leader_election.cc:343] T 474df41e4fe24a818fa45a4ce84d5474 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:49.858466 998 leader_election.cc:343] T ccd6bc8bd0504bf18d108c73685ef9ff P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:49.858536 998 leader_election.cc:304] T ccd6bc8bd0504bf18d108c73685ef9ff P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:49.858875 2740 raft_consensus.cc:2749] T ccd6bc8bd0504bf18d108c73685ef9ff P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:49.859369 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "29c996781cb947f3a1aee5dd0a9c4828" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:49.860216 2808 tablet.cc:1722] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 6
W20260202 02:52:49.861598 998 leader_election.cc:343] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:49.862994 521 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_e9fb673e50b44eecab9fddc7d896148e" } new_table_name: "default.Table_2d519304c313459a8b38c0fe0c65d21f" modify_external_catalogs: true
W20260202 02:52:49.863742 714 leader_election.cc:343] T 6df3d11cf578404ca41807f0777b6c53 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 6df3d11cf578404ca41807f0777b6c53
W20260202 02:52:49.864032 714 leader_election.cc:343] T 6df3d11cf578404ca41807f0777b6c53 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 6df3d11cf578404ca41807f0777b6c53
W20260202 02:52:49.864233 714 leader_election.cc:343] T b7880cb051c24bc998003f0effed3cf8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: b7880cb051c24bc998003f0effed3cf8
I20260202 02:52:49.864705 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 08d8d781bb4e4acf915f11afb4fb2e22 (table default.Table_2acd399bc33146fe9f1b6e2a15e0b101 [id=487a204c7845401ab7391ae8464fdf1d]) successfully deleted
W20260202 02:52:49.864995 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 79e65a55f29449a0b13349bc152c651b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 79e65a55f29449a0b13349bc152c651b already in progress: creating tablet
W20260202 02:52:49.865684 999 leader_election.cc:343] T 474df41e4fe24a818fa45a4ce84d5474 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 474df41e4fe24a818fa45a4ce84d5474
I20260202 02:52:49.865926 999 leader_election.cc:304] T 474df41e4fe24a818fa45a4ce84d5474 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:49.866199 2740 raft_consensus.cc:2749] T 474df41e4fe24a818fa45a4ce84d5474 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:49.763219 886 tablet_service.cc:1505] Processing CreateTablet for tablet 910f7b96740c44b6af683109f5732824 (DEFAULT_TABLE table=default.Table_7d923e5f4b2f4f5993cf00febdffa326 [id=3afdf689960a47638cb9a8273d4d8dc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:49.849874 923 raft_consensus.cc:1275] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Refusing update from remote peer b1f05820b6034228bdad884cc348716d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260202 02:52:49.850255 919 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "29c996781cb947f3a1aee5dd0a9c4828" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:49.866984 919 raft_consensus.cc:3060] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:49.867153 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 910f7b96740c44b6af683109f5732824. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:49.864881 969 heartbeater.cc:344] Connected to a master server at 127.0.104.190:33981
I20260202 02:52:49.868947 1173 tablet_bootstrap.cc:492] T a0b5d82e5fb94b60b4aa324b978a5378 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:49.869621 919 raft_consensus.cc:2468] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:49.874336 773 raft_consensus.cc:1275] T 26b890ed87c3471b9fc0b172c90e0a11 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Refusing update from remote peer b1f05820b6034228bdad884cc348716d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260202 02:52:49.876502 1173 ts_tablet_manager.cc:1403] T a0b5d82e5fb94b60b4aa324b978a5378 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.213s user 0.007s sys 0.000s
I20260202 02:52:49.876931 2753 tablet.cc:1722] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 6
I20260202 02:52:49.869609 2794 consensus_queue.cc:1048] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260202 02:52:49.878465 999 leader_election.cc:304] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:49.878819 2740 consensus_queue.cc:1048] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:49.879015 891 tablet_service.cc:1552] Processing DeleteTablet for tablet abfe7f76b2424c4d9ab07b29b66ec103 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:47 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:49.879230 2740 raft_consensus.cc:2804] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:49.879240 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 3284213e4b0447b49a7718ba896d3fa8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:49.879335 2740 raft_consensus.cc:697] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:49.879560 2740 consensus_queue.cc:237] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:49.884218 525 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3ee6153db90b43c5866a8bb09bcebcb3 consensus state has no leader
I20260202 02:52:49.885933 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet 3284213e4b0447b49a7718ba896d3fa8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:49.888993 522 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_aafa37fc93f045a2b0525cea42a60866" } modify_external_catalogs: true
I20260202 02:52:49.889246 522 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_aafa37fc93f045a2b0525cea42a60866" } modify_external_catalogs: true
I20260202 02:52:49.892539 1173 raft_consensus.cc:359] T a0b5d82e5fb94b60b4aa324b978a5378 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.893419 1173 raft_consensus.cc:385] T a0b5d82e5fb94b60b4aa324b978a5378 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:49.893918 1173 raft_consensus.cc:740] T a0b5d82e5fb94b60b4aa324b978a5378 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
W20260202 02:52:49.894037 714 consensus_peers.cc:597] T dada209d982d48f3b74a49905f19c5b6 P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dada209d982d48f3b74a49905f19c5b6. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:49.894641 744 tablet_service.cc:1552] Processing DeleteTablet for tablet 3284213e4b0447b49a7718ba896d3fa8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:49.896421 1173 consensus_queue.cc:260] T a0b5d82e5fb94b60b4aa324b978a5378 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.898900 1174 tablet_bootstrap.cc:492] T 474df41e4fe24a818fa45a4ce84d5474 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:49.900233 1174 ts_tablet_manager.cc:1403] T 474df41e4fe24a818fa45a4ce84d5474 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.047s user 0.001s sys 0.004s
I20260202 02:52:49.900578 1173 ts_tablet_manager.cc:1434] T a0b5d82e5fb94b60b4aa324b978a5378 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.024s user 0.002s sys 0.000s
I20260202 02:52:49.901090 1174 raft_consensus.cc:359] T 474df41e4fe24a818fa45a4ce84d5474 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.901269 1174 raft_consensus.cc:385] T 474df41e4fe24a818fa45a4ce84d5474 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:49.901309 1175 tablet_bootstrap.cc:654] T 3284213e4b0447b49a7718ba896d3fa8 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:49.901326 1174 raft_consensus.cc:740] T 474df41e4fe24a818fa45a4ce84d5474 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:49.901597 1174 consensus_queue.cc:260] T 474df41e4fe24a818fa45a4ce84d5474 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.902079 1174 ts_tablet_manager.cc:1434] T 474df41e4fe24a818fa45a4ce84d5474 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:49.902400 1174 tablet_bootstrap.cc:492] T fbcb4a522df648ba9542250450c08f1b P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:49.905306 519 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_df42f88a6807472088543ed874ac5e76" } new_table_name: "default.Table_a8e411c625a34f08855442d60cf25b8f" modify_external_catalogs: true
I20260202 02:52:49.911485 1174 tablet_bootstrap.cc:654] T fbcb4a522df648ba9542250450c08f1b P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:49.926975 524 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 63a9029bc39c4fbd8ac4bf4bc37d6b4e
I20260202 02:52:49.927965 744 tablet_service.cc:1552] Processing DeleteTablet for tablet 63a9029bc39c4fbd8ac4bf4bc37d6b4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:49.932757 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 63a9029bc39c4fbd8ac4bf4bc37d6b4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:49.933380 2445 raft_consensus.cc:493] T e2edc6660b9a41e39e9529627fafcb03 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:49.933524 2445 raft_consensus.cc:515] T e2edc6660b9a41e39e9529627fafcb03 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:49.934124 2445 leader_election.cc:290] T e2edc6660b9a41e39e9529627fafcb03 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:49.935426 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e2edc6660b9a41e39e9529627fafcb03" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:49.935395 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e2edc6660b9a41e39e9529627fafcb03" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:49.936414 859 leader_election.cc:343] T e2edc6660b9a41e39e9529627fafcb03 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:49.936729 857 leader_election.cc:343] T e2edc6660b9a41e39e9529627fafcb03 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:49.937022 1173 tablet_bootstrap.cc:492] T 587cfd0ae62f4cfd937faf6890e78f4a P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:49.937867 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_442347079e5c4da39ef0908bcae95ae5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:49.938535 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_442347079e5c4da39ef0908bcae95ae5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:49.938578 1173 tablet_bootstrap.cc:654] T 587cfd0ae62f4cfd937faf6890e78f4a P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:49.938829 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_ef8313f59c0d44e3bb7ec522181f6c25"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:49.939309 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ef8313f59c0d44e3bb7ec522181f6c25 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:49.940188 744 tablet_service.cc:1552] Processing DeleteTablet for tablet 6df3d11cf578404ca41807f0777b6c53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:49.940659 744 tablet_service.cc:1552] Processing DeleteTablet for tablet 072bb7f0e4ab4935be49461bc8a3c163 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:49.940762 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_3638d0d39d6943e28507ce3f8aaf5829"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:49.941728 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3638d0d39d6943e28507ce3f8aaf5829 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:49.944530 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet 63a9029bc39c4fbd8ac4bf4bc37d6b4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:49.945930 2445 raft_consensus.cc:493] T 59ad016e15ec4a488921eb564029b689 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:49.946076 2445 raft_consensus.cc:515] T 59ad016e15ec4a488921eb564029b689 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:49.946663 2445 leader_election.cc:290] T 59ad016e15ec4a488921eb564029b689 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:49.948189 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "59ad016e15ec4a488921eb564029b689" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:49.948855 859 leader_election.cc:343] T 59ad016e15ec4a488921eb564029b689 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:49.936843 857 leader_election.cc:304] T e2edc6660b9a41e39e9529627fafcb03 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:49.951604 2445 raft_consensus.cc:2749] T e2edc6660b9a41e39e9529627fafcb03 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:49.955755 1182 ts_tablet_manager.cc:1929] T 587cfd0ae62f4cfd937faf6890e78f4a P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:49.956009 1182 log.cc:1199] T 587cfd0ae62f4cfd937faf6890e78f4a P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/587cfd0ae62f4cfd937faf6890e78f4a
I20260202 02:52:49.956516 1182 ts_tablet_manager.cc:1950] T 587cfd0ae62f4cfd937faf6890e78f4a P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:49.957149 1173 tablet_bootstrap.cc:492] T 587cfd0ae62f4cfd937faf6890e78f4a P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:49.957298 1173 ts_tablet_manager.cc:1403] T 587cfd0ae62f4cfd937faf6890e78f4a P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.001s
I20260202 02:52:49.958084 1173 raft_consensus.cc:359] T 587cfd0ae62f4cfd937faf6890e78f4a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:49.958283 1173 raft_consensus.cc:385] T 587cfd0ae62f4cfd937faf6890e78f4a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:49.958333 1173 raft_consensus.cc:740] T 587cfd0ae62f4cfd937faf6890e78f4a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:49.958530 1173 consensus_queue.cc:260] T 587cfd0ae62f4cfd937faf6890e78f4a P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:49.959030 1173 ts_tablet_manager.cc:1434] T 587cfd0ae62f4cfd937faf6890e78f4a P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260202 02:52:49.959306 1173 tablet_bootstrap.cc:492] T 65bd1c1a382e42e4a68077264563c55b P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:49.960754 1173 tablet_bootstrap.cc:654] T 65bd1c1a382e42e4a68077264563c55b P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:49.965371 1173 tablet_bootstrap.cc:492] T 65bd1c1a382e42e4a68077264563c55b P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:49.965615 1173 ts_tablet_manager.cc:1403] T 65bd1c1a382e42e4a68077264563c55b P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260202 02:52:49.966295 1173 raft_consensus.cc:359] T 65bd1c1a382e42e4a68077264563c55b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:49.966631 1173 raft_consensus.cc:385] T 65bd1c1a382e42e4a68077264563c55b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:49.966737 1173 raft_consensus.cc:740] T 65bd1c1a382e42e4a68077264563c55b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:49.967028 1173 consensus_queue.cc:260] T 65bd1c1a382e42e4a68077264563c55b P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:49.967715 1173 ts_tablet_manager.cc:1434] T 65bd1c1a382e42e4a68077264563c55b P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:49.968040 1173 tablet_bootstrap.cc:492] T ac2ab12f5040488aa306cb69b107b485 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:49.960852 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 713) took 19344 ms. Trace:
I20260202 02:52:49.968763 1882 rpcz_store.cc:276] 0202 02:52:30.616775 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:35.770749 (+5153974us) service_pool.cc:225] Handling call
0202 02:52:49.811917 (+14041168us) ts_tablet_manager.cc:1151] Deleting tablet 33150e1456144b8e9c64cc2a2ac6dcca
0202 02:52:49.811927 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.832416 (+ 20489us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.835329 (+ 2913us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.960830 (+125501us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":14041066}
I20260202 02:52:49.969117 1882 tablet_replica.cc:333] T fbcb4a522df648ba9542250450c08f1b P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:49.969280 1882 raft_consensus.cc:2243] T fbcb4a522df648ba9542250450c08f1b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:49.969429 1882 raft_consensus.cc:2272] T fbcb4a522df648ba9542250450c08f1b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:49.969578 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 33150e1456144b8e9c64cc2a2ac6dcca (table default.Table_b30dfc448e1646a7b90534a8fa5fa24a [id=97791994f92a47519b28e44ec02df5ba]) successfully deleted
I20260202 02:52:49.970000 1882 ts_tablet_manager.cc:1916] T fbcb4a522df648ba9542250450c08f1b P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:49.970803 1173 tablet_bootstrap.cc:654] T ac2ab12f5040488aa306cb69b107b485 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:49.972092 1110 heartbeater.cc:344] Connected to a master server at 127.0.104.190:33981
I20260202 02:52:49.972235 1174 tablet_bootstrap.cc:492] T fbcb4a522df648ba9542250450c08f1b P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:49.972453 1174 ts_tablet_manager.cc:1403] T fbcb4a522df648ba9542250450c08f1b P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.070s user 0.005s sys 0.000s
I20260202 02:52:49.973316 1174 raft_consensus.cc:359] T fbcb4a522df648ba9542250450c08f1b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.973519 1174 raft_consensus.cc:385] T fbcb4a522df648ba9542250450c08f1b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:49.973624 1174 raft_consensus.cc:740] T fbcb4a522df648ba9542250450c08f1b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:49.973912 1174 consensus_queue.cc:260] T fbcb4a522df648ba9542250450c08f1b P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.974617 1174 ts_tablet_manager.cc:1434] T fbcb4a522df648ba9542250450c08f1b P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:49.974960 1174 tablet_bootstrap.cc:492] T ccd6bc8bd0504bf18d108c73685ef9ff P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:49.935426 1180 ts_tablet_manager.cc:1929] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:49.984602 1182 tablet_replica.cc:333] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:49.984905 2740 raft_consensus.cc:493] T 842e54adae2a452a98521de24326eaff P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:49.984935 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 587cfd0ae62f4cfd937faf6890e78f4a (table default.Table_698a93b0be2e4203836e0522ac44acc3 [id=9f2441734085417fb56dc05a42df54db]) successfully deleted
I20260202 02:52:49.985033 1182 raft_consensus.cc:2243] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:49.985036 2740 raft_consensus.cc:515] T 842e54adae2a452a98521de24326eaff P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:49.985201 1182 raft_consensus.cc:2272] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:49.985651 2740 leader_election.cc:290] T 842e54adae2a452a98521de24326eaff P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:49.986279 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "842e54adae2a452a98521de24326eaff" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:49.989128 1180 log.cc:1199] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/13cd1383348b458894fdecc22bf24b74
I20260202 02:52:49.989217 1174 tablet_bootstrap.cc:654] T ccd6bc8bd0504bf18d108c73685ef9ff P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:49.989548 998 leader_election.cc:343] T 842e54adae2a452a98521de24326eaff P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:49.992020 1180 ts_tablet_manager.cc:1950] T 13cd1383348b458894fdecc22bf24b74 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:49.996955 520 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_2d42dcf81d2d4b2c90fe09860b9b2318" } new_table_name: "default.Table_38201dffd3a349508fdedab6e81e4965" modify_external_catalogs: true
W20260202 02:52:49.998610 521 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2becb2f65a124c3990ea3a8318ff1d8c has no consensus state
I20260202 02:52:49.999053 521 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_c57661c84d424228a977c159e893db51" } new_table_name: "default.Table_8bf4f8ccac504f31893caaad3fc4431d" modify_external_catalogs: true
I20260202 02:52:50.002521 744 tablet_service.cc:1552] Processing DeleteTablet for tablet a0b5d82e5fb94b60b4aa324b978a5378 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:50.003856 744 tablet_service.cc:1552] Processing DeleteTablet for tablet 474df41e4fe24a818fa45a4ce84d5474 with delete_type TABLET_DATA_DELETED (replaced by tablet f30b698c177c48ee9a4e577552bf7eee) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:50.004865 528 master_service.cc:954] ReplaceTablet: received request to replace tablet 47edd9f65ef343b9a3be74c30708daca from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:50.007665 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 666) took 20742 ms. Trace:
I20260202 02:52:50.008952 1180 rpcz_store.cc:276] 0202 02:52:29.264940 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.265202 (+ 262us) service_pool.cc:225] Handling call
0202 02:52:49.824205 (+20559003us) ts_tablet_manager.cc:1151] Deleting tablet 13cd1383348b458894fdecc22bf24b74
0202 02:52:49.824215 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:49.931880 (+107665us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:49.935254 (+ 3374us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:50.007614 (+ 72360us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":20558872}
I20260202 02:52:50.010720 1175 tablet_bootstrap.cc:492] T 3284213e4b0447b49a7718ba896d3fa8 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:50.010993 1175 ts_tablet_manager.cc:1403] T 3284213e4b0447b49a7718ba896d3fa8 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.190s user 0.004s sys 0.002s
I20260202 02:52:50.011879 1175 raft_consensus.cc:359] T 3284213e4b0447b49a7718ba896d3fa8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.012146 1175 raft_consensus.cc:385] T 3284213e4b0447b49a7718ba896d3fa8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:50.012252 1175 raft_consensus.cc:740] T 3284213e4b0447b49a7718ba896d3fa8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:50.012550 1175 consensus_queue.cc:260] T 3284213e4b0447b49a7718ba896d3fa8 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.014766 1175 ts_tablet_manager.cc:1434] T 3284213e4b0447b49a7718ba896d3fa8 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260202 02:52:50.017504 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_3133535fffb44f4fb4f2d30a4ab61f55"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:50.018260 1175 tablet_bootstrap.cc:492] T 2b352d9a815e4bccafb7e2251500c5bc P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
W20260202 02:52:50.018308 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3133535fffb44f4fb4f2d30a4ab61f55 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:50.018883 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 13cd1383348b458894fdecc22bf24b74 (table default.Table_f92d6341db3143e490f6d64418ef6ff2 [id=ebc82aae55584b8ba2ba5c4223d9e258]) successfully deleted
I20260202 02:52:50.019343 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 667) took 20754 ms. Trace:
I20260202 02:52:50.010736 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "59ad016e15ec4a488921eb564029b689" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:50.013008 778 raft_consensus.cc:1240] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Rejecting Update request from peer b1f05820b6034228bdad884cc348716d for earlier term 1. Current term is 2. Ops: []
W20260202 02:52:50.013127 715 consensus_peers.cc:597] T dada209d982d48f3b74a49905f19c5b6 P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dada209d982d48f3b74a49905f19c5b6. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:50.023164 2740 consensus_queue.cc:1059] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260202 02:52:50.023492 2740 raft_consensus.cc:3055] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Stepping down as leader of term 1
I20260202 02:52:50.023554 2740 raft_consensus.cc:740] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:50.023447 1180 rpcz_store.cc:276] 0202 02:52:29.265053 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.265102 (+ 49us) service_pool.cc:225] Handling call
0202 02:52:50.019200 (+20754098us) ts_tablet_manager.cc:1151] Deleting tablet 2b8ba9dd450948ada395207ae1c5b325
0202 02:52:50.019214 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:50.019306 (+ 92us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":469,"tablet-delete.queue_time_us":20753862}
I20260202 02:52:50.023799 2740 consensus_queue.cc:260] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:50.024070 513 catalog_manager.cc:4952] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 2b8ba9dd450948ada395207ae1c5b325 because the tablet was not found. No further retry: Not found: Tablet not found: 2b8ba9dd450948ada395207ae1c5b325
I20260202 02:52:50.024101 2740 raft_consensus.cc:3060] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:50.024472 1180 tablet_replica.cc:333] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:50.024549 1174 tablet_bootstrap.cc:492] T ccd6bc8bd0504bf18d108c73685ef9ff P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:50.024684 1180 raft_consensus.cc:2243] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:50.024718 1174 ts_tablet_manager.cc:1403] T ccd6bc8bd0504bf18d108c73685ef9ff P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.050s user 0.004s sys 0.000s
I20260202 02:52:50.024847 1180 raft_consensus.cc:2272] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:50.025355 1180 ts_tablet_manager.cc:1916] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:50.025511 1174 raft_consensus.cc:359] T ccd6bc8bd0504bf18d108c73685ef9ff P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:50.025700 1174 raft_consensus.cc:385] T ccd6bc8bd0504bf18d108c73685ef9ff P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:50.025769 1174 raft_consensus.cc:740] T ccd6bc8bd0504bf18d108c73685ef9ff P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:50.026032 1174 consensus_queue.cc:260] T ccd6bc8bd0504bf18d108c73685ef9ff P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:50.028581 1175 tablet_bootstrap.cc:654] T 2b352d9a815e4bccafb7e2251500c5bc P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:50.029527 522 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet a81e1d10ca49459989260fbd6a2b8645
I20260202 02:52:50.030417 744 tablet_service.cc:1552] Processing DeleteTablet for tablet a81e1d10ca49459989260fbd6a2b8645 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:50.030903 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet a81e1d10ca49459989260fbd6a2b8645 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:50.033960 749 tablet_service.cc:1552] Processing DeleteTablet for tablet ccd6bc8bd0504bf18d108c73685ef9ff with delete_type TABLET_DATA_DELETED (replaced by tablet 8ba28ab7d7ae44059006179067dff3c0) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:50.039018 1173 tablet_bootstrap.cc:492] T ac2ab12f5040488aa306cb69b107b485 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:50.039264 1173 ts_tablet_manager.cc:1403] T ac2ab12f5040488aa306cb69b107b485 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.071s user 0.035s sys 0.012s
I20260202 02:52:50.039896 1882 ts_tablet_manager.cc:1929] T fbcb4a522df648ba9542250450c08f1b P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:50.040009 1882 log.cc:1199] T fbcb4a522df648ba9542250450c08f1b P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/fbcb4a522df648ba9542250450c08f1b
I20260202 02:52:50.040138 1173 raft_consensus.cc:359] T ac2ab12f5040488aa306cb69b107b485 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.040408 1173 raft_consensus.cc:385] T ac2ab12f5040488aa306cb69b107b485 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:50.040529 1173 raft_consensus.cc:740] T ac2ab12f5040488aa306cb69b107b485 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:50.040833 1173 consensus_queue.cc:260] T ac2ab12f5040488aa306cb69b107b485 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.041615 1173 ts_tablet_manager.cc:1434] T ac2ab12f5040488aa306cb69b107b485 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:50.042091 1173 tablet_bootstrap.cc:492] T 1f379a29e2454f689d68f5951837512b P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:50.043788 1173 tablet_bootstrap.cc:654] T 1f379a29e2454f689d68f5951837512b P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:50.043995 522 master_service.cc:954] ReplaceTablet: received request to replace tablet c977cd5aa66a4f6c8eeb37bf67f18fa7 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:50.046823 2445 raft_consensus.cc:493] T abfe7f76b2424c4d9ab07b29b66ec103 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:50.047013 2445 raft_consensus.cc:515] T abfe7f76b2424c4d9ab07b29b66ec103 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.047711 2445 leader_election.cc:290] T abfe7f76b2424c4d9ab07b29b66ec103 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:50.048717 1174 ts_tablet_manager.cc:1434] T ccd6bc8bd0504bf18d108c73685ef9ff P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.024s user 0.002s sys 0.000s
I20260202 02:52:50.048951 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "abfe7f76b2424c4d9ab07b29b66ec103" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:50.049492 859 leader_election.cc:343] T abfe7f76b2424c4d9ab07b29b66ec103 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:50.052644 1173 tablet_bootstrap.cc:492] T 1f379a29e2454f689d68f5951837512b P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:50.052903 1173 ts_tablet_manager.cc:1403] T 1f379a29e2454f689d68f5951837512b P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260202 02:52:50.053685 1182 ts_tablet_manager.cc:1916] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:50.053637 1173 raft_consensus.cc:359] T 1f379a29e2454f689d68f5951837512b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.053797 1173 raft_consensus.cc:385] T 1f379a29e2454f689d68f5951837512b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:50.053912 1173 raft_consensus.cc:740] T 1f379a29e2454f689d68f5951837512b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:50.054214 1173 consensus_queue.cc:260] T 1f379a29e2454f689d68f5951837512b P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.054608 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "abfe7f76b2424c4d9ab07b29b66ec103" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:50.054894 1173 ts_tablet_manager.cc:1434] T 1f379a29e2454f689d68f5951837512b P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:50.055389 1173 tablet_bootstrap.cc:492] T 44c8d5b74c8e4c499f7f68e8185efc17 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:50.056902 1173 tablet_bootstrap.cc:654] T 44c8d5b74c8e4c499f7f68e8185efc17 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:50.057875 1174 tablet_bootstrap.cc:492] T 842e54adae2a452a98521de24326eaff P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:50.059305 1174 tablet_bootstrap.cc:654] T 842e54adae2a452a98521de24326eaff P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:50.061928 519 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fe97009dbcc64846844e381817148d7b consensus state has no leader
I20260202 02:52:50.062304 519 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_03cb720f804b44e2b29f8cc47336b2db" } modify_external_catalogs: true
I20260202 02:52:50.062522 519 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_03cb720f804b44e2b29f8cc47336b2db" } modify_external_catalogs: true
W20260202 02:52:50.063702 857 leader_election.cc:343] T 59ad016e15ec4a488921eb564029b689 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:50.040284 1882 ts_tablet_manager.cc:1950] T fbcb4a522df648ba9542250450c08f1b P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:50.063841 857 leader_election.cc:304] T 59ad016e15ec4a488921eb564029b689 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
W20260202 02:52:50.064229 857 leader_election.cc:343] T abfe7f76b2424c4d9ab07b29b66ec103 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:50.064348 857 leader_election.cc:304] T abfe7f76b2424c4d9ab07b29b66ec103 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:50.064643 2782 raft_consensus.cc:2749] T 59ad016e15ec4a488921eb564029b689 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:50.064836 2782 raft_consensus.cc:2749] T abfe7f76b2424c4d9ab07b29b66ec103 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:50.065471 522 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_a0621e0a3e8047fda1e5226ee396d4e8" } modify_external_catalogs: true
I20260202 02:52:50.065667 522 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_a0621e0a3e8047fda1e5226ee396d4e8" } modify_external_catalogs: true
W20260202 02:52:50.067616 999 leader_election.cc:343] T 842e54adae2a452a98521de24326eaff P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 842e54adae2a452a98521de24326eaff
I20260202 02:52:50.067749 999 leader_election.cc:304] T 842e54adae2a452a98521de24326eaff P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:50.068275 2794 raft_consensus.cc:2749] T 842e54adae2a452a98521de24326eaff P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:50.071660 886 tablet_service.cc:1552] Processing DeleteTablet for tablet a81e1d10ca49459989260fbd6a2b8645 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:50.086818 1182 ts_tablet_manager.cc:1929] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260202 02:52:50.089521 525 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_e6e3448ca5cf4a07a1493a341487187a" } new_table_name: "default.Table_a0dab118c0584cbdbb2926ebba1d91c2" modify_external_catalogs: true
I20260202 02:52:50.089890 1182 log.cc:1199] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/26b890ed87c3471b9fc0b172c90e0a11
W20260202 02:52:50.094781 520 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1b2b4f429b6a484ea6ef7c19124b236b consensus state has no leader
I20260202 02:52:50.095319 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_12cb0a1d830f490dab4efb00188ddecb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:50.095976 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_12cb0a1d830f490dab4efb00188ddecb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:50.096010 521 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6466c63eb97a4a21b29f4f0871e8973c consensus state has no leader
I20260202 02:52:50.100524 825 heartbeater.cc:344] Connected to a master server at 127.0.104.190:33981
I20260202 02:52:50.108039 1180 ts_tablet_manager.cc:1929] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:50.110318 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 714) took 19493 ms. Trace:
I20260202 02:52:50.110569 1882 rpcz_store.cc:276] 0202 02:52:30.616873 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:35.770861 (+5153988us) service_pool.cc:225] Handling call
0202 02:52:49.969069 (+14198208us) ts_tablet_manager.cc:1151] Deleting tablet fbcb4a522df648ba9542250450c08f1b
0202 02:52:49.969079 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:50.032313 (+ 63234us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:50.039793 (+ 7480us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:50.110289 (+ 70496us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":14198109}
I20260202 02:52:50.111162 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 715) took 19494 ms. Trace:
I20260202 02:52:50.111138 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_5215b49fa20a4fff88c538478889acd0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:50.111282 1882 rpcz_store.cc:276] 0202 02:52:30.616968 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:35.770968 (+5154000us) service_pool.cc:225] Handling call
0202 02:52:50.111034 (+14340066us) ts_tablet_manager.cc:1151] Deleting tablet f1fadf2cb5be4ec8980871fc96ee9730
0202 02:52:50.111046 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:50.111147 (+ 101us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":14336238}
I20260202 02:52:50.111603 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 718) took 4407 ms. Trace:
I20260202 02:52:50.111689 1882 rpcz_store.cc:276] 0202 02:52:45.704647 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.704706 (+ 59us) service_pool.cc:225] Handling call
0202 02:52:50.111492 (+4406786us) ts_tablet_manager.cc:1151] Deleting tablet 3dcf4f794ec14b2f9d181d015fc437fb
0202 02:52:50.111500 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:50.111588 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4395577}
W20260202 02:52:50.111780 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5215b49fa20a4fff88c538478889acd0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:50.111963 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 719) took 4399 ms. Trace:
I20260202 02:52:50.112028 1882 rpcz_store.cc:276] 0202 02:52:45.712476 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.712753 (+ 277us) service_pool.cc:225] Handling call
0202 02:52:50.111873 (+4399120us) ts_tablet_manager.cc:1151] Deleting tablet 09a8aa4dd70b4fe2806293d2620e521c
0202 02:52:50.111880 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:50.111950 (+ 70us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4395131}
I20260202 02:52:50.112303 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 720) took 4399 ms. Trace:
I20260202 02:52:50.112389 1882 rpcz_store.cc:276] 0202 02:52:45.712596 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.715950 (+ 3354us) service_pool.cc:225] Handling call
0202 02:52:50.112210 (+4396260us) ts_tablet_manager.cc:1151] Deleting tablet 75a24a1bf54d4c799938a5723ac6fa99
0202 02:52:50.112218 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:50.112288 (+ 70us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4395325}
I20260202 02:52:50.112653 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 721) took 4399 ms. Trace:
I20260202 02:52:50.112710 1882 rpcz_store.cc:276] 0202 02:52:45.712703 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:45.715992 (+ 3289us) service_pool.cc:225] Handling call
0202 02:52:50.112567 (+4396575us) ts_tablet_manager.cc:1151] Deleting tablet b8f2b34b4f1e4ff2b2be4ff40aeb0c66
0202 02:52:50.112575 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:50.112638 (+ 63us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4376861}
I20260202 02:52:50.112923 1882 tablet_replica.cc:333] T f94f25f4059a43b8babedb6b45a08a19 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:50.113088 1882 raft_consensus.cc:2243] T f94f25f4059a43b8babedb6b45a08a19 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:50.113159 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet fbcb4a522df648ba9542250450c08f1b (table default.Table_0cd8f9947fbd430f8992cc1543aed9f7 [id=163eefc3d38a4b28bce88b948cc5df3c]) successfully deleted
I20260202 02:52:50.113230 1882 raft_consensus.cc:2272] T f94f25f4059a43b8babedb6b45a08a19 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:50.113468 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet f1fadf2cb5be4ec8980871fc96ee9730 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f1fadf2cb5be4ec8980871fc96ee9730 already in progress: creating tablet
W20260202 02:52:50.113711 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 3dcf4f794ec14b2f9d181d015fc437fb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3dcf4f794ec14b2f9d181d015fc437fb already in progress: creating tablet
W20260202 02:52:50.113920 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 09a8aa4dd70b4fe2806293d2620e521c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 09a8aa4dd70b4fe2806293d2620e521c already in progress: creating tablet
I20260202 02:52:50.113930 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_7a2b9560ae174e75a43d23cf0e3f708f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:50.114154 1882 ts_tablet_manager.cc:1916] T f94f25f4059a43b8babedb6b45a08a19 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:50.114207 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet b8f2b34b4f1e4ff2b2be4ff40aeb0c66 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b8f2b34b4f1e4ff2b2be4ff40aeb0c66 already in progress: creating tablet
W20260202 02:52:50.114557 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7a2b9560ae174e75a43d23cf0e3f708f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:50.120308 1182 ts_tablet_manager.cc:1950] T 26b890ed87c3471b9fc0b172c90e0a11 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:50.120777 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 777) took 1043 ms. Trace:
I20260202 02:52:50.120872 1182 rpcz_store.cc:276] 0202 02:52:49.077712 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:49.078678 (+ 966us) service_pool.cc:225] Handling call
0202 02:52:49.984540 (+905862us) ts_tablet_manager.cc:1151] Deleting tablet 26b890ed87c3471b9fc0b172c90e0a11
0202 02:52:49.984551 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:50.082472 (+ 97921us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:50.086670 (+ 4198us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:50.120759 (+ 34089us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":905612}
I20260202 02:52:50.121286 1182 tablet_replica.cc:333] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:50.121462 1182 raft_consensus.cc:2243] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:50.121641 1182 pending_rounds.cc:70] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5: Trying to abort 1 pending ops.
I20260202 02:52:50.121574 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 26b890ed87c3471b9fc0b172c90e0a11 (table default.Table_044ef6a2b1af4e3cab3c8cf00505c6a6 [id=d2e4220a0cbf443e8380433da4f064f8]) successfully deleted
I20260202 02:52:50.121677 1182 pending_rounds.cc:77] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5: Aborting op as it isn't in flight: id { term: 2 index: 4 } timestamp: 7249923073148674048 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "28a900bcaa8b41718a285e595886d805" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.Table_911bb209b4554ee8b9bc09edaf488d63" dest_uuid: "1f442ef85029461091de847118c34c0d" new_extra_config { } }
I20260202 02:52:50.122017 1182 raft_consensus.cc:2272] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:50.122660 1182 ts_tablet_manager.cc:1916] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:50.135557 1182 ts_tablet_manager.cc:1929] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.4
I20260202 02:52:50.136981 1182 log.cc:1199] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/28a900bcaa8b41718a285e595886d805
I20260202 02:52:50.137458 1182 ts_tablet_manager.cc:1950] T 28a900bcaa8b41718a285e595886d805 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:50.138183 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 778) took 1055 ms. Trace:
I20260202 02:52:50.138329 1182 rpcz_store.cc:276] 0202 02:52:49.082443 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:49.082498 (+ 55us) service_pool.cc:225] Handling call
0202 02:52:50.121237 (+1038739us) ts_tablet_manager.cc:1151] Deleting tablet 28a900bcaa8b41718a285e595886d805
0202 02:52:50.121246 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:50.131049 (+ 9803us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:50.135437 (+ 4388us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:50.138158 (+ 2721us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1034386}
I20260202 02:52:50.138955 1182 tablet_replica.cc:333] T 65bd1c1a382e42e4a68077264563c55b P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:50.139047 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 28a900bcaa8b41718a285e595886d805 (table default.Table_911bb209b4554ee8b9bc09edaf488d63 [id=cd07f9746ec94d9ebe956a7ca0d19b1c]) successfully deleted
I20260202 02:52:50.139269 1182 raft_consensus.cc:2243] T 65bd1c1a382e42e4a68077264563c55b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:50.139474 1182 raft_consensus.cc:2272] T 65bd1c1a382e42e4a68077264563c55b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:50.140108 1175 tablet_bootstrap.cc:492] T 2b352d9a815e4bccafb7e2251500c5bc P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:50.140125 1182 ts_tablet_manager.cc:1916] T 65bd1c1a382e42e4a68077264563c55b P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:50.140264 1175 ts_tablet_manager.cc:1403] T 2b352d9a815e4bccafb7e2251500c5bc P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.122s user 0.003s sys 0.028s
I20260202 02:52:50.141245 1175 raft_consensus.cc:359] T 2b352d9a815e4bccafb7e2251500c5bc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.141448 1175 raft_consensus.cc:385] T 2b352d9a815e4bccafb7e2251500c5bc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:50.141503 1175 raft_consensus.cc:740] T 2b352d9a815e4bccafb7e2251500c5bc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:50.141721 1175 consensus_queue.cc:260] T 2b352d9a815e4bccafb7e2251500c5bc P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.142423 1175 ts_tablet_manager.cc:1434] T 2b352d9a815e4bccafb7e2251500c5bc P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:50.143134 1175 tablet_bootstrap.cc:492] T 99e0b5e9a9874aec886155e0d614dec2 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:50.144619 1175 tablet_bootstrap.cc:654] T 99e0b5e9a9874aec886155e0d614dec2 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:50.145725 524 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_c0f4096c1e384c37b990a33d78d9cfc0" } modify_external_catalogs: true
I20260202 02:52:50.145951 524 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_c0f4096c1e384c37b990a33d78d9cfc0" } modify_external_catalogs: true
W20260202 02:52:50.150041 423 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 127883us
I20260202 02:52:50.152411 1182 ts_tablet_manager.cc:1929] T 65bd1c1a382e42e4a68077264563c55b P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:50.152801 1182 log.cc:1199] T 65bd1c1a382e42e4a68077264563c55b P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/65bd1c1a382e42e4a68077264563c55b
I20260202 02:52:50.153214 1182 ts_tablet_manager.cc:1950] T 65bd1c1a382e42e4a68077264563c55b P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:50.154551 1175 tablet_bootstrap.cc:492] T 99e0b5e9a9874aec886155e0d614dec2 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:50.154692 1175 ts_tablet_manager.cc:1403] T 99e0b5e9a9874aec886155e0d614dec2 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.000s
I20260202 02:52:50.155355 1175 raft_consensus.cc:359] T 99e0b5e9a9874aec886155e0d614dec2 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.155525 1175 raft_consensus.cc:385] T 99e0b5e9a9874aec886155e0d614dec2 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:50.155599 1175 raft_consensus.cc:740] T 99e0b5e9a9874aec886155e0d614dec2 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:50.157258 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 779) took 1028 ms. Trace:
I20260202 02:52:50.157353 1175 consensus_queue.cc:260] T 99e0b5e9a9874aec886155e0d614dec2 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.157416 1182 rpcz_store.cc:276] 0202 02:52:49.128674 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:49.128742 (+ 68us) service_pool.cc:225] Handling call
0202 02:52:50.138898 (+1010156us) ts_tablet_manager.cc:1151] Deleting tablet 65bd1c1a382e42e4a68077264563c55b
0202 02:52:50.138910 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:50.144116 (+ 5206us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:50.152255 (+ 8139us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:50.157236 (+ 4981us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1009944}
I20260202 02:52:50.158078 1182 tablet_replica.cc:333] T da5435f969074364bc8f8a6ddc5e4f8a P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:50.158126 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 65bd1c1a382e42e4a68077264563c55b (table default.Table_1cc0209a40ad471fb1c41012a476df79 [id=b941bc2edc03444394d1ba8a596a0d27]) successfully deleted
I20260202 02:52:50.158576 1175 ts_tablet_manager.cc:1434] T 99e0b5e9a9874aec886155e0d614dec2 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260202 02:52:50.158723 1182 raft_consensus.cc:2243] T da5435f969074364bc8f8a6ddc5e4f8a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:50.158944 1182 raft_consensus.cc:2272] T da5435f969074364bc8f8a6ddc5e4f8a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:50.159339 1175 tablet_bootstrap.cc:492] T 7ea7a5d8a92b4ac0903483627258ae66 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:50.161072 1175 tablet_bootstrap.cc:654] T 7ea7a5d8a92b4ac0903483627258ae66 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:50.161623 1182 ts_tablet_manager.cc:1916] T da5435f969074364bc8f8a6ddc5e4f8a P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:50.110852 773 raft_consensus.cc:1240] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Rejecting Update request from peer b1f05820b6034228bdad884cc348716d for earlier term 1. Current term is 2. Ops: []
I20260202 02:52:50.129415 2633 raft_consensus.cc:493] T a0b5d82e5fb94b60b4aa324b978a5378 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:50.163072 2633 raft_consensus.cc:515] T a0b5d82e5fb94b60b4aa324b978a5378 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:50.163686 2633 leader_election.cc:290] T a0b5d82e5fb94b60b4aa324b978a5378 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:50.163625 2794 consensus_queue.cc:1059] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260202 02:52:50.136863 1174 tablet_bootstrap.cc:492] T 842e54adae2a452a98521de24326eaff P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:50.163889 1174 ts_tablet_manager.cc:1403] T 842e54adae2a452a98521de24326eaff P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.106s user 0.006s sys 0.003s
I20260202 02:52:50.163928 2794 raft_consensus.cc:3055] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Stepping down as leader of term 1
I20260202 02:52:50.163987 2794 raft_consensus.cc:740] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:50.164215 2794 consensus_queue.cc:260] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:50.164575 2794 raft_consensus.cc:3060] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:50.164677 1174 raft_consensus.cc:359] T 842e54adae2a452a98521de24326eaff P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:50.164839 1174 raft_consensus.cc:385] T 842e54adae2a452a98521de24326eaff P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:50.164884 1174 raft_consensus.cc:740] T 842e54adae2a452a98521de24326eaff P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:50.165063 1174 consensus_queue.cc:260] T 842e54adae2a452a98521de24326eaff P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:50.165490 1174 ts_tablet_manager.cc:1434] T 842e54adae2a452a98521de24326eaff P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:50.165766 1174 tablet_bootstrap.cc:492] T 29c996781cb947f3a1aee5dd0a9c4828 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:50.167109 1174 tablet_bootstrap.cc:654] T 29c996781cb947f3a1aee5dd0a9c4828 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:50.167464 1180 log.cc:1199] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/38fdbc6df7ec40b9afe4e0dfe0119e3e
I20260202 02:52:50.167834 1180 ts_tablet_manager.cc:1950] T 38fdbc6df7ec40b9afe4e0dfe0119e3e P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:50.168700 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 669) took 20902 ms. Trace:
I20260202 02:52:50.168815 1180 rpcz_store.cc:276] 0202 02:52:29.266415 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.268247 (+ 1832us) service_pool.cc:225] Handling call
0202 02:52:50.024386 (+20756139us) ts_tablet_manager.cc:1151] Deleting tablet 38fdbc6df7ec40b9afe4e0dfe0119e3e
0202 02:52:50.024397 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:50.028243 (+ 3846us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:50.107894 (+ 79651us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:50.168679 (+ 60785us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":20755936}
I20260202 02:52:50.169159 1180 tablet_replica.cc:333] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:50.169355 1180 raft_consensus.cc:2243] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:50.171511 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a0b5d82e5fb94b60b4aa324b978a5378" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:50.171733 1045 raft_consensus.cc:2468] T a0b5d82e5fb94b60b4aa324b978a5378 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:50.174736 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 38fdbc6df7ec40b9afe4e0dfe0119e3e (table default.Table_87277c5d80184738b96b773f026b32a5 [id=52687dbd0e554cfb81d88e8fcdd4dd5d]) successfully deleted
I20260202 02:52:50.173339 715 leader_election.cc:304] T a0b5d82e5fb94b60b4aa324b978a5378 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:50.186158 2619 raft_consensus.cc:2804] T a0b5d82e5fb94b60b4aa324b978a5378 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:50.186311 2619 raft_consensus.cc:493] T a0b5d82e5fb94b60b4aa324b978a5378 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:50.186389 2619 raft_consensus.cc:3060] T a0b5d82e5fb94b60b4aa324b978a5378 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:50.189136 714 leader_election.cc:343] T a0b5d82e5fb94b60b4aa324b978a5378 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: a0b5d82e5fb94b60b4aa324b978a5378
I20260202 02:52:50.190898 2619 raft_consensus.cc:515] T a0b5d82e5fb94b60b4aa324b978a5378 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:50.192039 2619 leader_election.cc:290] T a0b5d82e5fb94b60b4aa324b978a5378 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:50.192744 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a0b5d82e5fb94b60b4aa324b978a5378" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:50.192947 1045 raft_consensus.cc:3060] T a0b5d82e5fb94b60b4aa324b978a5378 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:50.194407 1180 pending_rounds.cc:70] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d: Trying to abort 1 pending ops.
I20260202 02:52:50.194473 1180 pending_rounds.cc:77] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923069169709056 op_type: NO_OP noop_request { }
I20260202 02:52:50.194562 1180 raft_consensus.cc:2889] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:50.194641 1180 raft_consensus.cc:2272] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:50.195206 1180 ts_tablet_manager.cc:1916] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:50.195413 1045 raft_consensus.cc:2468] T a0b5d82e5fb94b60b4aa324b978a5378 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:50.196141 715 leader_election.cc:304] T a0b5d82e5fb94b60b4aa324b978a5378 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:50.196448 2633 raft_consensus.cc:2804] T a0b5d82e5fb94b60b4aa324b978a5378 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:50.196560 2633 raft_consensus.cc:697] T a0b5d82e5fb94b60b4aa324b978a5378 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:50.196795 2633 consensus_queue.cc:237] T a0b5d82e5fb94b60b4aa324b978a5378 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:50.197052 714 leader_election.cc:343] T a0b5d82e5fb94b60b4aa324b978a5378 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: a0b5d82e5fb94b60b4aa324b978a5378
I20260202 02:52:50.202708 1174 tablet_bootstrap.cc:492] T 29c996781cb947f3a1aee5dd0a9c4828 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:50.204895 1174 ts_tablet_manager.cc:1403] T 29c996781cb947f3a1aee5dd0a9c4828 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.039s user 0.004s sys 0.004s
I20260202 02:52:50.205847 519 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 25765fb687294e9db1289f76530dd877
I20260202 02:52:50.207136 744 tablet_service.cc:1552] Processing DeleteTablet for tablet 25765fb687294e9db1289f76530dd877 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:50 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:50.207885 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 25765fb687294e9db1289f76530dd877 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:50 UTC) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:50.210842 525 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4453a3e7ef784a528d8683e28d14e604 consensus state has no leader
I20260202 02:52:50.211506 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_131ee47bb13948ae8c0a3e59a88dd689"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:50.211679 1174 raft_consensus.cc:359] T 29c996781cb947f3a1aee5dd0a9c4828 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:50.211855 1174 raft_consensus.cc:385] T 29c996781cb947f3a1aee5dd0a9c4828 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:50.211902 1174 raft_consensus.cc:740] T 29c996781cb947f3a1aee5dd0a9c4828 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:50.211957 1882 ts_tablet_manager.cc:1929] T f94f25f4059a43b8babedb6b45a08a19 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:50.212054 1882 log.cc:1199] T f94f25f4059a43b8babedb6b45a08a19 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/f94f25f4059a43b8babedb6b45a08a19
W20260202 02:52:50.212131 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_131ee47bb13948ae8c0a3e59a88dd689 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:50.212739 1174 consensus_queue.cc:260] T 29c996781cb947f3a1aee5dd0a9c4828 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:50.212973 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_a17341b47cbe40b5863c554e7f56daaf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:50.213521 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a17341b47cbe40b5863c554e7f56daaf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:50.213811 1174 ts_tablet_manager.cc:1434] T 29c996781cb947f3a1aee5dd0a9c4828 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260202 02:52:50.214115 1174 tablet_bootstrap.cc:492] T d8418a9dd0944456b4d4820dde0cde02 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:50.215140 520 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_61d946ae91ff440c9960b0136900fa6e" } new_table_name: "default.Table_81806aedef8d4436a842564190ca8f1d" modify_external_catalogs: true
I20260202 02:52:50.215780 1174 tablet_bootstrap.cc:654] T d8418a9dd0944456b4d4820dde0cde02 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:50.217893 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet 25765fb687294e9db1289f76530dd877 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:50 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:50.219223 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_ca027855a06748118a962c4fdadfd6e8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:50.219727 1182 ts_tablet_manager.cc:1929] T da5435f969074364bc8f8a6ddc5e4f8a P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:50.219869 1182 log.cc:1199] T da5435f969074364bc8f8a6ddc5e4f8a P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/da5435f969074364bc8f8a6ddc5e4f8a
W20260202 02:52:50.219933 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ca027855a06748118a962c4fdadfd6e8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:50.220198 1182 ts_tablet_manager.cc:1950] T da5435f969074364bc8f8a6ddc5e4f8a P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:50.221083 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 780) took 1064 ms. Trace:
I20260202 02:52:50.221169 1182 rpcz_store.cc:276] 0202 02:52:49.156771 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:49.157027 (+ 256us) service_pool.cc:225] Handling call
0202 02:52:50.158024 (+1000997us) ts_tablet_manager.cc:1151] Deleting tablet da5435f969074364bc8f8a6ddc5e4f8a
0202 02:52:50.158035 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:50.183269 (+ 25234us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:50.189656 (+ 6387us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:50.221056 (+ 31400us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":212,"tablet-delete.queue_time_us":997339}
I20260202 02:52:50.222169 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet da5435f969074364bc8f8a6ddc5e4f8a (table default.Table_51648e2a42614c3d9c5820901d153932 [id=ad256e6219784dff9aa916cf07f67fbe]) successfully deleted
I20260202 02:52:50.222743 1182 tablet_replica.cc:333] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:50.222994 1182 raft_consensus.cc:2243] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:50.223413 1182 raft_consensus.cc:2272] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:50.224174 1182 ts_tablet_manager.cc:1916] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:50.236750 523 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 29c996781cb947f3a1aee5dd0a9c4828
I20260202 02:52:50.237219 527 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_1cc0209a40ad471fb1c41012a476df79" } modify_external_catalogs: true
I20260202 02:52:50.237449 527 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_1cc0209a40ad471fb1c41012a476df79" } modify_external_catalogs: true
I20260202 02:52:50.238768 773 raft_consensus.cc:1240] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Rejecting Update request from peer b1f05820b6034228bdad884cc348716d for earlier term 1. Current term is 2. Ops: []
I20260202 02:52:50.239359 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 29c996781cb947f3a1aee5dd0a9c4828 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:50.239480 886 tablet_service.cc:1505] Processing CreateTablet for tablet 1139db8ea6744e168e7b223c23665c58 (DEFAULT_TABLE table=default.Table_c0f4096c1e384c37b990a33d78d9cfc0 [id=657090e094a34feeaa3eb746befa2672]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.239889 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1139db8ea6744e168e7b223c23665c58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.241743 891 tablet_service.cc:1505] Processing CreateTablet for tablet 9bb08b3a18854111a096db52ab566a69 (DEFAULT_TABLE table=default.Table_e28fec940ad64825a9e8eeece80cce05 [id=2ccdd753af8b42168719e7afb7e0b8e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.242132 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bb08b3a18854111a096db52ab566a69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.245079 2740 consensus_queue.cc:1059] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260202 02:52:50.245446 2740 raft_consensus.cc:3055] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Stepping down as leader of term 1
I20260202 02:52:50.245532 2740 raft_consensus.cc:740] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:50.247509 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet 29c996781cb947f3a1aee5dd0a9c4828 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:50.247972 1041 tablet_service.cc:1505] Processing CreateTablet for tablet 1139db8ea6744e168e7b223c23665c58 (DEFAULT_TABLE table=default.Table_c0f4096c1e384c37b990a33d78d9cfc0 [id=657090e094a34feeaa3eb746befa2672]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.248389 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1139db8ea6744e168e7b223c23665c58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.248571 1180 ts_tablet_manager.cc:1929] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260202 02:52:50.256851 514 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 6466c63eb97a4a21b29f4f0871e8973c (table default.Table_8bf4f8ccac504f31893caaad3fc4431d [id=9e3c1e6d5d7b44caa219cb744275d833], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 6466c63eb97a4a21b29f4f0871e8973c consensus state has no leader
I20260202 02:52:50.258461 893 tablet_service.cc:1552] Processing DeleteTablet for tablet 2b352d9a815e4bccafb7e2251500c5bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:22 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:50.261283 1175 tablet_bootstrap.cc:492] T 7ea7a5d8a92b4ac0903483627258ae66 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:50.261437 1175 ts_tablet_manager.cc:1403] T 7ea7a5d8a92b4ac0903483627258ae66 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.102s user 0.018s sys 0.012s
I20260202 02:52:50.262202 1175 raft_consensus.cc:359] T 7ea7a5d8a92b4ac0903483627258ae66 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.262382 1175 raft_consensus.cc:385] T 7ea7a5d8a92b4ac0903483627258ae66 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:50.266178 1175 raft_consensus.cc:740] T 7ea7a5d8a92b4ac0903483627258ae66 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:50.266469 1175 consensus_queue.cc:260] T 7ea7a5d8a92b4ac0903483627258ae66 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.267781 886 tablet_service.cc:1505] Processing CreateTablet for tablet 924dc86358534e2ab494fbb6ecfc304b (DEFAULT_TABLE table=default.Table_c88baa3501324c49a705daab8b694d0c [id=4f3d74dfcb6545739f494624de747674]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.268102 1175 ts_tablet_manager.cc:1434] T 7ea7a5d8a92b4ac0903483627258ae66 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260202 02:52:50.268999 893 tablet_service.cc:1505] Processing CreateTablet for tablet e5029247e9e54fe7a7097f0356abddda (DEFAULT_TABLE table=default.Table_9d1b173cf04349108d4b0b4d810e9cb0 [id=4ce7f79aacbd42228a492227c43cc616]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.269999 1175 tablet_bootstrap.cc:492] T f3eebefa74584c1791d52377f5ba1290 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:50.269011 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 924dc86358534e2ab494fbb6ecfc304b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.271039 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5029247e9e54fe7a7097f0356abddda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.245787 2740 consensus_queue.cc:260] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:50.274346 2740 raft_consensus.cc:3060] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:50.274704 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet a0b5d82e5fb94b60b4aa324b978a5378 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:50.275388 1038 tablet_service.cc:1505] Processing CreateTablet for tablet 924dc86358534e2ab494fbb6ecfc304b (DEFAULT_TABLE table=default.Table_c88baa3501324c49a705daab8b694d0c [id=4f3d74dfcb6545739f494624de747674]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.275755 899 tablet_service.cc:1505] Processing CreateTablet for tablet fa57a267b0484262ac31039862f40a76 (DEFAULT_TABLE table=default.Table_f61ca1b0d7424868993ccd4657d50429 [id=3c38914fb1644aa79f39fb66939ebc92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.276142 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa57a267b0484262ac31039862f40a76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.279546 1035 tablet_service.cc:1505] Processing CreateTablet for tablet e5029247e9e54fe7a7097f0356abddda (DEFAULT_TABLE table=default.Table_9d1b173cf04349108d4b0b4d810e9cb0 [id=4ce7f79aacbd42228a492227c43cc616]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.280092 1036 tablet_service.cc:1505] Processing CreateTablet for tablet 026a2660ee0e4490804cb19182aa318f (DEFAULT_TABLE table=default.Table_5d897c15ea4046ac9869bce1c15a1445 [id=2a0da4ac8cc2456c825ec6c16d83fcbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.275352 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 9bb08b3a18854111a096db52ab566a69 (DEFAULT_TABLE table=default.Table_e28fec940ad64825a9e8eeece80cce05 [id=2ccdd753af8b42168719e7afb7e0b8e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.280853 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 924dc86358534e2ab494fbb6ecfc304b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.281006 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bb08b3a18854111a096db52ab566a69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.281131 891 tablet_service.cc:1505] Processing CreateTablet for tablet 026a2660ee0e4490804cb19182aa318f (DEFAULT_TABLE table=default.Table_5d897c15ea4046ac9869bce1c15a1445 [id=2a0da4ac8cc2456c825ec6c16d83fcbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.281473 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 026a2660ee0e4490804cb19182aa318f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.285636 1033 tablet_service.cc:1505] Processing CreateTablet for tablet fa57a267b0484262ac31039862f40a76 (DEFAULT_TABLE table=default.Table_f61ca1b0d7424868993ccd4657d50429 [id=3c38914fb1644aa79f39fb66939ebc92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.286347 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa57a267b0484262ac31039862f40a76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.288564 1882 ts_tablet_manager.cc:1950] T f94f25f4059a43b8babedb6b45a08a19 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:50.288615 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5029247e9e54fe7a7097f0356abddda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.290371 901 tablet_service.cc:1505] Processing CreateTablet for tablet fc8d1895cf0a4e838c473fb18f908770 (DEFAULT_TABLE table=default.Table_cda21f6dd8d34e47abae4a73c38b597c [id=4ef762b02b7c4604975d280238d67615]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.290786 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc8d1895cf0a4e838c473fb18f908770. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.291332 886 tablet_service.cc:1505] Processing CreateTablet for tablet 8f69ed5fd2cd454d948dd1a525faad00 (DEFAULT_TABLE table=default.Table_b40b893658b34e4593e3eb6f6af5b829 [id=8242640a462148cfa6bd4e16bc1869b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.292572 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f69ed5fd2cd454d948dd1a525faad00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.294425 892 tablet_service.cc:1505] Processing CreateTablet for tablet 7d45eb2e11ed4974b9a803e953b42e40 (DEFAULT_TABLE table=default.Table_a486c34ee08d4e55978c0e884529329b [id=a21a6ddf72d645318a1ce7e634e13eca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.294778 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d45eb2e11ed4974b9a803e953b42e40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.295423 1041 tablet_service.cc:1505] Processing CreateTablet for tablet fc8d1895cf0a4e838c473fb18f908770 (DEFAULT_TABLE table=default.Table_cda21f6dd8d34e47abae4a73c38b597c [id=4ef762b02b7c4604975d280238d67615]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.295833 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc8d1895cf0a4e838c473fb18f908770. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.297977 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 735) took 2871 ms. Trace:
I20260202 02:52:50.298076 1882 rpcz_store.cc:276] 0202 02:52:47.426621 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:47.736552 (+309931us) service_pool.cc:225] Handling call
0202 02:52:50.112883 (+2376331us) ts_tablet_manager.cc:1151] Deleting tablet f94f25f4059a43b8babedb6b45a08a19
0202 02:52:50.112891 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:50.208375 (+ 95484us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:50.211835 (+ 3460us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:50.297951 (+ 86116us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2376132}
I20260202 02:52:50.298475 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 736) took 2336 ms. Trace:
I20260202 02:52:50.298548 1882 rpcz_store.cc:276] 0202 02:52:47.962410 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:47.962483 (+ 73us) service_pool.cc:225] Handling call
0202 02:52:50.298349 (+2335866us) ts_tablet_manager.cc:1151] Deleting tablet abfe7f76b2424c4d9ab07b29b66ec103
0202 02:52:50.298361 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:50.298457 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2248795}
I20260202 02:52:50.298823 1882 tablet_replica.cc:333] T 842e54adae2a452a98521de24326eaff P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:50.298848 1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 026a2660ee0e4490804cb19182aa318f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.299144 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet f94f25f4059a43b8babedb6b45a08a19 (table default.Table_22445ccfd7b64298b92a0f70733f3625 [id=300ea019d2724bee8211d7462710dc26]) successfully deleted
W20260202 02:52:50.299510 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet abfe7f76b2424c4d9ab07b29b66ec103 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet abfe7f76b2424c4d9ab07b29b66ec103 already in progress: creating tablet
I20260202 02:52:50.299428 1038 tablet_service.cc:1505] Processing CreateTablet for tablet 8f69ed5fd2cd454d948dd1a525faad00 (DEFAULT_TABLE table=default.Table_b40b893658b34e4593e3eb6f6af5b829 [id=8242640a462148cfa6bd4e16bc1869b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.300514 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 7d45eb2e11ed4974b9a803e953b42e40 (DEFAULT_TABLE table=default.Table_a486c34ee08d4e55978c0e884529329b [id=a21a6ddf72d645318a1ce7e634e13eca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.302181 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_c783f2691903413c90a5e8502f5ddffa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:50.302814 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c783f2691903413c90a5e8502f5ddffa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:50.303462 522 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 39eaa2848cf4450dbe563ddd3b11df06
I20260202 02:52:50.304195 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_86f302ed9e76457888b61469aa6b2e7f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:50.304581 528 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_fbe3212129904307af7133ead4053a59" } new_table_name: "default.Table_0d3570fd2973408a96cbbeb133b39355" modify_external_catalogs: true
W20260202 02:52:50.304656 520 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 49dfbf0149874e1d904b78913daf2c84 consensus state has no leader
W20260202 02:52:50.304874 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_86f302ed9e76457888b61469aa6b2e7f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:50.305012 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_d866f96ecc014b50a4c39f4f93653c9e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:50.305445 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d866f96ecc014b50a4c39f4f93653c9e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:50.304412 521 master_service.cc:954] ReplaceTablet: received request to replace tablet e04960479c92479b8bb864544dd0c8ac from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:50.306134 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_1f4c697ec6ea43519f5330b2274557f3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:50.306612 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1f4c697ec6ea43519f5330b2274557f3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:50.310357 519 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_aee0b3d530954ca3ba6492f8263dd7ff" } new_table_name: "default.Table_cd2f525ec9b7457fa8c8a42370f00c43" modify_external_catalogs: true
I20260202 02:52:50.310433 898 tablet_service.cc:1505] Processing CreateTablet for tablet b8bfa4fd24e6466bb1aa3cd9107ccd82 (DEFAULT_TABLE table=default.Table_a0f2c4c878d9452a8de3ba0040b72195 [id=5a06d7acb4554f85b9c2def17fdba04d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.310678 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 7ea7a5d8a92b4ac0903483627258ae66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:50.310820 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8bfa4fd24e6466bb1aa3cd9107ccd82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.310900 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 39eaa2848cf4450dbe563ddd3b11df06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:50 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:50.311223 903 tablet_service.cc:1505] Processing CreateTablet for tablet 11610f2d5220411aa28b47d2dd724f00 (DEFAULT_TABLE table=default.Table_6eea808cf1ea46998a599819ee2cbd5c [id=5e3c0659d49a4e698d7ab0ff80dfe69a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.311578 903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11610f2d5220411aa28b47d2dd724f00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.314391 896 tablet_service.cc:1505] Processing CreateTablet for tablet fab08cafb1fa46709030f2b036a64899 (DEFAULT_TABLE table=default.Table_4e00e3aee35945da8f96cd16d666323a [id=a32c5d853bce42a9ab17cdf6fb074e1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.314787 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fab08cafb1fa46709030f2b036a64899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.316651 895 tablet_service.cc:1505] Processing CreateTablet for tablet 2becb2f65a124c3990ea3a8318ff1d8c (DEFAULT_TABLE table=default.Table_2d519304c313459a8b38c0fe0c65d21f [id=022bce98e93d4821a275d6a26dd033aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.317042 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2becb2f65a124c3990ea3a8318ff1d8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.317461 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f69ed5fd2cd454d948dd1a525faad00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.317971 1182 ts_tablet_manager.cc:1929] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20260202 02:52:50.320055 1182 log.cc:1199] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/80eb3013f5354022a2dd78bb294a8497
I20260202 02:52:50.320638 1182 ts_tablet_manager.cc:1950] T 80eb3013f5354022a2dd78bb294a8497 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:50.321110 889 tablet_service.cc:1505] Processing CreateTablet for tablet 97d6313d74ba4fe09d46d0fa5b4fe71c (DEFAULT_TABLE table=default.Table_b308f5c7056f4abda5b3c64a7a4173d3 [id=91a16261163c4e7a9645bca63a9b246e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.321486 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97d6313d74ba4fe09d46d0fa5b4fe71c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.321650 886 tablet_service.cc:1505] Processing CreateTablet for tablet 05567ae398ee4666ab26ac98ed8b7777 (DEFAULT_TABLE table=default.Table_f02b61a92b1e4552968931092b466a76 [id=e9f96615c3f04dfdab83eefb8be22eba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.322055 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05567ae398ee4666ab26ac98ed8b7777. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.325055 1029 tablet_service.cc:1505] Processing CreateTablet for tablet b8bfa4fd24e6466bb1aa3cd9107ccd82 (DEFAULT_TABLE table=default.Table_a0f2c4c878d9452a8de3ba0040b72195 [id=5a06d7acb4554f85b9c2def17fdba04d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.325438 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8bfa4fd24e6466bb1aa3cd9107ccd82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.327591 894 tablet_service.cc:1505] Processing CreateTablet for tablet 87670d9fd1e845ed937b6435c4d622b7 (DEFAULT_TABLE table=default.Table_645e1697bb644d629ba1c4b4f6c5ae33 [id=2efc04d5045348f8b5e8cad1a2b614e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.327926 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 781) took 1038 ms. Trace:
I20260202 02:52:50.328028 887 tablet_service.cc:1505] Processing CreateTablet for tablet 31d992fe8cbb4632829a976397d54e7c (DEFAULT_TABLE table=default.Table_dd3beb3a0d7147db9f5e8739810746df [id=dce837bc9c12417582e3c61267d263bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.328073 1182 rpcz_store.cc:276] 0202 02:52:49.289479 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:49.289547 (+ 68us) service_pool.cc:225] Handling call
0202 02:52:50.222685 (+933138us) ts_tablet_manager.cc:1151] Deleting tablet 80eb3013f5354022a2dd78bb294a8497
0202 02:52:50.222698 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:50.230950 (+ 8252us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:50.317890 (+ 86940us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:50.327904 (+ 10014us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":922161}
I20260202 02:52:50.328599 888 tablet_service.cc:1505] Processing CreateTablet for tablet 2a59be2615b54d6c921b9cfa13f01c04 (DEFAULT_TABLE table=default.Table_95f6cd1339444962a8731fb7629fce47 [id=7a8906ffadc649449e359be10128fd55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.329140 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 80eb3013f5354022a2dd78bb294a8497 (table default.Table_4e4d461d49574c68a7d93ac891db67c6 [id=316b8a276d9b415fafe8511aa7c32fc5]) successfully deleted
I20260202 02:52:50.332998 885 tablet_service.cc:1505] Processing CreateTablet for tablet 9c2be6ba0d7448c8a6096148230ca891 (DEFAULT_TABLE table=default.Table_9e2f89fa77414b56a2ebd040c0e80085 [id=a26ed8d436fa436f802e3ad110766f98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.333078 900 tablet_service.cc:1505] Processing CreateTablet for tablet 8457781dd88e4a9c8e343797cc781e22 (DEFAULT_TABLE table=default.Table_7655d1ccf43246cfbeaf0c7b30fc5d32 [id=49c21c897521434c922d18a65b4e2c45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.333407 897 tablet_service.cc:1505] Processing CreateTablet for tablet b9bcbc1527e748ee9dcb4912b050407b (DEFAULT_TABLE table=default.Table_edb640fc9a204021b59ae38f17059e3b [id=650459d6101946a7bc70513f09fba73a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.335098 902 tablet_service.cc:1505] Processing CreateTablet for tablet 8cc5e23a86a3467c89794ae8c403230e (DEFAULT_TABLE table=default.Table_988dd0b38bd141bf8cba853c4d6685b0 [id=d32c18fb23514443a7774a6bb56030ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.335501 884 tablet_service.cc:1505] Processing CreateTablet for tablet 52ea22e5add5480fb2b1ab803ce01844 (DEFAULT_TABLE table=default.Table_cf40ea51a80d469e9cacf1e520c28248 [id=922575e63e734afbb50aa15dbee53e54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.335098 890 tablet_service.cc:1505] Processing CreateTablet for tablet 7a4fc28a9fc54ba7a6cf5dd425773d60 (DEFAULT_TABLE table=default.Table_ee8db454beae4d16a36f06dab8f41364 [id=e63ad20f8d97429497c7ccd0c2b23e39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.336239 1034 tablet_service.cc:1552] Processing DeleteTablet for tablet 39eaa2848cf4450dbe563ddd3b11df06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:50 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:50.337179 1040 tablet_service.cc:1505] Processing CreateTablet for tablet 2becb2f65a124c3990ea3a8318ff1d8c (DEFAULT_TABLE table=default.Table_2d519304c313459a8b38c0fe0c65d21f [id=022bce98e93d4821a275d6a26dd033aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.337627 1034 tablet_service.cc:1505] Processing CreateTablet for tablet 11610f2d5220411aa28b47d2dd724f00 (DEFAULT_TABLE table=default.Table_6eea808cf1ea46998a599819ee2cbd5c [id=5e3c0659d49a4e698d7ab0ff80dfe69a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.337639 1042 tablet_service.cc:1505] Processing CreateTablet for tablet 97d6313d74ba4fe09d46d0fa5b4fe71c (DEFAULT_TABLE table=default.Table_b308f5c7056f4abda5b3c64a7a4173d3 [id=91a16261163c4e7a9645bca63a9b246e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.337807 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 05567ae398ee4666ab26ac98ed8b7777 (DEFAULT_TABLE table=default.Table_f02b61a92b1e4552968931092b466a76 [id=e9f96615c3f04dfdab83eefb8be22eba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.338480 1030 tablet_service.cc:1505] Processing CreateTablet for tablet 31d992fe8cbb4632829a976397d54e7c (DEFAULT_TABLE table=default.Table_dd3beb3a0d7147db9f5e8739810746df [id=dce837bc9c12417582e3c61267d263bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.338564 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d45eb2e11ed4974b9a803e953b42e40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.340983 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05567ae398ee4666ab26ac98ed8b7777. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.341217 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31d992fe8cbb4632829a976397d54e7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.341450 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2becb2f65a124c3990ea3a8318ff1d8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.342366 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97d6313d74ba4fe09d46d0fa5b4fe71c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.343676 1039 tablet_service.cc:1505] Processing CreateTablet for tablet 2a59be2615b54d6c921b9cfa13f01c04 (DEFAULT_TABLE table=default.Table_95f6cd1339444962a8731fb7629fce47 [id=7a8906ffadc649449e359be10128fd55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.343958 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11610f2d5220411aa28b47d2dd724f00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.344118 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a59be2615b54d6c921b9cfa13f01c04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.344445 1032 tablet_service.cc:1505] Processing CreateTablet for tablet b9bcbc1527e748ee9dcb4912b050407b (DEFAULT_TABLE table=default.Table_edb640fc9a204021b59ae38f17059e3b [id=650459d6101946a7bc70513f09fba73a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.345371 1031 tablet_service.cc:1505] Processing CreateTablet for tablet 8cc5e23a86a3467c89794ae8c403230e (DEFAULT_TABLE table=default.Table_988dd0b38bd141bf8cba853c4d6685b0 [id=d32c18fb23514443a7774a6bb56030ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.345696 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cc5e23a86a3467c89794ae8c403230e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.345676 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 8457781dd88e4a9c8e343797cc781e22 (DEFAULT_TABLE table=default.Table_7655d1ccf43246cfbeaf0c7b30fc5d32 [id=49c21c897521434c922d18a65b4e2c45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.345984 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8457781dd88e4a9c8e343797cc781e22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.346740 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9bcbc1527e748ee9dcb4912b050407b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.348676 1043 tablet_service.cc:1505] Processing CreateTablet for tablet 87670d9fd1e845ed937b6435c4d622b7 (DEFAULT_TABLE table=default.Table_645e1697bb644d629ba1c4b4f6c5ae33 [id=2efc04d5045348f8b5e8cad1a2b614e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.251065 749 tablet_service.cc:1505] Processing CreateTablet for tablet 1139db8ea6744e168e7b223c23665c58 (DEFAULT_TABLE table=default.Table_c0f4096c1e384c37b990a33d78d9cfc0 [id=657090e094a34feeaa3eb746befa2672]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.349007 1180 log.cc:1199] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/38301e13e17c41fcae912c4b38c26a45
I20260202 02:52:50.349277 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1139db8ea6744e168e7b223c23665c58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.349401 1180 ts_tablet_manager.cc:1950] T 38301e13e17c41fcae912c4b38c26a45 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:50.349676 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 668) took 21083 ms. Trace:
I20260202 02:52:50.349759 1180 rpcz_store.cc:276] 0202 02:52:29.266243 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.268526 (+ 2283us) service_pool.cc:225] Handling call
0202 02:52:50.169115 (+20900589us) ts_tablet_manager.cc:1151] Deleting tablet 38301e13e17c41fcae912c4b38c26a45
0202 02:52:50.169123 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:50.245414 (+ 76291us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:50.248461 (+ 3047us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:50.349658 (+101197us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":20900422}
I20260202 02:52:50.350198 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 38301e13e17c41fcae912c4b38c26a45 (table default.Table_80912cc76f934f139da7f866546ae97a [id=2f3bf7e0cf4c41ac8a2554360b57214a]) successfully deleted
I20260202 02:52:50.351655 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87670d9fd1e845ed937b6435c4d622b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.351850 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a59be2615b54d6c921b9cfa13f01c04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.264205 748 tablet_service.cc:1505] Processing CreateTablet for tablet 9bb08b3a18854111a096db52ab566a69 (DEFAULT_TABLE table=default.Table_e28fec940ad64825a9e8eeece80cce05 [id=2ccdd753af8b42168719e7afb7e0b8e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.267390 743 tablet_service.cc:1505] Processing CreateTablet for tablet 026a2660ee0e4490804cb19182aa318f (DEFAULT_TABLE table=default.Table_5d897c15ea4046ac9869bce1c15a1445 [id=2a0da4ac8cc2456c825ec6c16d83fcbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.268932 757 tablet_service.cc:1505] Processing CreateTablet for tablet 924dc86358534e2ab494fbb6ecfc304b (DEFAULT_TABLE table=default.Table_c88baa3501324c49a705daab8b694d0c [id=4f3d74dfcb6545739f494624de747674]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.272648 741 tablet_service.cc:1505] Processing CreateTablet for tablet e5029247e9e54fe7a7097f0356abddda (DEFAULT_TABLE table=default.Table_9d1b173cf04349108d4b0b4d810e9cb0 [id=4ce7f79aacbd42228a492227c43cc616]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.352699 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bb08b3a18854111a096db52ab566a69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.352854 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5029247e9e54fe7a7097f0356abddda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.355659 743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 026a2660ee0e4490804cb19182aa318f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.356454 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31d992fe8cbb4632829a976397d54e7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.280794 744 tablet_service.cc:1552] Processing DeleteTablet for tablet 29c996781cb947f3a1aee5dd0a9c4828 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:50.356948 744 tablet_service.cc:1505] Processing CreateTablet for tablet 8457781dd88e4a9c8e343797cc781e22 (DEFAULT_TABLE table=default.Table_7655d1ccf43246cfbeaf0c7b30fc5d32 [id=49c21c897521434c922d18a65b4e2c45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.357314 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8457781dd88e4a9c8e343797cc781e22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.357648 894 tablet_service.cc:1505] Processing CreateTablet for tablet e5ab9384afe14e4a875be18ffdb863c3 (DEFAULT_TABLE table=default.Table_044ef6a2b1af4e3cab3c8cf00505c6a6 [id=d2e4220a0cbf443e8380433da4f064f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.357998 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5ab9384afe14e4a875be18ffdb863c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.288312 754 tablet_service.cc:1505] Processing CreateTablet for tablet fa57a267b0484262ac31039862f40a76 (DEFAULT_TABLE table=default.Table_f61ca1b0d7424868993ccd4657d50429 [id=3c38914fb1644aa79f39fb66939ebc92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.360461 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 924dc86358534e2ab494fbb6ecfc304b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.360659 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa57a267b0484262ac31039862f40a76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.289286 751 tablet_service.cc:1505] Processing CreateTablet for tablet 8f69ed5fd2cd454d948dd1a525faad00 (DEFAULT_TABLE table=default.Table_b40b893658b34e4593e3eb6f6af5b829 [id=8242640a462148cfa6bd4e16bc1869b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.292553 758 tablet_service.cc:1505] Processing CreateTablet for tablet fc8d1895cf0a4e838c473fb18f908770 (DEFAULT_TABLE table=default.Table_cda21f6dd8d34e47abae4a73c38b597c [id=4ef762b02b7c4604975d280238d67615]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.363826 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f69ed5fd2cd454d948dd1a525faad00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.364116 1028 tablet_service.cc:1505] Processing CreateTablet for tablet 7a4fc28a9fc54ba7a6cf5dd425773d60 (DEFAULT_TABLE table=default.Table_ee8db454beae4d16a36f06dab8f41364 [id=e63ad20f8d97429497c7ccd0c2b23e39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.364529 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a4fc28a9fc54ba7a6cf5dd425773d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.367063 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc8d1895cf0a4e838c473fb18f908770. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.367956 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 52ea22e5add5480fb2b1ab803ce01844 (DEFAULT_TABLE table=default.Table_cf40ea51a80d469e9cacf1e520c28248 [id=922575e63e734afbb50aa15dbee53e54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.368714 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52ea22e5add5480fb2b1ab803ce01844. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.369176 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 9c2be6ba0d7448c8a6096148230ca891 (DEFAULT_TABLE table=default.Table_9e2f89fa77414b56a2ebd040c0e80085 [id=a26ed8d436fa436f802e3ad110766f98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.369506 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c2be6ba0d7448c8a6096148230ca891. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.307139 756 tablet_service.cc:1505] Processing CreateTablet for tablet b8bfa4fd24e6466bb1aa3cd9107ccd82 (DEFAULT_TABLE table=default.Table_a0f2c4c878d9452a8de3ba0040b72195 [id=5a06d7acb4554f85b9c2def17fdba04d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.370646 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8bfa4fd24e6466bb1aa3cd9107ccd82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.308494 742 tablet_service.cc:1505] Processing CreateTablet for tablet 7d45eb2e11ed4974b9a803e953b42e40 (DEFAULT_TABLE table=default.Table_a486c34ee08d4e55978c0e884529329b [id=a21a6ddf72d645318a1ce7e634e13eca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.374254 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d45eb2e11ed4974b9a803e953b42e40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.314551 1174 tablet_bootstrap.cc:492] T d8418a9dd0944456b4d4820dde0cde02 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:50.376506 1174 ts_tablet_manager.cc:1403] T d8418a9dd0944456b4d4820dde0cde02 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.162s user 0.003s sys 0.003s
I20260202 02:52:50.377157 1182 tablet_replica.cc:333] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:50.377399 1174 raft_consensus.cc:359] T d8418a9dd0944456b4d4820dde0cde02 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:50.377573 1174 raft_consensus.cc:385] T d8418a9dd0944456b4d4820dde0cde02 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:50.377625 1174 raft_consensus.cc:740] T d8418a9dd0944456b4d4820dde0cde02 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:50.377822 1174 consensus_queue.cc:260] T d8418a9dd0944456b4d4820dde0cde02 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:50.378135 1182 raft_consensus.cc:2243] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:50.378312 1174 ts_tablet_manager.cc:1434] T d8418a9dd0944456b4d4820dde0cde02 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:50.378623 1174 tablet_bootstrap.cc:492] T 550895b4010f4088b2cc34acff50e03f P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:50.381264 1182 raft_consensus.cc:2272] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:50.323029 740 tablet_service.cc:1552] Processing DeleteTablet for tablet 39eaa2848cf4450dbe563ddd3b11df06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:50 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:50.382768 740 tablet_service.cc:1505] Processing CreateTablet for tablet 9c2be6ba0d7448c8a6096148230ca891 (DEFAULT_TABLE table=default.Table_9e2f89fa77414b56a2ebd040c0e80085 [id=a26ed8d436fa436f802e3ad110766f98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.382962 1175 tablet_bootstrap.cc:654] T f3eebefa74584c1791d52377f5ba1290 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:50.383221 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c2be6ba0d7448c8a6096148230ca891. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.383451 1182 ts_tablet_manager.cc:1916] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:50.323156 759 tablet_service.cc:1505] Processing CreateTablet for tablet 97d6313d74ba4fe09d46d0fa5b4fe71c (DEFAULT_TABLE table=default.Table_b308f5c7056f4abda5b3c64a7a4173d3 [id=91a16261163c4e7a9645bca63a9b246e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.388754 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97d6313d74ba4fe09d46d0fa5b4fe71c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.392624 2619 raft_consensus.cc:493] T fbcb4a522df648ba9542250450c08f1b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:50.392817 2619 raft_consensus.cc:515] T fbcb4a522df648ba9542250450c08f1b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:50.393517 2619 leader_election.cc:290] T fbcb4a522df648ba9542250450c08f1b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:50.323316 753 tablet_service.cc:1505] Processing CreateTablet for tablet fab08cafb1fa46709030f2b036a64899 (DEFAULT_TABLE table=default.Table_4e00e3aee35945da8f96cd16d666323a [id=a32c5d853bce42a9ab17cdf6fb074e1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.393957 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fab08cafb1fa46709030f2b036a64899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.323611 750 tablet_service.cc:1505] Processing CreateTablet for tablet 11610f2d5220411aa28b47d2dd724f00 (DEFAULT_TABLE table=default.Table_6eea808cf1ea46998a599819ee2cbd5c [id=5e3c0659d49a4e698d7ab0ff80dfe69a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.396392 1182 ts_tablet_manager.cc:1929] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:50.396507 1182 log.cc:1199] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/73f8db8f30f14d369d2227c9be7c4647
I20260202 02:52:50.396688 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11610f2d5220411aa28b47d2dd724f00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.396884 1182 ts_tablet_manager.cc:1950] T 73f8db8f30f14d369d2227c9be7c4647 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:50.397392 1175 tablet_bootstrap.cc:492] T f3eebefa74584c1791d52377f5ba1290 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:50.397589 1175 ts_tablet_manager.cc:1403] T f3eebefa74584c1791d52377f5ba1290 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.128s user 0.002s sys 0.001s
I20260202 02:52:50.398320 1175 raft_consensus.cc:359] T f3eebefa74584c1791d52377f5ba1290 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.398492 1175 raft_consensus.cc:385] T f3eebefa74584c1791d52377f5ba1290 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:50.398550 1175 raft_consensus.cc:740] T f3eebefa74584c1791d52377f5ba1290 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:50.398761 1175 consensus_queue.cc:260] T f3eebefa74584c1791d52377f5ba1290 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.323899 746 tablet_service.cc:1505] Processing CreateTablet for tablet 2becb2f65a124c3990ea3a8318ff1d8c (DEFAULT_TABLE table=default.Table_2d519304c313459a8b38c0fe0c65d21f [id=022bce98e93d4821a275d6a26dd033aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.400262 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2becb2f65a124c3990ea3a8318ff1d8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.324184 745 tablet_service.cc:1505] Processing CreateTablet for tablet 05567ae398ee4666ab26ac98ed8b7777 (DEFAULT_TABLE table=default.Table_f02b61a92b1e4552968931092b466a76 [id=e9f96615c3f04dfdab83eefb8be22eba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.329124 747 tablet_service.cc:1505] Processing CreateTablet for tablet 31d992fe8cbb4632829a976397d54e7c (DEFAULT_TABLE table=default.Table_dd3beb3a0d7147db9f5e8739810746df [id=dce837bc9c12417582e3c61267d263bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.329417 755 tablet_service.cc:1505] Processing CreateTablet for tablet 87670d9fd1e845ed937b6435c4d622b7 (DEFAULT_TABLE table=default.Table_645e1697bb644d629ba1c4b4f6c5ae33 [id=2efc04d5045348f8b5e8cad1a2b614e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.400614 886 tablet_service.cc:1505] Processing CreateTablet for tablet fbb87b118b4b4df683ae0063bc2d77a4 (DEFAULT_TABLE table=default.Table_ebd39aa54af54c0e9b4efe9b5d986034 [id=e2f5c92d69e14b5db0761eed0e9d8352]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.330902 752 tablet_service.cc:1505] Processing CreateTablet for tablet 2a59be2615b54d6c921b9cfa13f01c04 (DEFAULT_TABLE table=default.Table_95f6cd1339444962a8731fb7629fce47 [id=7a8906ffadc649449e359be10128fd55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.400864 1026 tablet_service.cc:1505] Processing CreateTablet for tablet e5ab9384afe14e4a875be18ffdb863c3 (DEFAULT_TABLE table=default.Table_044ef6a2b1af4e3cab3c8cf00505c6a6 [id=d2e4220a0cbf443e8380433da4f064f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.400970 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbb87b118b4b4df683ae0063bc2d77a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.401217 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5ab9384afe14e4a875be18ffdb863c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.401144 1040 tablet_service.cc:1505] Processing CreateTablet for tablet fbb87b118b4b4df683ae0063bc2d77a4 (DEFAULT_TABLE table=default.Table_ebd39aa54af54c0e9b4efe9b5d986034 [id=e2f5c92d69e14b5db0761eed0e9d8352]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.401690 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbb87b118b4b4df683ae0063bc2d77a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.401872 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c2be6ba0d7448c8a6096148230ca891. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.404603 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8457781dd88e4a9c8e343797cc781e22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.337366 1025 tablet_service.cc:1505] Processing CreateTablet for tablet fab08cafb1fa46709030f2b036a64899 (DEFAULT_TABLE table=default.Table_4e00e3aee35945da8f96cd16d666323a [id=a32c5d853bce42a9ab17cdf6fb074e1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.405865 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fab08cafb1fa46709030f2b036a64899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.406134 1175 ts_tablet_manager.cc:1434] T f3eebefa74584c1791d52377f5ba1290 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260202 02:52:50.406489 1175 tablet_bootstrap.cc:492] T 2f5aae089fcc492f9ce55f84674a452c P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:50.408578 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9bcbc1527e748ee9dcb4912b050407b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.413151 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87670d9fd1e845ed937b6435c4d622b7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:50.420643 715 leader_election.cc:343] T fbcb4a522df648ba9542250450c08f1b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Not found: Tablet not found: fbcb4a522df648ba9542250450c08f1b
I20260202 02:52:50.424219 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05567ae398ee4666ab26ac98ed8b7777. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.424553 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a59be2615b54d6c921b9cfa13f01c04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.424711 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31d992fe8cbb4632829a976397d54e7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.427855 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87670d9fd1e845ed937b6435c4d622b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.427784 1037 tablet_service.cc:1505] Processing CreateTablet for tablet fea0fcb8854c4fc6acda80c8ffb8d341 (DEFAULT_TABLE table=default.Table_aee0b3d530954ca3ba6492f8263dd7ff [id=384621c2a9d94b0db7a7928a3bd8fffc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.428162 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fea0fcb8854c4fc6acda80c8ffb8d341. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.428195 889 tablet_service.cc:1505] Processing CreateTablet for tablet fea0fcb8854c4fc6acda80c8ffb8d341 (DEFAULT_TABLE table=default.Table_aee0b3d530954ca3ba6492f8263dd7ff [id=384621c2a9d94b0db7a7928a3bd8fffc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.428586 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fea0fcb8854c4fc6acda80c8ffb8d341. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.397603 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 782) took 1089 ms. Trace:
I20260202 02:52:50.435760 1182 rpcz_store.cc:276] 0202 02:52:49.307775 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:49.307833 (+ 58us) service_pool.cc:225] Handling call
0202 02:52:50.329173 (+1021340us) ts_tablet_manager.cc:1151] Deleting tablet 73f8db8f30f14d369d2227c9be7c4647
0202 02:52:50.377077 (+ 47904us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:50.386609 (+ 9532us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:50.396247 (+ 9638us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:50.397580 (+ 1333us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":995453}
I20260202 02:52:50.436913 1182 tablet_replica.cc:333] T 59ad016e15ec4a488921eb564029b689 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:50.437094 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 73f8db8f30f14d369d2227c9be7c4647 (table default.Table_0276ce9d8aaa44fa832bb88a8b1f9446 [id=a8eac3717bad4d1496acd1c857a510a6]) successfully deleted
I20260202 02:52:50.437433 1182 raft_consensus.cc:2243] T 59ad016e15ec4a488921eb564029b689 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:50.438805 1182 raft_consensus.cc:2272] T 59ad016e15ec4a488921eb564029b689 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:50.438967 892 tablet_service.cc:1505] Processing CreateTablet for tablet 0e0c43f15ba2437fb227a153241b4394 (DEFAULT_TABLE table=default.Table_1cbc82d8edee4ef58a1beda6efdca8bc [id=c754ee7b91d54c66871deab39f3fd834]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.441244 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e0c43f15ba2437fb227a153241b4394. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.442117 1173 tablet_bootstrap.cc:492] T 44c8d5b74c8e4c499f7f68e8185efc17 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:50.442261 1173 ts_tablet_manager.cc:1403] T 44c8d5b74c8e4c499f7f68e8185efc17 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.387s user 0.031s sys 0.041s
I20260202 02:52:50.443029 1173 raft_consensus.cc:359] T 44c8d5b74c8e4c499f7f68e8185efc17 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:50.443188 1173 raft_consensus.cc:385] T 44c8d5b74c8e4c499f7f68e8185efc17 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:50.443228 1173 raft_consensus.cc:740] T 44c8d5b74c8e4c499f7f68e8185efc17 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:50.443410 1173 consensus_queue.cc:260] T 44c8d5b74c8e4c499f7f68e8185efc17 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:50.447768 1173 ts_tablet_manager.cc:1434] T 44c8d5b74c8e4c499f7f68e8185efc17 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260202 02:52:50.448088 1173 tablet_bootstrap.cc:492] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:50.449580 1173 tablet_bootstrap.cc:654] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:50.454452 969 heartbeater.cc:344] Connected to a master server at 127.0.104.190:33981
W20260202 02:52:50.456398 714 leader_election.cc:343] T fbcb4a522df648ba9542250450c08f1b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: fbcb4a522df648ba9542250450c08f1b
I20260202 02:52:50.456557 714 leader_election.cc:304] T fbcb4a522df648ba9542250450c08f1b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:50.456817 2619 raft_consensus.cc:2749] T fbcb4a522df648ba9542250450c08f1b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:50.458384 1175 tablet_bootstrap.cc:654] T 2f5aae089fcc492f9ce55f84674a452c P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:50.461138 1174 tablet_bootstrap.cc:654] T 550895b4010f4088b2cc34acff50e03f P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:50.462049 1182 ts_tablet_manager.cc:1916] T 59ad016e15ec4a488921eb564029b689 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:50.463279 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 0e0c43f15ba2437fb227a153241b4394 (DEFAULT_TABLE table=default.Table_1cbc82d8edee4ef58a1beda6efdca8bc [id=c754ee7b91d54c66871deab39f3fd834]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.465094 1035 tablet_service.cc:1505] Processing CreateTablet for tablet b00b4a0b85374bfe83928162c4da6479 (DEFAULT_TABLE table=default.Table_619bb68d71834b3a9f5e0b2b0fdaed09 [id=616ea3074afc41bdae8f70c0d38bb591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.468135 757 tablet_service.cc:1505] Processing CreateTablet for tablet b9bcbc1527e748ee9dcb4912b050407b (DEFAULT_TABLE table=default.Table_edb640fc9a204021b59ae38f17059e3b [id=650459d6101946a7bc70513f09fba73a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.468576 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9bcbc1527e748ee9dcb4912b050407b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.469023 900 tablet_service.cc:1505] Processing CreateTablet for tablet b00b4a0b85374bfe83928162c4da6479 (DEFAULT_TABLE table=default.Table_619bb68d71834b3a9f5e0b2b0fdaed09 [id=616ea3074afc41bdae8f70c0d38bb591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.469401 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b00b4a0b85374bfe83928162c4da6479. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.470145 740 tablet_service.cc:1505] Processing CreateTablet for tablet 8cc5e23a86a3467c89794ae8c403230e (DEFAULT_TABLE table=default.Table_988dd0b38bd141bf8cba853c4d6685b0 [id=d32c18fb23514443a7774a6bb56030ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.470467 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cc5e23a86a3467c89794ae8c403230e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.471130 748 tablet_service.cc:1505] Processing CreateTablet for tablet 7a4fc28a9fc54ba7a6cf5dd425773d60 (DEFAULT_TABLE table=default.Table_ee8db454beae4d16a36f06dab8f41364 [id=e63ad20f8d97429497c7ccd0c2b23e39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.471447 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a4fc28a9fc54ba7a6cf5dd425773d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.475524 2740 raft_consensus.cc:493] T 44c8d5b74c8e4c499f7f68e8185efc17 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:50.475687 2740 raft_consensus.cc:515] T 44c8d5b74c8e4c499f7f68e8185efc17 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:50.475862 749 tablet_service.cc:1505] Processing CreateTablet for tablet 52ea22e5add5480fb2b1ab803ce01844 (DEFAULT_TABLE table=default.Table_cf40ea51a80d469e9cacf1e520c28248 [id=922575e63e734afbb50aa15dbee53e54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.476225 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52ea22e5add5480fb2b1ab803ce01844. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.476382 2740 leader_election.cc:290] T 44c8d5b74c8e4c499f7f68e8185efc17 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:50.477353 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "44c8d5b74c8e4c499f7f68e8185efc17" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:50.477809 752 tablet_service.cc:1505] Processing CreateTablet for tablet e5ab9384afe14e4a875be18ffdb863c3 (DEFAULT_TABLE table=default.Table_044ef6a2b1af4e3cab3c8cf00505c6a6 [id=d2e4220a0cbf443e8380433da4f064f8]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:50.477946 998 leader_election.cc:343] T 44c8d5b74c8e4c499f7f68e8185efc17 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:50.478428 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5ab9384afe14e4a875be18ffdb863c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.478515 919 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "44c8d5b74c8e4c499f7f68e8185efc17" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:50.478698 919 raft_consensus.cc:2468] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:50.478895 751 tablet_service.cc:1505] Processing CreateTablet for tablet fea0fcb8854c4fc6acda80c8ffb8d341 (DEFAULT_TABLE table=default.Table_aee0b3d530954ca3ba6492f8263dd7ff [id=384621c2a9d94b0db7a7928a3bd8fffc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.479216 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fea0fcb8854c4fc6acda80c8ffb8d341. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.479207 999 leader_election.cc:304] T 44c8d5b74c8e4c499f7f68e8185efc17 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:50.478895 744 tablet_service.cc:1505] Processing CreateTablet for tablet fbb87b118b4b4df683ae0063bc2d77a4 (DEFAULT_TABLE table=default.Table_ebd39aa54af54c0e9b4efe9b5d986034 [id=e2f5c92d69e14b5db0761eed0e9d8352]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.479838 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbb87b118b4b4df683ae0063bc2d77a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.484917 2740 raft_consensus.cc:2804] T 44c8d5b74c8e4c499f7f68e8185efc17 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:50.485028 2740 raft_consensus.cc:493] T 44c8d5b74c8e4c499f7f68e8185efc17 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:50.485085 2740 raft_consensus.cc:3060] T 44c8d5b74c8e4c499f7f68e8185efc17 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:50.487520 2740 raft_consensus.cc:515] T 44c8d5b74c8e4c499f7f68e8185efc17 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:50.488698 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "44c8d5b74c8e4c499f7f68e8185efc17" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
W20260202 02:52:50.489248 998 leader_election.cc:343] T 44c8d5b74c8e4c499f7f68e8185efc17 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:50.490134 2740 leader_election.cc:290] T 44c8d5b74c8e4c499f7f68e8185efc17 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:50.490298 919 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "44c8d5b74c8e4c499f7f68e8185efc17" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:50.490537 919 raft_consensus.cc:3060] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:50.493567 919 raft_consensus.cc:2468] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:50.494463 999 leader_election.cc:304] T 44c8d5b74c8e4c499f7f68e8185efc17 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:50.502885 2740 raft_consensus.cc:2804] T 44c8d5b74c8e4c499f7f68e8185efc17 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:50.503127 2740 raft_consensus.cc:697] T 44c8d5b74c8e4c499f7f68e8185efc17 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:50.503441 2740 consensus_queue.cc:237] T 44c8d5b74c8e4c499f7f68e8185efc17 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:50.505585 2667 raft_consensus.cc:493] T 474df41e4fe24a818fa45a4ce84d5474 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:50.505726 2667 raft_consensus.cc:515] T 474df41e4fe24a818fa45a4ce84d5474 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:50.506292 2667 leader_election.cc:290] T 474df41e4fe24a818fa45a4ce84d5474 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:50.513718 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cc5e23a86a3467c89794ae8c403230e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:50.514576 714 leader_election.cc:343] T 474df41e4fe24a818fa45a4ce84d5474 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 474df41e4fe24a818fa45a4ce84d5474
I20260202 02:52:50.515321 755 tablet_service.cc:1505] Processing CreateTablet for tablet 40bb2baffa53450683e26e816cf3e11e (DEFAULT_TABLE table=default.Table_1f6c33e9af15400180eb052e679585e7 [id=28cc2ced6205455c98cebe9127e9257c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.515695 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40bb2baffa53450683e26e816cf3e11e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.516567 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52ea22e5add5480fb2b1ab803ce01844. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.519586 747 tablet_service.cc:1505] Processing CreateTablet for tablet 0e0c43f15ba2437fb227a153241b4394 (DEFAULT_TABLE table=default.Table_1cbc82d8edee4ef58a1beda6efdca8bc [id=c754ee7b91d54c66871deab39f3fd834]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.519790 1032 tablet_service.cc:1505] Processing CreateTablet for tablet 77dbfabcf2004eb98b9fdccc3d2dc16b (DEFAULT_TABLE table=default.Table_2fb99cc8383241959a5a1d845f684a98 [id=997f13a6b6fb4219920b8f3f5e757fa9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.519879 886 tablet_service.cc:1505] Processing CreateTablet for tablet 77dbfabcf2004eb98b9fdccc3d2dc16b (DEFAULT_TABLE table=default.Table_2fb99cc8383241959a5a1d845f684a98 [id=997f13a6b6fb4219920b8f3f5e757fa9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.519974 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e0c43f15ba2437fb227a153241b4394. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.520200 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77dbfabcf2004eb98b9fdccc3d2dc16b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.520227 1031 tablet_service.cc:1505] Processing CreateTablet for tablet 40bb2baffa53450683e26e816cf3e11e (DEFAULT_TABLE table=default.Table_1f6c33e9af15400180eb052e679585e7 [id=28cc2ced6205455c98cebe9127e9257c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.520620 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a4fc28a9fc54ba7a6cf5dd425773d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.523842 753 tablet_service.cc:1505] Processing CreateTablet for tablet b00b4a0b85374bfe83928162c4da6479 (DEFAULT_TABLE table=default.Table_619bb68d71834b3a9f5e0b2b0fdaed09 [id=616ea3074afc41bdae8f70c0d38bb591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.524251 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b00b4a0b85374bfe83928162c4da6479. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.524422 2445 raft_consensus.cc:493] T 2b352d9a815e4bccafb7e2251500c5bc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:50.524565 2445 raft_consensus.cc:515] T 2b352d9a815e4bccafb7e2251500c5bc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.525173 2445 leader_election.cc:290] T 2b352d9a815e4bccafb7e2251500c5bc P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:50.526217 1182 ts_tablet_manager.cc:1929] T 59ad016e15ec4a488921eb564029b689 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:50.528079 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2b352d9a815e4bccafb7e2251500c5bc" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:50.528784 1182 log.cc:1199] T 59ad016e15ec4a488921eb564029b689 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/59ad016e15ec4a488921eb564029b689
I20260202 02:52:50.527846 896 tablet_service.cc:1505] Processing CreateTablet for tablet 40bb2baffa53450683e26e816cf3e11e (DEFAULT_TABLE table=default.Table_1f6c33e9af15400180eb052e679585e7 [id=28cc2ced6205455c98cebe9127e9257c]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:50.529268 857 leader_election.cc:343] T 2b352d9a815e4bccafb7e2251500c5bc P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:50.529497 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40bb2baffa53450683e26e816cf3e11e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.530455 750 tablet_service.cc:1505] Processing CreateTablet for tablet 77dbfabcf2004eb98b9fdccc3d2dc16b (DEFAULT_TABLE table=default.Table_2fb99cc8383241959a5a1d845f684a98 [id=997f13a6b6fb4219920b8f3f5e757fa9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:50.530906 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77dbfabcf2004eb98b9fdccc3d2dc16b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.531265 1182 ts_tablet_manager.cc:1950] T 59ad016e15ec4a488921eb564029b689 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:50.532084 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 796) took 1058 ms. Trace:
I20260202 02:52:50.533571 1182 rpcz_store.cc:276] 0202 02:52:49.473426 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:49.473816 (+ 390us) service_pool.cc:225] Handling call
0202 02:52:50.436860 (+963044us) ts_tablet_manager.cc:1151] Deleting tablet 59ad016e15ec4a488921eb564029b689
0202 02:52:50.436871 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:50.514616 (+ 77745us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:50.526120 (+ 11504us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:50.532063 (+ 5943us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":962879}
I20260202 02:52:50.534399 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 59ad016e15ec4a488921eb564029b689 (table default.Table_49cb996504bf45e9a24c6f3c01f457ff [id=ec1d1bc3e38643f7ad1c57e98ef66388]) successfully deleted
W20260202 02:52:50.537039 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet c03a12f911084222be15ea6f482ebe32 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c03a12f911084222be15ea6f482ebe32 already in progress: creating tablet
I20260202 02:52:50.537160 2740 raft_consensus.cc:493] T 587cfd0ae62f4cfd937faf6890e78f4a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:50.537317 2740 raft_consensus.cc:515] T 587cfd0ae62f4cfd937faf6890e78f4a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.537990 2740 leader_election.cc:290] T 587cfd0ae62f4cfd937faf6890e78f4a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:50.538794 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "587cfd0ae62f4cfd937faf6890e78f4a" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:50.539383 998 leader_election.cc:343] T 587cfd0ae62f4cfd937faf6890e78f4a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:50.540375 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 4482b24826884d0da0d34f43f69e1646 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4482b24826884d0da0d34f43f69e1646 already in progress: creating tablet
W20260202 02:52:50.540460 999 leader_election.cc:343] T 587cfd0ae62f4cfd937faf6890e78f4a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 587cfd0ae62f4cfd937faf6890e78f4a
I20260202 02:52:50.540585 999 leader_election.cc:304] T 587cfd0ae62f4cfd937faf6890e78f4a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:50.541018 2740 raft_consensus.cc:2749] T 587cfd0ae62f4cfd937faf6890e78f4a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:50.542045 1174 tablet_bootstrap.cc:492] T 550895b4010f4088b2cc34acff50e03f P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:50.542189 1174 ts_tablet_manager.cc:1403] T 550895b4010f4088b2cc34acff50e03f P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.164s user 0.002s sys 0.000s
I20260202 02:52:50.542948 1174 raft_consensus.cc:359] T 550895b4010f4088b2cc34acff50e03f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:50.543119 1174 raft_consensus.cc:385] T 550895b4010f4088b2cc34acff50e03f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:50.543170 1174 raft_consensus.cc:740] T 550895b4010f4088b2cc34acff50e03f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:50.543388 1174 consensus_queue.cc:260] T 550895b4010f4088b2cc34acff50e03f P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:50.543977 1174 ts_tablet_manager.cc:1434] T 550895b4010f4088b2cc34acff50e03f P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:50.544270 1174 tablet_bootstrap.cc:492] T 2c31b45dc8004eae993e972e9faa3ff8 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:50.546208 1174 tablet_bootstrap.cc:654] T 2c31b45dc8004eae993e972e9faa3ff8 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:50.548293 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 00f2d3bafb2d472aaffc0a7ba926c825 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 00f2d3bafb2d472aaffc0a7ba926c825 already in progress: creating tablet
W20260202 02:52:50.549268 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 6750a1212ceb4e6e82e4ffa93caab590 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6750a1212ceb4e6e82e4ffa93caab590 already in progress: creating tablet
I20260202 02:52:50.550523 1182 tablet_replica.cc:333] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:50.550765 1182 raft_consensus.cc:2243] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:50.552106 1182 raft_consensus.cc:2272] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:50.553774 1182 ts_tablet_manager.cc:1916] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:50.559672 1174 tablet_bootstrap.cc:492] T 2c31b45dc8004eae993e972e9faa3ff8 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:50.560024 1174 ts_tablet_manager.cc:1403] T 2c31b45dc8004eae993e972e9faa3ff8 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.005s
I20260202 02:52:50.560247 2740 raft_consensus.cc:493] T 1f379a29e2454f689d68f5951837512b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:50.560463 2740 raft_consensus.cc:515] T 1f379a29e2454f689d68f5951837512b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.561193 2740 leader_election.cc:290] T 1f379a29e2454f689d68f5951837512b P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:50.562238 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f379a29e2454f689d68f5951837512b" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:50.563022 998 leader_election.cc:343] T 1f379a29e2454f689d68f5951837512b P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:50.563967 999 leader_election.cc:343] T 1f379a29e2454f689d68f5951837512b P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 1f379a29e2454f689d68f5951837512b
I20260202 02:52:50.564081 999 leader_election.cc:304] T 1f379a29e2454f689d68f5951837512b P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:50.564512 2740 raft_consensus.cc:2749] T 1f379a29e2454f689d68f5951837512b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:50.564914 1174 raft_consensus.cc:359] T 2c31b45dc8004eae993e972e9faa3ff8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:50.566289 1174 raft_consensus.cc:385] T 2c31b45dc8004eae993e972e9faa3ff8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:50.566542 1174 raft_consensus.cc:740] T 2c31b45dc8004eae993e972e9faa3ff8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:50.566924 1174 consensus_queue.cc:260] T 2c31b45dc8004eae993e972e9faa3ff8 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:50.567422 1882 raft_consensus.cc:2243] T 842e54adae2a452a98521de24326eaff P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:50.568984 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 670) took 21267 ms. Trace:
I20260202 02:52:50.573109 1882 raft_consensus.cc:2272] T 842e54adae2a452a98521de24326eaff P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:50.575070 1182 ts_tablet_manager.cc:1929] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:50.575803 1174 ts_tablet_manager.cc:1434] T 2c31b45dc8004eae993e972e9faa3ff8 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.016s user 0.000s sys 0.003s
I20260202 02:52:50.577039 1180 rpcz_store.cc:276] 0202 02:52:29.301822 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.301875 (+ 53us) service_pool.cc:225] Handling call
0202 02:52:50.568805 (+21266930us) ts_tablet_manager.cc:1151] Deleting tablet 7665dbea16d14f619af57ea056639eef
0202 02:52:50.568820 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:50.568960 (+ 140us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":21266663}
I20260202 02:52:50.577487 1174 tablet_bootstrap.cc:492] T 65bd1c1a382e42e4a68077264563c55b P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
W20260202 02:52:50.579046 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 7665dbea16d14f619af57ea056639eef because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7665dbea16d14f619af57ea056639eef already in progress: creating tablet
I20260202 02:52:50.580308 1174 tablet_bootstrap.cc:654] T 65bd1c1a382e42e4a68077264563c55b P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:50.581866 1182 log.cc:1199] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/8bb14bf551f441dcbaaff7057b51b2be
I20260202 02:52:50.582224 1182 ts_tablet_manager.cc:1950] T 8bb14bf551f441dcbaaff7057b51b2be P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:50.583519 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 8bb14bf551f441dcbaaff7057b51b2be (table default.Table_7a314585590647ca8fd0f51e125b0622 [id=0e44d4587c694edc90c651e2f9db649b]) successfully deleted
W20260202 02:52:50.583889 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 745fc215251e4ccb9a2abed02573dd84 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 745fc215251e4ccb9a2abed02573dd84 already in progress: creating tablet
W20260202 02:52:50.584151 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 42321f4e77484027a485fdd584de3ae3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 42321f4e77484027a485fdd584de3ae3 already in progress: creating tablet
I20260202 02:52:50.585507 1174 tablet_bootstrap.cc:492] T 65bd1c1a382e42e4a68077264563c55b P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:50.585664 1174 ts_tablet_manager.cc:1403] T 65bd1c1a382e42e4a68077264563c55b P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.003s
I20260202 02:52:50.586462 1174 raft_consensus.cc:359] T 65bd1c1a382e42e4a68077264563c55b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.586621 1174 raft_consensus.cc:385] T 65bd1c1a382e42e4a68077264563c55b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:50.586668 1174 raft_consensus.cc:740] T 65bd1c1a382e42e4a68077264563c55b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:50.587733 1182 tablet_replica.cc:333] T abfe7f76b2424c4d9ab07b29b66ec103 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:50.588003 1182 raft_consensus.cc:2243] T abfe7f76b2424c4d9ab07b29b66ec103 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:50.588208 1182 raft_consensus.cc:2272] T abfe7f76b2424c4d9ab07b29b66ec103 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:50.588786 1174 consensus_queue.cc:260] T 65bd1c1a382e42e4a68077264563c55b P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.589170 1182 ts_tablet_manager.cc:1916] T abfe7f76b2424c4d9ab07b29b66ec103 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:50.589563 1174 ts_tablet_manager.cc:1434] T 65bd1c1a382e42e4a68077264563c55b P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260202 02:52:50.589896 1174 tablet_bootstrap.cc:492] T 1d8143f529b74e6ab779dd695920413c P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:50.594552 1174 tablet_bootstrap.cc:654] T 1d8143f529b74e6ab779dd695920413c P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:50.595203 2445 raft_consensus.cc:493] T 3284213e4b0447b49a7718ba896d3fa8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:50.595364 2445 raft_consensus.cc:515] T 3284213e4b0447b49a7718ba896d3fa8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.595925 2445 leader_election.cc:290] T 3284213e4b0447b49a7718ba896d3fa8 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:50.596675 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3284213e4b0447b49a7718ba896d3fa8" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:50.597361 857 leader_election.cc:343] T 3284213e4b0447b49a7718ba896d3fa8 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:50.597936 1182 ts_tablet_manager.cc:1929] T abfe7f76b2424c4d9ab07b29b66ec103 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:50.598155 1182 log.cc:1199] T abfe7f76b2424c4d9ab07b29b66ec103 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/abfe7f76b2424c4d9ab07b29b66ec103
I20260202 02:52:50.598575 1182 ts_tablet_manager.cc:1950] T abfe7f76b2424c4d9ab07b29b66ec103 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:50.598773 527 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 87ff5fbc95b84171a94fe1a83a67a7ba
I20260202 02:52:50.599787 1182 tablet_replica.cc:333] T 3284213e4b0447b49a7718ba896d3fa8 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:50.600049 1182 raft_consensus.cc:2243] T 3284213e4b0447b49a7718ba896d3fa8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:50.600565 1182 raft_consensus.cc:2272] T 3284213e4b0447b49a7718ba896d3fa8 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:50.601178 1182 ts_tablet_manager.cc:1916] T 3284213e4b0447b49a7718ba896d3fa8 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:50.603808 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_7d832c4a3a854a09909c552c8f6229a8" } new_table_name: "default.Table_c243d282b25e4f33a724fe686ffb8848" modify_external_catalogs: true
I20260202 02:52:50.604463 1174 tablet_bootstrap.cc:492] T 1d8143f529b74e6ab779dd695920413c P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:50.604581 524 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 1139db8ea6744e168e7b223c23665c58
I20260202 02:52:50.604645 1174 ts_tablet_manager.cc:1403] T 1d8143f529b74e6ab779dd695920413c P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.008s
I20260202 02:52:50.605579 1174 raft_consensus.cc:359] T 1d8143f529b74e6ab779dd695920413c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.605780 1174 raft_consensus.cc:385] T 1d8143f529b74e6ab779dd695920413c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:50.605832 1174 raft_consensus.cc:740] T 1d8143f529b74e6ab779dd695920413c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:50.606120 1174 consensus_queue.cc:260] T 1d8143f529b74e6ab779dd695920413c P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.606397 524 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet f3eebefa74584c1791d52377f5ba1290 (table default.Table_37b8a9e1666a4ddbaf5a13d828e9702c [id=e0cea4989f66443faf074eacf434658c]). Expected version 1 got 0
I20260202 02:52:50.606875 1174 ts_tablet_manager.cc:1434] T 1d8143f529b74e6ab779dd695920413c P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:50.607156 1174 tablet_bootstrap.cc:492] T da5435f969074364bc8f8a6ddc5e4f8a P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:50.607190 524 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_a7f3566b065c4bfbafe1349368f05d99" } modify_external_catalogs: true
I20260202 02:52:50.607429 524 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_a7f3566b065c4bfbafe1349368f05d99" } modify_external_catalogs: true
I20260202 02:52:50.608285 2445 raft_consensus.cc:493] T 7ea7a5d8a92b4ac0903483627258ae66 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:50.608457 2445 raft_consensus.cc:515] T 7ea7a5d8a92b4ac0903483627258ae66 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
W20260202 02:52:50.608588 528 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8a3ebe597da64da48db881cef46cfe4e consensus state has no leader
I20260202 02:52:50.609103 1174 tablet_bootstrap.cc:654] T da5435f969074364bc8f8a6ddc5e4f8a P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:50.609032 528 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_19b44b1161d14170bb703846ec8fe6d8" } new_table_name: "default.Table_d376bfb6c4d8479b9a306b2f9c124fdd" modify_external_catalogs: true
I20260202 02:52:50.609107 2445 leader_election.cc:290] T 7ea7a5d8a92b4ac0903483627258ae66 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:50.609984 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7ea7a5d8a92b4ac0903483627258ae66" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:50.610926 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_fe61f7e6b8c74ca1acb5bdf456cc6bf5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:50.610997 857 leader_election.cc:343] T 7ea7a5d8a92b4ac0903483627258ae66 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:50.611526 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fe61f7e6b8c74ca1acb5bdf456cc6bf5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:50.612884 521 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet e04960479c92479b8bb864544dd0c8ac
I20260202 02:52:50.613355 521 catalog_manager.cc:6724] ReplaceTablet: tablet e04960479c92479b8bb864544dd0c8ac deleted and replaced by tablet f7bf82f274234991b0bab4c3fa3cf4bd
I20260202 02:52:50.616848 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet abfe7f76b2424c4d9ab07b29b66ec103 (table default.Table_ff54266e8b5a4334ad1fbeda5270f0f2 [id=b6c09db68b4542a3bb11bf20b1d54571]) successfully deleted
I20260202 02:52:50.617204 902 tablet_service.cc:1552] Processing DeleteTablet for tablet 1139db8ea6744e168e7b223c23665c58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:50 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:50.617305 1174 tablet_bootstrap.cc:492] T da5435f969074364bc8f8a6ddc5e4f8a P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:50.617295 896 tablet_service.cc:1552] Processing DeleteTablet for tablet 87ff5fbc95b84171a94fe1a83a67a7ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:50 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:50.617417 890 tablet_service.cc:1552] Processing DeleteTablet for tablet e04960479c92479b8bb864544dd0c8ac with delete_type TABLET_DATA_DELETED (replaced by tablet f7bf82f274234991b0bab4c3fa3cf4bd) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:50.617434 1174 ts_tablet_manager.cc:1403] T da5435f969074364bc8f8a6ddc5e4f8a P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.005s
I20260202 02:52:50.617977 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_e0bea82391b342c1a81564e3b2789f94"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:50.618261 1174 raft_consensus.cc:359] T da5435f969074364bc8f8a6ddc5e4f8a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.618444 1174 raft_consensus.cc:385] T da5435f969074364bc8f8a6ddc5e4f8a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:50.618503 1174 raft_consensus.cc:740] T da5435f969074364bc8f8a6ddc5e4f8a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
W20260202 02:52:50.618654 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e0bea82391b342c1a81564e3b2789f94 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:50.618755 1174 consensus_queue.cc:260] T da5435f969074364bc8f8a6ddc5e4f8a P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.619261 1174 ts_tablet_manager.cc:1434] T da5435f969074364bc8f8a6ddc5e4f8a P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:50.619571 1174 tablet_bootstrap.cc:492] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:50.621428 1174 tablet_bootstrap.cc:654] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:50.621624 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_d64d39313f1448de84c04387a8a6d33d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:50.622232 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d64d39313f1448de84c04387a8a6d33d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:50.625495 521 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_9412ac9f6f6c495f917ebcf68650e5f1" } modify_external_catalogs: true
I20260202 02:52:50.625793 521 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_9412ac9f6f6c495f917ebcf68650e5f1" } modify_external_catalogs: true
I20260202 02:52:50.628849 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 87ff5fbc95b84171a94fe1a83a67a7ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:50 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:50.629021 745 tablet_service.cc:1552] Processing DeleteTablet for tablet 550895b4010f4088b2cc34acff50e03f with delete_type TABLET_DATA_DELETED (replaced by tablet 5416eab7a0914ecb855fbccde4894285) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:50.629158 744 tablet_service.cc:1552] Processing DeleteTablet for tablet 65bd1c1a382e42e4a68077264563c55b with delete_type TABLET_DATA_DELETED (replaced by tablet 3706562a22634eb6b72e36bb6a2d1ae5) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:50.629303 748 tablet_service.cc:1552] Processing DeleteTablet for tablet 2c31b45dc8004eae993e972e9faa3ff8 with delete_type TABLET_DATA_DELETED (replaced by tablet aed1081136f7431b9b14784c21112d1c) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:50.629705 743 tablet_service.cc:1552] Processing DeleteTablet for tablet 1139db8ea6744e168e7b223c23665c58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:50 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:50.629889 756 tablet_service.cc:1552] Processing DeleteTablet for tablet e04960479c92479b8bb864544dd0c8ac with delete_type TABLET_DATA_DELETED (replaced by tablet f7bf82f274234991b0bab4c3fa3cf4bd) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:50.630592 1174 tablet_bootstrap.cc:492] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:50.630728 1174 ts_tablet_manager.cc:1403] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.007s
I20260202 02:52:50.631604 1174 raft_consensus.cc:359] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:50.631807 1174 raft_consensus.cc:385] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:50.631865 1174 raft_consensus.cc:740] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:50.632097 1174 consensus_queue.cc:260] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:50.632588 1174 ts_tablet_manager.cc:1434] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:50.632899 1174 tablet_bootstrap.cc:492] T d0ce2121d65b4a228596e0ca9018883d P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:50.634172 1182 ts_tablet_manager.cc:1929] T 3284213e4b0447b49a7718ba896d3fa8 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:50.634289 1182 log.cc:1199] T 3284213e4b0447b49a7718ba896d3fa8 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/3284213e4b0447b49a7718ba896d3fa8
I20260202 02:52:50.634567 1182 ts_tablet_manager.cc:1950] T 3284213e4b0447b49a7718ba896d3fa8 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:50.634804 1174 tablet_bootstrap.cc:654] T d0ce2121d65b4a228596e0ca9018883d P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:50.635358 1182 tablet_replica.cc:333] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:50.635356 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 3284213e4b0447b49a7718ba896d3fa8 (table default.Table_e71a7347855d447cabe6ea2cc64b293a [id=e490dd882e8e4e51ad5a2073c891ba91]) successfully deleted
I20260202 02:52:50.635586 1182 raft_consensus.cc:2243] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:50.635778 1182 raft_consensus.cc:2272] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:50.637764 1182 ts_tablet_manager.cc:1916] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:50.642146 714 consensus_peers.cc:597] T 072bb7f0e4ab4935be49461bc8a3c163 P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 072bb7f0e4ab4935be49461bc8a3c163. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:50.643147 1174 tablet_bootstrap.cc:492] T d0ce2121d65b4a228596e0ca9018883d P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:50.643321 1174 ts_tablet_manager.cc:1403] T d0ce2121d65b4a228596e0ca9018883d P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260202 02:52:50.612833 1037 tablet_service.cc:1552] Processing DeleteTablet for tablet 1139db8ea6744e168e7b223c23665c58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:50 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:50.622545 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet 587cfd0ae62f4cfd937faf6890e78f4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:50.622598 1030 tablet_service.cc:1552] Processing DeleteTablet for tablet 65bd1c1a382e42e4a68077264563c55b with delete_type TABLET_DATA_DELETED (replaced by tablet 3706562a22634eb6b72e36bb6a2d1ae5) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:50.577533 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 671) took 21272 ms. Trace:
I20260202 02:52:50.622671 1028 tablet_service.cc:1552] Processing DeleteTablet for tablet 1f379a29e2454f689d68f5951837512b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:50.600073 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet 87ff5fbc95b84171a94fe1a83a67a7ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:50 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:50.640179 2740 consensus_queue.cc:1048] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260202 02:52:50.616415 1039 tablet_service.cc:1552] Processing DeleteTablet for tablet e04960479c92479b8bb864544dd0c8ac with delete_type TABLET_DATA_DELETED (replaced by tablet f7bf82f274234991b0bab4c3fa3cf4bd) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:50.622686 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet ac2ab12f5040488aa306cb69b107b485 with delete_type TABLET_DATA_DELETED (replaced by tablet 898778f8f0b548ec89cb1417d6f07926) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:50.644447 1180 rpcz_store.cc:276] 0202 02:52:29.304788 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.306772 (+ 1984us) service_pool.cc:225] Handling call
0202 02:52:50.577372 (+21270600us) ts_tablet_manager.cc:1151] Deleting tablet 842197c689064660b791b40e521d346b
0202 02:52:50.577380 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:50.577499 (+ 119us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":21270331}
I20260202 02:52:50.644928 1174 raft_consensus.cc:359] T d0ce2121d65b4a228596e0ca9018883d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.645085 1174 raft_consensus.cc:385] T d0ce2121d65b4a228596e0ca9018883d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:50.645139 1174 raft_consensus.cc:740] T d0ce2121d65b4a228596e0ca9018883d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:50.645323 1174 consensus_queue.cc:260] T d0ce2121d65b4a228596e0ca9018883d P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.645437 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 672) took 21322 ms. Trace:
I20260202 02:52:50.645695 1180 rpcz_store.cc:276] 0202 02:52:29.322688 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.329407 (+ 6719us) service_pool.cc:225] Handling call
0202 02:52:50.645282 (+21315875us) ts_tablet_manager.cc:1151] Deleting tablet e7df9a78735547cbbaa006c604b95fec
0202 02:52:50.645293 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:50.645419 (+ 126us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":21315640}
I20260202 02:52:50.646246 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 673) took 21297 ms. Trace:
I20260202 02:52:50.646368 1180 rpcz_store.cc:276] 0202 02:52:29.348674 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.348769 (+ 95us) service_pool.cc:225] Handling call
0202 02:52:50.646116 (+21297347us) ts_tablet_manager.cc:1151] Deleting tablet fc3bfd542b8e41d98586645358061261
0202 02:52:50.646125 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:50.646230 (+ 105us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":21297109}
I20260202 02:52:50.646808 1180 tablet_replica.cc:333] T 3d1a7578e83447669e3c3972a426ad44 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:50.645810 1174 ts_tablet_manager.cc:1434] T d0ce2121d65b4a228596e0ca9018883d P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:50.647027 1180 raft_consensus.cc:2243] T 3d1a7578e83447669e3c3972a426ad44 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:50.647260 1180 raft_consensus.cc:2272] T 3d1a7578e83447669e3c3972a426ad44 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:50.647302 1174 tablet_bootstrap.cc:492] T 93f958924885457ab15f2aac5509e0aa P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:50.647902 1180 ts_tablet_manager.cc:1916] T 3d1a7578e83447669e3c3972a426ad44 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:50.649192 1182 ts_tablet_manager.cc:1929] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.7
I20260202 02:52:50.649389 1182 log.cc:1199] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/63a9029bc39c4fbd8ac4bf4bc37d6b4e
I20260202 02:52:50.649389 1174 tablet_bootstrap.cc:654] T 93f958924885457ab15f2aac5509e0aa P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:50.649808 1182 ts_tablet_manager.cc:1950] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:50.650537 1182 tablet_replica.cc:333] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:50.650501 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 63a9029bc39c4fbd8ac4bf4bc37d6b4e (table default.Table_4062ed5efe774bc2b380c3ae7fa53abe [id=fa95e04db2d443b5972de98c7c0e6457]) successfully deleted
I20260202 02:52:50.650806 1182 raft_consensus.cc:2243] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260202 02:52:50.650869 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet a81e1d10ca49459989260fbd6a2b8645 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a81e1d10ca49459989260fbd6a2b8645 already in progress: creating tablet
W20260202 02:52:50.651103 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 842197c689064660b791b40e521d346b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 842197c689064660b791b40e521d346b already in progress: creating tablet
W20260202 02:52:50.652279 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet e7df9a78735547cbbaa006c604b95fec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e7df9a78735547cbbaa006c604b95fec already in progress: creating tablet
I20260202 02:52:50.652582 1182 raft_consensus.cc:2272] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:50.653185 1182 ts_tablet_manager.cc:1916] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:50.653489 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet fc3bfd542b8e41d98586645358061261 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fc3bfd542b8e41d98586645358061261 already in progress: creating tablet
I20260202 02:52:50.578158 1882 ts_tablet_manager.cc:1916] T 842e54adae2a452a98521de24326eaff P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:50.651531 2619 raft_consensus.cc:493] T 65bd1c1a382e42e4a68077264563c55b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:50.657239 2619 raft_consensus.cc:515] T 65bd1c1a382e42e4a68077264563c55b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.657999 2619 leader_election.cc:290] T 65bd1c1a382e42e4a68077264563c55b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:50.660216 714 leader_election.cc:343] T 65bd1c1a382e42e4a68077264563c55b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 65bd1c1a382e42e4a68077264563c55b
I20260202 02:52:50.661671 1182 ts_tablet_manager.cc:1929] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:50.661813 1182 log.cc:1199] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/25765fb687294e9db1289f76530dd877
I20260202 02:52:50.662153 1182 ts_tablet_manager.cc:1950] T 25765fb687294e9db1289f76530dd877 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:50.662823 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 25765fb687294e9db1289f76530dd877 (table default.Table_03cb720f804b44e2b29f8cc47336b2db [id=ea0c52a72da8422ba14345fbfc242d27]) successfully deleted
I20260202 02:52:50.663132 1182 tablet_replica.cc:333] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:50.664086 1174 tablet_bootstrap.cc:492] T 93f958924885457ab15f2aac5509e0aa P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:50.664239 1174 ts_tablet_manager.cc:1403] T 93f958924885457ab15f2aac5509e0aa P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.006s
I20260202 02:52:50.665061 1174 raft_consensus.cc:359] T 93f958924885457ab15f2aac5509e0aa P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.665238 1174 raft_consensus.cc:385] T 93f958924885457ab15f2aac5509e0aa P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:50.665297 1174 raft_consensus.cc:740] T 93f958924885457ab15f2aac5509e0aa P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:50.665637 1174 consensus_queue.cc:260] T 93f958924885457ab15f2aac5509e0aa P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.666211 1174 ts_tablet_manager.cc:1434] T 93f958924885457ab15f2aac5509e0aa P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:50.666482 1174 tablet_bootstrap.cc:492] T 1f379a29e2454f689d68f5951837512b P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:50.668303 1174 tablet_bootstrap.cc:654] T 1f379a29e2454f689d68f5951837512b P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:50.682456 1174 tablet_bootstrap.cc:492] T 1f379a29e2454f689d68f5951837512b P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:50.682749 1174 ts_tablet_manager.cc:1403] T 1f379a29e2454f689d68f5951837512b P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.010s
I20260202 02:52:50.683678 1174 raft_consensus.cc:359] T 1f379a29e2454f689d68f5951837512b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.683967 1174 raft_consensus.cc:385] T 1f379a29e2454f689d68f5951837512b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:50.684067 1174 raft_consensus.cc:740] T 1f379a29e2454f689d68f5951837512b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:50.684371 1174 consensus_queue.cc:260] T 1f379a29e2454f689d68f5951837512b P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.685108 1174 ts_tablet_manager.cc:1434] T 1f379a29e2454f689d68f5951837512b P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:50.685458 1174 tablet_bootstrap.cc:492] T 75a24a1bf54d4c799938a5723ac6fa99 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:50.687464 1174 tablet_bootstrap.cc:654] T 75a24a1bf54d4c799938a5723ac6fa99 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:50.689276 1173 tablet_bootstrap.cc:492] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:50.689497 1173 ts_tablet_manager.cc:1403] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.241s user 0.002s sys 0.001s
I20260202 02:52:50.688745 2667 raft_consensus.cc:493] T d8418a9dd0944456b4d4820dde0cde02 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:50.689708 2667 raft_consensus.cc:515] T d8418a9dd0944456b4d4820dde0cde02 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:50.690436 2667 leader_election.cc:290] T d8418a9dd0944456b4d4820dde0cde02 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:50.691285 2740 raft_consensus.cc:493] T ac2ab12f5040488aa306cb69b107b485 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:50.691461 2740 raft_consensus.cc:515] T ac2ab12f5040488aa306cb69b107b485 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
W20260202 02:52:50.691546 714 leader_election.cc:343] T d8418a9dd0944456b4d4820dde0cde02 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: d8418a9dd0944456b4d4820dde0cde02
I20260202 02:52:50.691797 1173 raft_consensus.cc:359] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.692005 1173 raft_consensus.cc:385] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:50.692097 1173 raft_consensus.cc:740] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:50.692121 2740 leader_election.cc:290] T ac2ab12f5040488aa306cb69b107b485 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:50.692469 1173 consensus_queue.cc:260] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.692836 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ac2ab12f5040488aa306cb69b107b485" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:50.693368 998 leader_election.cc:343] T ac2ab12f5040488aa306cb69b107b485 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:50.693598 999 leader_election.cc:343] T ac2ab12f5040488aa306cb69b107b485 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: ac2ab12f5040488aa306cb69b107b485
I20260202 02:52:50.693696 999 leader_election.cc:304] T ac2ab12f5040488aa306cb69b107b485 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:50.694044 2740 raft_consensus.cc:2749] T ac2ab12f5040488aa306cb69b107b485 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:50.694532 1173 ts_tablet_manager.cc:1434] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260202 02:52:50.694943 1053 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "474df41e4fe24a818fa45a4ce84d5474" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:50.695091 1053 raft_consensus.cc:2468] T 474df41e4fe24a818fa45a4ce84d5474 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:50.695307 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2b352d9a815e4bccafb7e2251500c5bc" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:50.695380 1056 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3284213e4b0447b49a7718ba896d3fa8" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:50.696427 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7ea7a5d8a92b4ac0903483627258ae66" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:50.697549 715 consensus_peers.cc:597] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f94f25f4059a43b8babedb6b45a08a19. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:50.697912 715 leader_election.cc:304] T 474df41e4fe24a818fa45a4ce84d5474 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:50.698765 2633 consensus_queue.cc:1048] T b7880cb051c24bc998003f0effed3cf8 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:50.699200 2633 raft_consensus.cc:2804] T 474df41e4fe24a818fa45a4ce84d5474 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:50.699271 2633 raft_consensus.cc:493] T 474df41e4fe24a818fa45a4ce84d5474 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:50.699330 2633 raft_consensus.cc:3060] T 474df41e4fe24a818fa45a4ce84d5474 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:50.699373 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "65bd1c1a382e42e4a68077264563c55b" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:50.699589 1059 raft_consensus.cc:2468] T 65bd1c1a382e42e4a68077264563c55b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:50.700014 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e0c43f15ba2437fb227a153241b4394. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:50.700443 859 leader_election.cc:343] T 2b352d9a815e4bccafb7e2251500c5bc P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:50.700588 859 leader_election.cc:304] T 2b352d9a815e4bccafb7e2251500c5bc P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:50.701468 2739 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92a8191cf1e14e4d99f1c378c2d3c85b)
I20260202 02:52:50.701763 2664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92a8191cf1e14e4d99f1c378c2d3c85b. Latest consensus state: current_term: 2 leader_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } } }
I20260202 02:52:50.701743 2739 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } }
I20260202 02:52:50.702023 2664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:52:50.702371 2445 raft_consensus.cc:2749] T 2b352d9a815e4bccafb7e2251500c5bc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:50.703052 2633 raft_consensus.cc:515] T 474df41e4fe24a818fa45a4ce84d5474 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:50.703630 2633 leader_election.cc:290] T 474df41e4fe24a818fa45a4ce84d5474 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:50.709522 714 leader_election.cc:343] T 474df41e4fe24a818fa45a4ce84d5474 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 474df41e4fe24a818fa45a4ce84d5474
W20260202 02:52:50.713857 714 consensus_peers.cc:597] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1116872cc25a4ab5b1a3a6bbba1f0da7. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:50.714099 2739 leader_election.cc:290] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 92a8191cf1e14e4d99f1c378c2d3c85b (127.0.104.189:46271), a9c3e3612bb94fb7b94c308041acfaeb (127.0.104.188:39077)
I20260202 02:52:50.714337 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40bb2baffa53450683e26e816cf3e11e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.714524 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b00b4a0b85374bfe83928162c4da6479. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.714684 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77dbfabcf2004eb98b9fdccc3d2dc16b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.731787 2332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47282 (request call id 0) took 20324 ms. Trace:
W20260202 02:52:50.734477 519 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fea0fcb8854c4fc6acda80c8ffb8d341 consensus state has no leader
I20260202 02:52:50.736711 526 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_88cb19d3d7684dc1a8aa41ed28fed4b8" } modify_external_catalogs: true
I20260202 02:52:50.736958 526 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_88cb19d3d7684dc1a8aa41ed28fed4b8" } modify_external_catalogs: true
W20260202 02:52:50.739877 423 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 73287.8us
W20260202 02:52:50.741322 528 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b34dc1cab5874fa6aee55d79c6e09ade consensus state has no leader
I20260202 02:52:50.741776 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_82031bd5aec3477bb4ccf0183961a528"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:50.742395 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_82031bd5aec3477bb4ccf0183961a528 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:50.752408 714 consensus_peers.cc:597] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f94f25f4059a43b8babedb6b45a08a19. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:50.755190 1882 ts_tablet_manager.cc:1929] T 842e54adae2a452a98521de24326eaff P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:50.755343 1882 log.cc:1199] T 842e54adae2a452a98521de24326eaff P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/842e54adae2a452a98521de24326eaff
I20260202 02:52:50.755663 1882 ts_tablet_manager.cc:1950] T 842e54adae2a452a98521de24326eaff P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:50.756085 1173 tablet_bootstrap.cc:492] T 939fbcaf75c7439d9208bbbf83a72717 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:50.757735 1173 tablet_bootstrap.cc:654] T 939fbcaf75c7439d9208bbbf83a72717 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:50.778152 714 consensus_peers.cc:597] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 63a9029bc39c4fbd8ac4bf4bc37d6b4e. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:50.785358 715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 87056.6us
I20260202 02:52:50.787577 2619 consensus_queue.cc:1048] T 072bb7f0e4ab4935be49461bc8a3c163 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:50.788679 715 leader_election.cc:304] T 65bd1c1a382e42e4a68077264563c55b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:50.789745 2619 raft_consensus.cc:2804] T 65bd1c1a382e42e4a68077264563c55b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:50.789853 2619 raft_consensus.cc:493] T 65bd1c1a382e42e4a68077264563c55b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:50.789914 2619 raft_consensus.cc:3060] T 65bd1c1a382e42e4a68077264563c55b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:50.792827 2619 raft_consensus.cc:515] T 65bd1c1a382e42e4a68077264563c55b P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
W20260202 02:52:50.794643 714 leader_election.cc:343] T 65bd1c1a382e42e4a68077264563c55b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 65bd1c1a382e42e4a68077264563c55b
I20260202 02:52:50.794984 2619 leader_election.cc:290] T 65bd1c1a382e42e4a68077264563c55b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:50.795425 715 leader_election.cc:343] T d8418a9dd0944456b4d4820dde0cde02 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Not found: Tablet not found: d8418a9dd0944456b4d4820dde0cde02
I20260202 02:52:50.796178 715 leader_election.cc:304] T d8418a9dd0944456b4d4820dde0cde02 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:50.796981 2619 raft_consensus.cc:2749] T d8418a9dd0944456b4d4820dde0cde02 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:50.797011 1173 tablet_bootstrap.cc:492] T 939fbcaf75c7439d9208bbbf83a72717 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:50.797161 1173 ts_tablet_manager.cc:1403] T 939fbcaf75c7439d9208bbbf83a72717 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.041s user 0.003s sys 0.000s
I20260202 02:52:50.802104 2619 raft_consensus.cc:493] T ccd6bc8bd0504bf18d108c73685ef9ff P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:50.802389 2619 raft_consensus.cc:515] T ccd6bc8bd0504bf18d108c73685ef9ff P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:50.803506 2619 leader_election.cc:290] T ccd6bc8bd0504bf18d108c73685ef9ff P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:50.803977 714 leader_election.cc:343] T ccd6bc8bd0504bf18d108c73685ef9ff P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: ccd6bc8bd0504bf18d108c73685ef9ff
I20260202 02:52:50.807088 2619 consensus_queue.cc:1048] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:50.807848 2619 raft_consensus.cc:493] T 842e54adae2a452a98521de24326eaff P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:50.808040 2619 raft_consensus.cc:515] T 842e54adae2a452a98521de24326eaff P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:50.808730 2619 leader_election.cc:290] T 842e54adae2a452a98521de24326eaff P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:50.809732 714 leader_election.cc:343] T 842e54adae2a452a98521de24326eaff P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 842e54adae2a452a98521de24326eaff
I20260202 02:52:50.809940 1175 tablet_bootstrap.cc:492] T 2f5aae089fcc492f9ce55f84674a452c P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:50.810110 1175 ts_tablet_manager.cc:1403] T 2f5aae089fcc492f9ce55f84674a452c P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.404s user 0.057s sys 0.131s
W20260202 02:52:50.810132 715 consensus_peers.cc:597] T 33150e1456144b8e9c64cc2a2ac6dcca P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 33150e1456144b8e9c64cc2a2ac6dcca. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:50.810971 1175 raft_consensus.cc:359] T 2f5aae089fcc492f9ce55f84674a452c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:50.811148 1175 raft_consensus.cc:385] T 2f5aae089fcc492f9ce55f84674a452c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:50.811208 1175 raft_consensus.cc:740] T 2f5aae089fcc492f9ce55f84674a452c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:50.811215 2619 consensus_queue.cc:1048] T 6df3d11cf578404ca41807f0777b6c53 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:50.811231 2633 raft_consensus.cc:493] T da5435f969074364bc8f8a6ddc5e4f8a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:50.811471 2633 raft_consensus.cc:515] T da5435f969074364bc8f8a6ddc5e4f8a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.811506 1175 consensus_queue.cc:260] T 2f5aae089fcc492f9ce55f84674a452c P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:50.811967 2633 leader_election.cc:290] T da5435f969074364bc8f8a6ddc5e4f8a P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:50.812057 859 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 111618us
W20260202 02:52:50.812278 859 leader_election.cc:343] T 3284213e4b0447b49a7718ba896d3fa8 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:50.812393 859 leader_election.cc:304] T 3284213e4b0447b49a7718ba896d3fa8 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
W20260202 02:52:50.812794 714 leader_election.cc:343] T da5435f969074364bc8f8a6ddc5e4f8a P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: da5435f969074364bc8f8a6ddc5e4f8a
I20260202 02:52:50.812971 1175 ts_tablet_manager.cc:1434] T 2f5aae089fcc492f9ce55f84674a452c P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260202 02:52:50.813235 1175 tablet_bootstrap.cc:492] T 8153510d5d154c7895b1ee794585612e P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:50.814604 1175 tablet_bootstrap.cc:654] T 8153510d5d154c7895b1ee794585612e P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:50.828598 1173 raft_consensus.cc:359] T 939fbcaf75c7439d9208bbbf83a72717 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:50.829079 1173 raft_consensus.cc:385] T 939fbcaf75c7439d9208bbbf83a72717 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:50.829164 1173 raft_consensus.cc:740] T 939fbcaf75c7439d9208bbbf83a72717 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:50.829423 1173 consensus_queue.cc:260] T 939fbcaf75c7439d9208bbbf83a72717 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:50.830015 1173 ts_tablet_manager.cc:1434] T 939fbcaf75c7439d9208bbbf83a72717 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260202 02:52:50.830369 1173 tablet_bootstrap.cc:492] T 2b352d9a815e4bccafb7e2251500c5bc P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:50.831971 1173 tablet_bootstrap.cc:654] T 2b352d9a815e4bccafb7e2251500c5bc P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:50.832204 859 raft_consensus.cc:2686] T 3284213e4b0447b49a7718ba896d3fa8 P 1521a6d262b7451bbef8debe970dd6b5: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260202 02:52:50.833006 859 leader_election.cc:343] T 7ea7a5d8a92b4ac0903483627258ae66 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:50.833176 859 leader_election.cc:304] T 7ea7a5d8a92b4ac0903483627258ae66 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:50.833643 2445 raft_consensus.cc:2749] T 7ea7a5d8a92b4ac0903483627258ae66 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:50.852818 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_a1067e637b814119a16c9a4d8039b649"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:50.853524 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a1067e637b814119a16c9a4d8039b649 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:50.861980 515 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.GetMasterRegistration -> {remote=127.0.104.190:33981, user_credentials={real_user=slave}} blocked reactor thread for 66221.1us
I20260202 02:52:50.867460 520 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_1cbc82d8edee4ef58a1beda6efdca8bc" } new_table_name: "default.Table_afc2a88e2e81424a8416a78521d1aa3b" modify_external_catalogs: true
W20260202 02:52:50.881937 714 consensus_peers.cc:597] T 6df3d11cf578404ca41807f0777b6c53 P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6df3d11cf578404ca41807f0777b6c53. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:50.886299 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_0bdc711d3bce45af9149f191f5d61ee2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:50.887023 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0bdc711d3bce45af9149f191f5d61ee2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:50.889386 778 raft_consensus.cc:3060] T 29c996781cb947f3a1aee5dd0a9c4828 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:50.893875 524 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 82680e577c504a729122cd4f6b4916a6
I20260202 02:52:50.893874 2858 raft_consensus.cc:493] T 99e0b5e9a9874aec886155e0d614dec2 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:50.894034 2858 raft_consensus.cc:515] T 99e0b5e9a9874aec886155e0d614dec2 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.894531 524 master_service.cc:954] ReplaceTablet: received request to replace tablet c977cd5aa66a4f6c8eeb37bf67f18fa7 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:50.895360 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "99e0b5e9a9874aec886155e0d614dec2" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:50.895960 857 leader_election.cc:343] T 99e0b5e9a9874aec886155e0d614dec2 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:50.897177 2858 leader_election.cc:290] T 99e0b5e9a9874aec886155e0d614dec2 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:50.898387 890 tablet_service.cc:1552] Processing DeleteTablet for tablet 82680e577c504a729122cd4f6b4916a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:50 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:50.905103 740 tablet_service.cc:1552] Processing DeleteTablet for tablet 82680e577c504a729122cd4f6b4916a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:50 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:50.922124 538 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7dab2abef14445d688ce1335b1f9b2f8" candidate_term: 3 candidate_status { last_received { term: 2 index: 858 } } ignore_live_leader: false dest_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" is_pre_election: true
I20260202 02:52:50.924834 524 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_3be2f2becb3746a8a26896e806ee480a" } modify_external_catalogs: true
I20260202 02:52:50.925060 524 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_3be2f2becb3746a8a26896e806ee480a" } modify_external_catalogs: true
I20260202 02:52:50.927985 1182 raft_consensus.cc:2243] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:50.928284 1182 pending_rounds.cc:70] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5: Trying to abort 1 pending ops.
I20260202 02:52:50.928368 1182 pending_rounds.cc:77] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923153421680640 op_type: NO_OP noop_request { }
I20260202 02:52:50.928453 1182 raft_consensus.cc:2889] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:50.928530 1182 raft_consensus.cc:2272] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:50.931546 527 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 57df8eb7aa164138845a453391e091da consensus state has no leader
I20260202 02:52:50.931973 527 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_cd2f525ec9b7457fa8c8a42370f00c43" } modify_external_catalogs: true
I20260202 02:52:50.932142 527 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_cd2f525ec9b7457fa8c8a42370f00c43" } modify_external_catalogs: true
W20260202 02:52:50.928282 2853 consensus_queue.cc:1175] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "1521a6d262b7451bbef8debe970dd6b5" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20260202 02:52:50.935979 1182 ts_tablet_manager.cc:1916] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:50.943648 1182 ts_tablet_manager.cc:1929] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:50.943894 1182 log.cc:1199] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/29c996781cb947f3a1aee5dd0a9c4828
I20260202 02:52:50.944350 1182 ts_tablet_manager.cc:1950] T 29c996781cb947f3a1aee5dd0a9c4828 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:50.945142 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 29c996781cb947f3a1aee5dd0a9c4828 (table default.Table_458f570486154241ad8c7f10a8135ace [id=d2eb691b1287473990f0a66d02f71554]) successfully deleted
I20260202 02:52:50.946750 528 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_a0dab118c0584cbdbb2926ebba1d91c2" } new_table_name: "default.Table_4ef550f5982b404da49a226ce70720fe" modify_external_catalogs: true
I20260202 02:52:50.801766 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "65bd1c1a382e42e4a68077264563c55b" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:50.948165 1045 raft_consensus.cc:3060] T 65bd1c1a382e42e4a68077264563c55b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:50.949455 1175 tablet_bootstrap.cc:492] T 8153510d5d154c7895b1ee794585612e P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:50.949584 1175 ts_tablet_manager.cc:1403] T 8153510d5d154c7895b1ee794585612e P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.136s user 0.001s sys 0.001s
I20260202 02:52:50.950322 1175 raft_consensus.cc:359] T 8153510d5d154c7895b1ee794585612e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.950482 1175 raft_consensus.cc:385] T 8153510d5d154c7895b1ee794585612e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:50.950524 1175 raft_consensus.cc:740] T 8153510d5d154c7895b1ee794585612e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:50.950716 1175 consensus_queue.cc:260] T 8153510d5d154c7895b1ee794585612e P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.950879 1045 raft_consensus.cc:2468] T 65bd1c1a382e42e4a68077264563c55b P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:50.804692 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ccd6bc8bd0504bf18d108c73685ef9ff" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:50.810731 999 consensus_peers.cc:597] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 26b890ed87c3471b9fc0b172c90e0a11. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:50.821374 1056 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "da5435f969074364bc8f8a6ddc5e4f8a" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:50.951181 1064 raft_consensus.cc:2468] T ccd6bc8bd0504bf18d108c73685ef9ff P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:50.951315 1175 ts_tablet_manager.cc:1434] T 8153510d5d154c7895b1ee794585612e P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
W20260202 02:52:50.951185 999 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 140486us
I20260202 02:52:50.800127 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "474df41e4fe24a818fa45a4ce84d5474" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:50.875793 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 737) took 2911 ms. Trace:
I20260202 02:52:50.876061 1054 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "842e54adae2a452a98521de24326eaff" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:50.893553 2740 consensus_queue.cc:1048] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:50.951602 1882 rpcz_store.cc:276] 0202 02:52:47.964466 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:47.964537 (+ 71us) service_pool.cc:225] Handling call
0202 02:52:50.298781 (+2334244us) ts_tablet_manager.cc:1151] Deleting tablet 842e54adae2a452a98521de24326eaff
0202 02:52:50.298789 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:50.660179 (+361390us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:50.755053 (+ 94874us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:50.875758 (+120705us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":53496,"tablet-delete.queue_time_us":2248991}
I20260202 02:52:50.896683 1061 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "99e0b5e9a9874aec886155e0d614dec2" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:50.952144 1882 tablet_replica.cc:333] T 79c73f4a92044bafb5b8384229cb3a4d P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:50.952351 1882 raft_consensus.cc:2243] T 79c73f4a92044bafb5b8384229cb3a4d P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:50.920441 1035 tablet_service.cc:1552] Processing DeleteTablet for tablet 82680e577c504a729122cd4f6b4916a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:50 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:50.951624 1059 raft_consensus.cc:3060] T 474df41e4fe24a818fa45a4ce84d5474 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:50.952636 859 leader_election.cc:343] T 99e0b5e9a9874aec886155e0d614dec2 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:50.952745 859 leader_election.cc:304] T 99e0b5e9a9874aec886155e0d614dec2 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:50.953105 2858 raft_consensus.cc:2749] T 99e0b5e9a9874aec886155e0d614dec2 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:50.953274 1882 raft_consensus.cc:2272] T 79c73f4a92044bafb5b8384229cb3a4d P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:50.955781 1182 tablet_replica.cc:333] T 2b352d9a815e4bccafb7e2251500c5bc P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:50.953879 1882 ts_tablet_manager.cc:1916] T 79c73f4a92044bafb5b8384229cb3a4d P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:50.955265 1059 raft_consensus.cc:2468] T 474df41e4fe24a818fa45a4ce84d5474 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:50.956300 1182 raft_consensus.cc:2243] T 2b352d9a815e4bccafb7e2251500c5bc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:50.956521 1182 raft_consensus.cc:2272] T 2b352d9a815e4bccafb7e2251500c5bc P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:50.957093 1182 ts_tablet_manager.cc:1916] T 2b352d9a815e4bccafb7e2251500c5bc P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:50.959189 999 consensus_peers.cc:597] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 29c996781cb947f3a1aee5dd0a9c4828. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:50.961210 1174 tablet_bootstrap.cc:492] T 75a24a1bf54d4c799938a5723ac6fa99 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:50.961360 1174 ts_tablet_manager.cc:1403] T 75a24a1bf54d4c799938a5723ac6fa99 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.276s user 0.019s sys 0.042s
I20260202 02:52:50.962092 1174 raft_consensus.cc:359] T 75a24a1bf54d4c799938a5723ac6fa99 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.962265 1174 raft_consensus.cc:385] T 75a24a1bf54d4c799938a5723ac6fa99 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:50.962311 1174 raft_consensus.cc:740] T 75a24a1bf54d4c799938a5723ac6fa99 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:50.962570 1174 consensus_queue.cc:260] T 75a24a1bf54d4c799938a5723ac6fa99 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.965974 2883 raft_consensus.cc:493] T 1d8143f529b74e6ab779dd695920413c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:50.968008 1175 tablet_bootstrap.cc:492] T 79badde1001b432b95f58b5161ace15c P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:50.969937 1447 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7dab2abef14445d688ce1335b1f9b2f8" candidate_term: 3 candidate_status { last_received { term: 2 index: 858 } } ignore_live_leader: false dest_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" is_pre_election: true
I20260202 02:52:50.971280 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 842e54adae2a452a98521de24326eaff (table default.Table_c6e682a233904983978e7abb23565e41 [id=0d7624b55c1a4bf0a449f651ac419d77]) successfully deleted
I20260202 02:52:50.977288 1174 ts_tablet_manager.cc:1434] T 75a24a1bf54d4c799938a5723ac6fa99 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.016s user 0.001s sys 0.001s
I20260202 02:52:50.977648 1174 tablet_bootstrap.cc:492] T 939fbcaf75c7439d9208bbbf83a72717 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:50.985209 1175 tablet_bootstrap.cc:654] T 79badde1001b432b95f58b5161ace15c P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:50.986930 1173 tablet_bootstrap.cc:492] T 2b352d9a815e4bccafb7e2251500c5bc P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:50.987131 1173 ts_tablet_manager.cc:1403] T 2b352d9a815e4bccafb7e2251500c5bc P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.157s user 0.003s sys 0.000s
W20260202 02:52:50.987950 714 consensus_peers.cc:597] T a0b5d82e5fb94b60b4aa324b978a5378 P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a0b5d82e5fb94b60b4aa324b978a5378. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:50.989048 1173 raft_consensus.cc:359] T 2b352d9a815e4bccafb7e2251500c5bc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.989301 1173 raft_consensus.cc:385] T 2b352d9a815e4bccafb7e2251500c5bc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:50.989392 1173 raft_consensus.cc:740] T 2b352d9a815e4bccafb7e2251500c5bc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:50.989676 1173 consensus_queue.cc:260] T 2b352d9a815e4bccafb7e2251500c5bc P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:50.990368 1173 ts_tablet_manager.cc:1434] T 2b352d9a815e4bccafb7e2251500c5bc P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260202 02:52:50.990957 1173 tablet_bootstrap.cc:492] T 7ea7a5d8a92b4ac0903483627258ae66 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:50.993026 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_dbb40510d128424fa8b9b668c4f372c8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:50.993170 890 tablet_service.cc:1552] Processing DeleteTablet for tablet 29c996781cb947f3a1aee5dd0a9c4828 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:50.993610 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dbb40510d128424fa8b9b668c4f372c8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:51.000648 520 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0e0c43f15ba2437fb227a153241b4394 consensus state has no leader
I20260202 02:52:51.001776 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_0af7d8f82d2b4503aca40ada95dde3fc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:51.002395 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0af7d8f82d2b4503aca40ada95dde3fc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:51.003361 1882 ts_tablet_manager.cc:1929] T 79c73f4a92044bafb5b8384229cb3a4d P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:51.003472 1882 log.cc:1199] T 79c73f4a92044bafb5b8384229cb3a4d P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/79c73f4a92044bafb5b8384229cb3a4d
I20260202 02:52:51.003762 1882 ts_tablet_manager.cc:1950] T 79c73f4a92044bafb5b8384229cb3a4d P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:51.009315 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 738) took 3044 ms. Trace:
I20260202 02:52:51.009474 1882 rpcz_store.cc:276] 0202 02:52:47.964790 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:47.964856 (+ 66us) service_pool.cc:225] Handling call
0202 02:52:50.952089 (+2987233us) ts_tablet_manager.cc:1151] Deleting tablet 79c73f4a92044bafb5b8384229cb3a4d
0202 02:52:50.952102 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:50.959027 (+ 6925us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:51.003245 (+ 44218us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:51.009276 (+ 6031us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2902236}
I20260202 02:52:51.009986 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 739) took 3041 ms. Trace:
I20260202 02:52:51.010066 1882 rpcz_store.cc:276] 0202 02:52:47.968956 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:47.969038 (+ 82us) service_pool.cc:225] Handling call
0202 02:52:51.009843 (+3040805us) ts_tablet_manager.cc:1151] Deleting tablet dd8f36d7a82f47adae1bd3c1f8628b6e
0202 02:52:51.009855 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:51.009970 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2959938}
I20260202 02:52:51.010306 1882 tablet_replica.cc:333] T b7880cb051c24bc998003f0effed3cf8 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:51.010509 1882 raft_consensus.cc:2243] T b7880cb051c24bc998003f0effed3cf8 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:51.010803 1882 raft_consensus.cc:2272] T b7880cb051c24bc998003f0effed3cf8 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:51.013111 1173 tablet_bootstrap.cc:654] T 7ea7a5d8a92b4ac0903483627258ae66 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:51.015986 2633 raft_consensus.cc:493] T 93f958924885457ab15f2aac5509e0aa P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:51.016178 2633 raft_consensus.cc:515] T 93f958924885457ab15f2aac5509e0aa P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.019090 1175 tablet_bootstrap.cc:492] T 79badde1001b432b95f58b5161ace15c P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:51.019327 1175 ts_tablet_manager.cc:1403] T 79badde1001b432b95f58b5161ace15c P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.068s user 0.004s sys 0.000s
I20260202 02:52:51.020131 1175 raft_consensus.cc:359] T 79badde1001b432b95f58b5161ace15c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:51.020401 1175 raft_consensus.cc:385] T 79badde1001b432b95f58b5161ace15c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:51.020506 1175 raft_consensus.cc:740] T 79badde1001b432b95f58b5161ace15c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:51.021123 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 79c73f4a92044bafb5b8384229cb3a4d (table default.Table_a15c97bb32284dcfb36cdc47921b2346 [id=dc77705cb5704fd58af1eed3edd45675]) successfully deleted
I20260202 02:52:51.020807 1175 consensus_queue.cc:260] T 79badde1001b432b95f58b5161ace15c P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
W20260202 02:52:51.023998 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet dd8f36d7a82f47adae1bd3c1f8628b6e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dd8f36d7a82f47adae1bd3c1f8628b6e already in progress: creating tablet
I20260202 02:52:50.938992 2311 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" instance_seqno: 1770000732912730) as {username='slave'} at 127.0.104.130:36353; Asking this server to re-register.
I20260202 02:52:50.970984 2256 leader_election.cc:304] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7dab2abef14445d688ce1335b1f9b2f8; no voters: 92a8191cf1e14e4d99f1c378c2d3c85b, a9c3e3612bb94fb7b94c308041acfaeb
W20260202 02:52:51.026705 2256 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.188:39077, user_credentials={real_user=slave}} blocked reactor thread for 55779.3us
I20260202 02:52:51.027851 969 heartbeater.cc:461] Registering TS with master...
I20260202 02:52:51.030050 2739 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260202 02:52:50.983635 2313 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "b1f05820b6034228bdad884cc348716d" instance_seqno: 1770000733441128) as {username='slave'} at 127.0.104.131:48919; Asking this server to re-register.
I20260202 02:52:51.031777 2311 ts_manager.cc:194] Registered new tserver with Master: 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:51.033991 1110 heartbeater.cc:461] Registering TS with master...
W20260202 02:52:51.038771 528 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4453a3e7ef784a528d8683e28d14e604 consensus state has no leader
I20260202 02:52:51.039108 528 master_service.cc:954] ReplaceTablet: received request to replace tablet ad7c98afaa3447d1aac8a243c6a98ed6 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:51.040495 2311 ts_manager.cc:194] Registered new tserver with Master: b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:51.041321 1182 ts_tablet_manager.cc:1929] T 2b352d9a815e4bccafb7e2251500c5bc P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:51.041435 1182 log.cc:1199] T 2b352d9a815e4bccafb7e2251500c5bc P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/2b352d9a815e4bccafb7e2251500c5bc
I20260202 02:52:51.044234 1182 ts_tablet_manager.cc:1950] T 2b352d9a815e4bccafb7e2251500c5bc P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:51.045433 2633 leader_election.cc:290] T 93f958924885457ab15f2aac5509e0aa P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:51.045938 714 leader_election.cc:343] T 93f958924885457ab15f2aac5509e0aa P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 93f958924885457ab15f2aac5509e0aa
W20260202 02:52:51.046085 715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 126463us
I20260202 02:52:51.047151 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "93f958924885457ab15f2aac5509e0aa" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:51.051092 714 consensus_peers.cc:597] T b7880cb051c24bc998003f0effed3cf8 P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7880cb051c24bc998003f0effed3cf8. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:51.051985 715 leader_election.cc:304] T 65bd1c1a382e42e4a68077264563c55b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:51.052024 2311 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "1f442ef85029461091de847118c34c0d" instance_seqno: 1770000732390773) as {username='slave'} at 127.0.104.129:44355; Asking this server to re-register.
I20260202 02:52:51.052909 2633 raft_consensus.cc:2804] T 65bd1c1a382e42e4a68077264563c55b P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:51.053218 2633 raft_consensus.cc:697] T 65bd1c1a382e42e4a68077264563c55b P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:51.053508 2633 consensus_queue.cc:237] T 65bd1c1a382e42e4a68077264563c55b P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.054067 715 leader_election.cc:304] T ccd6bc8bd0504bf18d108c73685ef9ff P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:51.055294 2884 raft_consensus.cc:2804] T ccd6bc8bd0504bf18d108c73685ef9ff P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:51.055375 2884 raft_consensus.cc:493] T ccd6bc8bd0504bf18d108c73685ef9ff P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:51.055437 2884 raft_consensus.cc:3060] T ccd6bc8bd0504bf18d108c73685ef9ff P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:51.056341 715 leader_election.cc:343] T da5435f969074364bc8f8a6ddc5e4f8a P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:51.057121 715 leader_election.cc:304] T da5435f969074364bc8f8a6ddc5e4f8a P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:51.057830 2619 raft_consensus.cc:2749] T da5435f969074364bc8f8a6ddc5e4f8a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:51.058291 715 leader_election.cc:343] T 842e54adae2a452a98521de24326eaff P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: Tablet not RUNNING: SHUTDOWN
I20260202 02:52:51.059151 715 leader_election.cc:304] T 842e54adae2a452a98521de24326eaff P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:51.059885 2619 raft_consensus.cc:2749] T 842e54adae2a452a98521de24326eaff P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:51.060230 523 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 8153510d5d154c7895b1ee794585612e (table default.Table_0e7ceba4956f408c8a9fc0f61fa67c90 [id=c0c6499675894350ac008b534d030c93]). Expected version 2 got 0
I20260202 02:52:51.060312 715 leader_election.cc:304] T 474df41e4fe24a818fa45a4ce84d5474 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:51.060681 519 catalog_manager.cc:5654] T 44c8d5b74c8e4c499f7f68e8185efc17 P b1f05820b6034228bdad884cc348716d reported cstate change: term changed from 0 to 1, leader changed from <none> to b1f05820b6034228bdad884cc348716d (127.0.104.131). New cstate: current_term: 1 leader_uuid: "b1f05820b6034228bdad884cc348716d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: HEALTHY } } }
I20260202 02:52:51.061124 519 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 44c8d5b74c8e4c499f7f68e8185efc17 (table default.Table_f48ed1c7e02349adad7ce22c0e51ed81 [id=46b5d47377474ff9bac2acdb26b08c65]). Expected version 3 got 0
I20260202 02:52:51.061519 2619 raft_consensus.cc:2804] T 474df41e4fe24a818fa45a4ce84d5474 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:51.061623 2619 raft_consensus.cc:697] T 474df41e4fe24a818fa45a4ce84d5474 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:51.062399 890 tablet_service.cc:1552] Processing DeleteTablet for tablet 2f5aae089fcc492f9ce55f84674a452c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:51.063467 715 leader_election.cc:343] T 93f958924885457ab15f2aac5509e0aa P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:51.063942 715 leader_election.cc:304] T 93f958924885457ab15f2aac5509e0aa P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:51.064261 1882 ts_tablet_manager.cc:1916] T b7880cb051c24bc998003f0effed3cf8 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:51.068074 1175 ts_tablet_manager.cc:1434] T 79badde1001b432b95f58b5161ace15c P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.049s user 0.002s sys 0.000s
I20260202 02:52:51.069730 890 tablet_service.cc:1505] Processing CreateTablet for tablet 22414f88afc444fd8d0fa241782a3803 (DEFAULT_TABLE table=default.Table_a17341b47cbe40b5863c554e7f56daaf [id=17fbdd452dbb4f599eb4212b987a1f27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.070267 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22414f88afc444fd8d0fa241782a3803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.071080 1035 tablet_service.cc:1505] Processing CreateTablet for tablet 22414f88afc444fd8d0fa241782a3803 (DEFAULT_TABLE table=default.Table_a17341b47cbe40b5863c554e7f56daaf [id=17fbdd452dbb4f599eb4212b987a1f27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.071344 750 tablet_service.cc:1505] Processing CreateTablet for tablet 22414f88afc444fd8d0fa241782a3803 (DEFAULT_TABLE table=default.Table_a17341b47cbe40b5863c554e7f56daaf [id=17fbdd452dbb4f599eb4212b987a1f27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.071447 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22414f88afc444fd8d0fa241782a3803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.072369 896 tablet_service.cc:1505] Processing CreateTablet for tablet 001900ceb27047f4a8a6a4f2221ead7c (DEFAULT_TABLE table=default.Table_ca027855a06748118a962c4fdadfd6e8 [id=0a697f9c58cc4e7fa0f6ef3c6aaac04f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.073089 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 001900ceb27047f4a8a6a4f2221ead7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:50.966338 2883 raft_consensus.cc:515] T 1d8143f529b74e6ab779dd695920413c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.080444 740 tablet_service.cc:1552] Processing DeleteTablet for tablet da5435f969074364bc8f8a6ddc5e4f8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:51.080870 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22414f88afc444fd8d0fa241782a3803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.082331 2858 raft_consensus.cc:493] T f3eebefa74584c1791d52377f5ba1290 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:51.082412 1175 tablet_bootstrap.cc:492] T b427c7e897c64977b924e7d586420e69 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:51.082473 2858 raft_consensus.cc:515] T f3eebefa74584c1791d52377f5ba1290 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.083105 2858 leader_election.cc:290] T f3eebefa74584c1791d52377f5ba1290 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:51.061853 2619 consensus_queue.cc:237] T 474df41e4fe24a818fa45a4ce84d5474 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:51.084703 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f3eebefa74584c1791d52377f5ba1290" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:51.086107 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_c8d5b0bad2904df78c01b87bbd5b8e7f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:51.086760 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c8d5b0bad2904df78c01b87bbd5b8e7f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:51.087142 859 leader_election.cc:343] T f3eebefa74584c1791d52377f5ba1290 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:51.087963 2878 raft_consensus.cc:2749] T 93f958924885457ab15f2aac5509e0aa P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:51.088148 525 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ab48976710ec44e1a6b293a951fab152" } new_table_name: "default.Table_4f4675b117e54fa99feb03a4d8300099" modify_external_catalogs: true
I20260202 02:52:51.088536 2884 raft_consensus.cc:515] T ccd6bc8bd0504bf18d108c73685ef9ff P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:51.088600 740 tablet_service.cc:1505] Processing CreateTablet for tablet 001900ceb27047f4a8a6a4f2221ead7c (DEFAULT_TABLE table=default.Table_ca027855a06748118a962c4fdadfd6e8 [id=0a697f9c58cc4e7fa0f6ef3c6aaac04f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.088994 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 001900ceb27047f4a8a6a4f2221ead7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.084317 1175 tablet_bootstrap.cc:654] T b427c7e897c64977b924e7d586420e69 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:51.091800 2883 leader_election.cc:290] T 1d8143f529b74e6ab779dd695920413c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:51.092697 919 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1d8143f529b74e6ab779dd695920413c" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:51.092936 919 raft_consensus.cc:2468] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:51.093919 714 leader_election.cc:304] T 1d8143f529b74e6ab779dd695920413c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters:
I20260202 02:52:51.093979 1032 tablet_service.cc:1505] Processing CreateTablet for tablet 001900ceb27047f4a8a6a4f2221ead7c (DEFAULT_TABLE table=default.Table_ca027855a06748118a962c4fdadfd6e8 [id=0a697f9c58cc4e7fa0f6ef3c6aaac04f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.094398 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 001900ceb27047f4a8a6a4f2221ead7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.098553 1173 tablet_bootstrap.cc:492] T 7ea7a5d8a92b4ac0903483627258ae66 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:51.098754 1173 ts_tablet_manager.cc:1403] T 7ea7a5d8a92b4ac0903483627258ae66 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.108s user 0.001s sys 0.002s
I20260202 02:52:51.100514 1173 raft_consensus.cc:359] T 7ea7a5d8a92b4ac0903483627258ae66 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.101109 1173 raft_consensus.cc:385] T 7ea7a5d8a92b4ac0903483627258ae66 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:51.101243 1173 raft_consensus.cc:740] T 7ea7a5d8a92b4ac0903483627258ae66 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:51.101526 1173 consensus_queue.cc:260] T 7ea7a5d8a92b4ac0903483627258ae66 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.102296 1173 ts_tablet_manager.cc:1434] T 7ea7a5d8a92b4ac0903483627258ae66 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260202 02:52:51.102588 1173 tablet_bootstrap.cc:492] T 3284213e4b0447b49a7718ba896d3fa8 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:51.110447 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 2b352d9a815e4bccafb7e2251500c5bc (table default.Table_408b8f21ff294ce9880d0b9a4d681f93 [id=c0fa167c7d1f47ea994fcf7dfdd4100f]) successfully deleted
I20260202 02:52:51.110913 1182 tablet_replica.cc:333] T 7ea7a5d8a92b4ac0903483627258ae66 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:51.111105 1182 raft_consensus.cc:2243] T 7ea7a5d8a92b4ac0903483627258ae66 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:51.111346 1182 raft_consensus.cc:2272] T 7ea7a5d8a92b4ac0903483627258ae66 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:51.112005 1182 ts_tablet_manager.cc:1916] T 7ea7a5d8a92b4ac0903483627258ae66 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:51.117158 1175 tablet_bootstrap.cc:492] T b427c7e897c64977b924e7d586420e69 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:51.117304 1175 ts_tablet_manager.cc:1403] T b427c7e897c64977b924e7d586420e69 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.035s user 0.003s sys 0.000s
I20260202 02:52:51.117671 896 tablet_service.cc:1505] Processing CreateTablet for tablet 94957936d8e644929a52b0b2a4ddba02 (DEFAULT_TABLE table=default.Table_12cb0a1d830f490dab4efb00188ddecb [id=874abd1d3dff4d148217ba5f41775bf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.117969 1175 raft_consensus.cc:359] T b427c7e897c64977b924e7d586420e69 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.118090 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94957936d8e644929a52b0b2a4ddba02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.118124 1175 raft_consensus.cc:385] T b427c7e897c64977b924e7d586420e69 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:51.118168 1175 raft_consensus.cc:740] T b427c7e897c64977b924e7d586420e69 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:51.118362 1175 consensus_queue.cc:260] T b427c7e897c64977b924e7d586420e69 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.119072 1175 ts_tablet_manager.cc:1434] T b427c7e897c64977b924e7d586420e69 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:51.119400 1175 tablet_bootstrap.cc:492] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:51.121169 1175 tablet_bootstrap.cc:654] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:51.126801 1032 tablet_service.cc:1505] Processing CreateTablet for tablet 94957936d8e644929a52b0b2a4ddba02 (DEFAULT_TABLE table=default.Table_12cb0a1d830f490dab4efb00188ddecb [id=874abd1d3dff4d148217ba5f41775bf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.127007 753 tablet_service.cc:1505] Processing CreateTablet for tablet 94957936d8e644929a52b0b2a4ddba02 (DEFAULT_TABLE table=default.Table_12cb0a1d830f490dab4efb00188ddecb [id=874abd1d3dff4d148217ba5f41775bf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.127256 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94957936d8e644929a52b0b2a4ddba02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.127225 890 tablet_service.cc:1505] Processing CreateTablet for tablet 1d94d7ebb29e4f329e059c7919d7b315 (DEFAULT_TABLE table=default.Table_7a2b9560ae174e75a43d23cf0e3f708f [id=c49f344483b141d3bf3c8c44edcc5680]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.127398 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94957936d8e644929a52b0b2a4ddba02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.127997 1035 tablet_service.cc:1505] Processing CreateTablet for tablet 1d94d7ebb29e4f329e059c7919d7b315 (DEFAULT_TABLE table=default.Table_7a2b9560ae174e75a43d23cf0e3f708f [id=c49f344483b141d3bf3c8c44edcc5680]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.128371 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d94d7ebb29e4f329e059c7919d7b315. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.128654 902 tablet_service.cc:1505] Processing CreateTablet for tablet 246d17116713466e8ea3968bd8bfaed4 (DEFAULT_TABLE table=default.Table_5215b49fa20a4fff88c538478889acd0 [id=6b89a856daa2433d9693c5912e7256d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.129479 755 tablet_service.cc:1505] Processing CreateTablet for tablet 246d17116713466e8ea3968bd8bfaed4 (DEFAULT_TABLE table=default.Table_5215b49fa20a4fff88c538478889acd0 [id=6b89a856daa2433d9693c5912e7256d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.129860 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 246d17116713466e8ea3968bd8bfaed4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.129956 884 tablet_service.cc:1505] Processing CreateTablet for tablet 981462ab4d7d462497b34e688b576990 (DEFAULT_TABLE table=default.Table_3133535fffb44f4fb4f2d30a4ab61f55 [id=fbda6605b14d4f2f97f3b564819c2ae4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.131853 751 tablet_service.cc:1505] Processing CreateTablet for tablet 981462ab4d7d462497b34e688b576990 (DEFAULT_TABLE table=default.Table_3133535fffb44f4fb4f2d30a4ab61f55 [id=fbda6605b14d4f2f97f3b564819c2ae4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.132206 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 981462ab4d7d462497b34e688b576990. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.134889 1031 tablet_service.cc:1505] Processing CreateTablet for tablet 246d17116713466e8ea3968bd8bfaed4 (DEFAULT_TABLE table=default.Table_5215b49fa20a4fff88c538478889acd0 [id=6b89a856daa2433d9693c5912e7256d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.135320 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 246d17116713466e8ea3968bd8bfaed4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.135648 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_75aa5c9f446e4b6898acfc002ee4e4a9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:51.135748 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 981462ab4d7d462497b34e688b576990 (DEFAULT_TABLE table=default.Table_3133535fffb44f4fb4f2d30a4ab61f55 [id=fbda6605b14d4f2f97f3b564819c2ae4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.136071 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 981462ab4d7d462497b34e688b576990. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:51.136222 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_75aa5c9f446e4b6898acfc002ee4e4a9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:51.137965 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 79badde1001b432b95f58b5161ace15c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:51.141901 423 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 51528.5us
I20260202 02:52:51.146903 1173 tablet_bootstrap.cc:654] T 3284213e4b0447b49a7718ba896d3fa8 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:51.147079 2858 raft_consensus.cc:493] T 2f5aae089fcc492f9ce55f84674a452c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:51.147220 2858 raft_consensus.cc:515] T 2f5aae089fcc492f9ce55f84674a452c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:51.147876 2858 leader_election.cc:290] T 2f5aae089fcc492f9ce55f84674a452c P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:51.150894 1175 tablet_bootstrap.cc:492] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:51.151051 1175 ts_tablet_manager.cc:1403] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.032s user 0.000s sys 0.006s
I20260202 02:52:51.150959 520 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_fdd03abf2a0e4d3a8f621f28cc062904" } new_table_name: "default.Table_8bc5510351e741f98d764c2ddec2d757" modify_external_catalogs: true
I20260202 02:52:51.154968 1174 tablet_bootstrap.cc:654] T 939fbcaf75c7439d9208bbbf83a72717 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:51.151826 1175 raft_consensus.cc:359] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.157083 1175 raft_consensus.cc:385] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:51.157162 1175 raft_consensus.cc:740] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:51.158463 1175 consensus_queue.cc:260] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.158716 524 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 23dfa02219d748919e3cf24beb2095be
I20260202 02:52:51.159370 524 master_service.cc:954] ReplaceTablet: received request to replace tablet 9d0212e64d934968bdbdc627c69874b1 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:51.160243 1175 ts_tablet_manager.cc:1434] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.009s user 0.000s sys 0.002s
I20260202 02:52:51.160801 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet 23dfa02219d748919e3cf24beb2095be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:50 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:51.161059 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 23dfa02219d748919e3cf24beb2095be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:50 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:51.165194 524 master_service.cc:954] ReplaceTablet: received request to replace tablet 21fe651f45bf43609cf972bef5eca149 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:51.166527 2858 raft_consensus.cc:493] T b427c7e897c64977b924e7d586420e69 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:51.166687 2858 raft_consensus.cc:515] T b427c7e897c64977b924e7d586420e69 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.167351 2858 leader_election.cc:290] T b427c7e897c64977b924e7d586420e69 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:51.168452 2878 raft_consensus.cc:2804] T 1d8143f529b74e6ab779dd695920413c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:51.172475 747 tablet_service.cc:1505] Processing CreateTablet for tablet 1d94d7ebb29e4f329e059c7919d7b315 (DEFAULT_TABLE table=default.Table_7a2b9560ae174e75a43d23cf0e3f708f [id=c49f344483b141d3bf3c8c44edcc5680]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.177685 521 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 0da286b7f598457ca4d0254a7073cb74
I20260202 02:52:51.179061 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 0da286b7f598457ca4d0254a7073cb74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:50 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:51.183818 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_32235d7fa05f42efb99094f7b8d3c93b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:51.184777 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_32235d7fa05f42efb99094f7b8d3c93b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:51.187942 886 tablet_service.cc:1505] Processing CreateTablet for tablet 3a15c5dfc3b24dbd8ef3ef7f3158edb2 (DEFAULT_TABLE table=default.Table_3638d0d39d6943e28507ce3f8aaf5829 [id=50e5158b6eab496e9e6693dc275b172f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.194698 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 981462ab4d7d462497b34e688b576990. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.195099 2884 leader_election.cc:290] T ccd6bc8bd0504bf18d108c73685ef9ff P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:51.196031 2740 raft_consensus.cc:493] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:51.196187 2740 raft_consensus.cc:515] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.196597 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 246d17116713466e8ea3968bd8bfaed4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.200605 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d94d7ebb29e4f329e059c7919d7b315. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.200825 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a15c5dfc3b24dbd8ef3ef7f3158edb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.205739 1175 tablet_bootstrap.cc:492] T 07580ad1d049475a9645acfad0b1d219 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:51.206833 740 tablet_service.cc:1552] Processing DeleteTablet for tablet 0da286b7f598457ca4d0254a7073cb74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:50 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:51.207271 1175 tablet_bootstrap.cc:654] T 07580ad1d049475a9645acfad0b1d219 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:51.210417 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet 0da286b7f598457ca4d0254a7073cb74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:50 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:51.211023 2740 leader_election.cc:290] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:51.216830 999 leader_election.cc:343] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: d0ce2121d65b4a228596e0ca9018883d
I20260202 02:52:51.218091 526 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 0aa023e74e74480498b721bbb5d15052
I20260202 02:52:51.219460 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b427c7e897c64977b924e7d586420e69" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:51.219808 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 3a15c5dfc3b24dbd8ef3ef7f3158edb2 (DEFAULT_TABLE table=default.Table_3638d0d39d6943e28507ce3f8aaf5829 [id=50e5158b6eab496e9e6693dc275b172f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.220199 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a15c5dfc3b24dbd8ef3ef7f3158edb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.219460 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f5aae089fcc492f9ce55f84674a452c" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:51.221932 859 leader_election.cc:343] T b427c7e897c64977b924e7d586420e69 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:51.222147 859 leader_election.cc:343] T 2f5aae089fcc492f9ce55f84674a452c P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:51.222302 1043 tablet_service.cc:1552] Processing DeleteTablet for tablet 0aa023e74e74480498b721bbb5d15052 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:50 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:51.226370 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 0aa023e74e74480498b721bbb5d15052 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:50 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:51.226945 2878 raft_consensus.cc:493] T 1d8143f529b74e6ab779dd695920413c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:51.228958 526 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet b427c7e897c64977b924e7d586420e69 (table default.Table_616a8ade4b5e481b8d4906b7a2a026d0 [id=98ee1a7044cd46d8b39dfda755ed8a7d]). Expected version 5 got 0
I20260202 02:52:51.230285 1043 tablet_service.cc:1505] Processing CreateTablet for tablet 4b502efa0a4c423b8ab2341931ef96aa (DEFAULT_TABLE table=default.Table_ef8313f59c0d44e3bb7ec522181f6c25 [id=db48c5322cf44770876f1d56cda035b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.230731 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b502efa0a4c423b8ab2341931ef96aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.231570 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d94d7ebb29e4f329e059c7919d7b315. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:51.232378 714 leader_election.cc:343] T ccd6bc8bd0504bf18d108c73685ef9ff P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: ccd6bc8bd0504bf18d108c73685ef9ff
I20260202 02:52:51.233546 886 tablet_service.cc:1505] Processing CreateTablet for tablet 4b502efa0a4c423b8ab2341931ef96aa (DEFAULT_TABLE table=default.Table_ef8313f59c0d44e3bb7ec522181f6c25 [id=db48c5322cf44770876f1d56cda035b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.245330 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b502efa0a4c423b8ab2341931ef96aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.245873 2740 raft_consensus.cc:493] T 939fbcaf75c7439d9208bbbf83a72717 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:51.244716 884 tablet_service.cc:1505] Processing CreateTablet for tablet 943835f6f43e41c99480295fbbb65adb (DEFAULT_TABLE table=default.Table_672611622d8d492f97aa14af71206575 [id=3c66c314193a4a6996d890303a273bd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.246320 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 943835f6f43e41c99480295fbbb65adb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.246022 2740 raft_consensus.cc:515] T 939fbcaf75c7439d9208bbbf83a72717 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:51.249094 2740 leader_election.cc:290] T 939fbcaf75c7439d9208bbbf83a72717 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:51.249647 919 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "939fbcaf75c7439d9208bbbf83a72717" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:51.249868 919 raft_consensus.cc:2468] T 939fbcaf75c7439d9208bbbf83a72717 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:51.250542 999 leader_election.cc:304] T 939fbcaf75c7439d9208bbbf83a72717 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:51.250854 2740 raft_consensus.cc:2804] T 939fbcaf75c7439d9208bbbf83a72717 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:51.250937 2740 raft_consensus.cc:493] T 939fbcaf75c7439d9208bbbf83a72717 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:51.251008 2740 raft_consensus.cc:3060] T 939fbcaf75c7439d9208bbbf83a72717 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:51.260025 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_7f793bd143b642d98a4f147a032541d3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:51.260792 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7f793bd143b642d98a4f147a032541d3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:51.261289 1182 ts_tablet_manager.cc:1929] T 7ea7a5d8a92b4ac0903483627258ae66 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:51.261399 1182 log.cc:1199] T 7ea7a5d8a92b4ac0903483627258ae66 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/7ea7a5d8a92b4ac0903483627258ae66
I20260202 02:52:51.261672 1182 ts_tablet_manager.cc:1950] T 7ea7a5d8a92b4ac0903483627258ae66 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:51.262400 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 7ea7a5d8a92b4ac0903483627258ae66 (table default.Table_d040708acb92480496bac8b012890006 [id=1c26d457c06345b7abeb1941bcee6c31]) successfully deleted
I20260202 02:52:51.262948 2740 raft_consensus.cc:515] T 939fbcaf75c7439d9208bbbf83a72717 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
W20260202 02:52:51.264442 512 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet de76ca9a98c840d0967a97154b940b60 (table default.Table_c57473bfb15e41da8627401aafd82bce [id=277a1bdd805e49a794c46be1f9a03316], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet de76ca9a98c840d0967a97154b940b60 consensus state has no leader
I20260202 02:52:51.264422 2740 leader_election.cc:290] T 939fbcaf75c7439d9208bbbf83a72717 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:51.264474 2794 raft_consensus.cc:493] T 2b352d9a815e4bccafb7e2251500c5bc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:51.264606 2794 raft_consensus.cc:515] T 2b352d9a815e4bccafb7e2251500c5bc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.264696 919 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "939fbcaf75c7439d9208bbbf83a72717" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:51.264874 919 raft_consensus.cc:3060] T 939fbcaf75c7439d9208bbbf83a72717 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:51.265151 2794 leader_election.cc:290] T 2b352d9a815e4bccafb7e2251500c5bc P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:51.265264 1035 tablet_service.cc:1505] Processing CreateTablet for tablet 943835f6f43e41c99480295fbbb65adb (DEFAULT_TABLE table=default.Table_672611622d8d492f97aa14af71206575 [id=3c66c314193a4a6996d890303a273bd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.265640 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 943835f6f43e41c99480295fbbb65adb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:51.271697 999 leader_election.cc:343] T 2b352d9a815e4bccafb7e2251500c5bc P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 2b352d9a815e4bccafb7e2251500c5bc
I20260202 02:52:51.278553 1173 tablet_bootstrap.cc:492] T 3284213e4b0447b49a7718ba896d3fa8 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:51.278724 1173 ts_tablet_manager.cc:1403] T 3284213e4b0447b49a7718ba896d3fa8 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.176s user 0.001s sys 0.005s
I20260202 02:52:51.279986 1173 raft_consensus.cc:359] T 3284213e4b0447b49a7718ba896d3fa8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.280172 1173 raft_consensus.cc:385] T 3284213e4b0447b49a7718ba896d3fa8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:51.280223 1173 raft_consensus.cc:740] T 3284213e4b0447b49a7718ba896d3fa8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:51.280519 1173 consensus_queue.cc:260] T 3284213e4b0447b49a7718ba896d3fa8 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.281181 1173 ts_tablet_manager.cc:1434] T 3284213e4b0447b49a7718ba896d3fa8 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260202 02:52:51.281471 1173 tablet_bootstrap.cc:492] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:51.283231 1173 tablet_bootstrap.cc:654] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:51.296164 919 raft_consensus.cc:2468] T 939fbcaf75c7439d9208bbbf83a72717 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:51.297223 528 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet ad7c98afaa3447d1aac8a243c6a98ed6
I20260202 02:52:51.297412 999 leader_election.cc:304] T 939fbcaf75c7439d9208bbbf83a72717 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:51.297755 2794 raft_consensus.cc:2804] T 939fbcaf75c7439d9208bbbf83a72717 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:51.297809 528 catalog_manager.cc:6724] ReplaceTablet: tablet ad7c98afaa3447d1aac8a243c6a98ed6 deleted and replaced by tablet d23d2b52f058471bb0da6e2608016fa5
I20260202 02:52:51.297891 2794 raft_consensus.cc:697] T 939fbcaf75c7439d9208bbbf83a72717 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:51.298136 2794 consensus_queue.cc:237] T 939fbcaf75c7439d9208bbbf83a72717 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:51.297223 527 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet fea0fcb8854c4fc6acda80c8ffb8d341
I20260202 02:52:51.300048 1031 tablet_service.cc:1552] Processing DeleteTablet for tablet ad7c98afaa3447d1aac8a243c6a98ed6 with delete_type TABLET_DATA_DELETED (replaced by tablet d23d2b52f058471bb0da6e2608016fa5) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:51.300542 902 tablet_service.cc:1552] Processing DeleteTablet for tablet fea0fcb8854c4fc6acda80c8ffb8d341 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:50 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:51.301599 884 tablet_service.cc:1552] Processing DeleteTablet for tablet ad7c98afaa3447d1aac8a243c6a98ed6 with delete_type TABLET_DATA_DELETED (replaced by tablet d23d2b52f058471bb0da6e2608016fa5) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:51.302693 1173 tablet_bootstrap.cc:492] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:51.302906 1173 ts_tablet_manager.cc:1403] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.005s
I20260202 02:52:51.303719 1173 raft_consensus.cc:359] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:51.303977 1173 raft_consensus.cc:385] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:51.304073 1173 raft_consensus.cc:740] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:51.304401 1173 consensus_queue.cc:260] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:51.305070 1173 ts_tablet_manager.cc:1434] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:51.306075 1173 tablet_bootstrap.cc:492] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:51.307813 1173 tablet_bootstrap.cc:654] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:51.313876 1882 ts_tablet_manager.cc:1929] T b7880cb051c24bc998003f0effed3cf8 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:51.314009 1882 log.cc:1199] T b7880cb051c24bc998003f0effed3cf8 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/b7880cb051c24bc998003f0effed3cf8
I20260202 02:52:51.314320 1882 ts_tablet_manager.cc:1950] T b7880cb051c24bc998003f0effed3cf8 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:51.325176 1175 tablet_bootstrap.cc:492] T 07580ad1d049475a9645acfad0b1d219 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:51.325354 1175 ts_tablet_manager.cc:1403] T 07580ad1d049475a9645acfad0b1d219 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.120s user 0.002s sys 0.009s
I20260202 02:52:51.326174 1175 raft_consensus.cc:359] T 07580ad1d049475a9645acfad0b1d219 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.326372 1175 raft_consensus.cc:385] T 07580ad1d049475a9645acfad0b1d219 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:51.326440 1175 raft_consensus.cc:740] T 07580ad1d049475a9645acfad0b1d219 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:51.326668 1175 consensus_queue.cc:260] T 07580ad1d049475a9645acfad0b1d219 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.327316 1175 ts_tablet_manager.cc:1434] T 07580ad1d049475a9645acfad0b1d219 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:51.329941 1175 tablet_bootstrap.cc:492] T 25a407001f964da6b51f42375136c79a P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:51.331605 1175 tablet_bootstrap.cc:654] T 25a407001f964da6b51f42375136c79a P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:51.333452 884 tablet_service.cc:1505] Processing CreateTablet for tablet 4794a8ef6f22462d8212f3610d22acd9 (DEFAULT_TABLE table=default.Table_a4b8204d7f474e539edf79a5e07fd706 [id=441f6f6d1057454d85f520e30439eba0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.333876 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4794a8ef6f22462d8212f3610d22acd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.336673 1031 tablet_service.cc:1505] Processing CreateTablet for tablet 4794a8ef6f22462d8212f3610d22acd9 (DEFAULT_TABLE table=default.Table_a4b8204d7f474e539edf79a5e07fd706 [id=441f6f6d1057454d85f520e30439eba0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.339552 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 829) took 1029 ms. Trace:
I20260202 02:52:51.341625 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 64e2fcb966fc4a06afb65506be1711a2 (DEFAULT_TABLE table=default.Table_7beba71f16ad4551b219e1a1e8b99a62 [id=3725d264950f46eabe35b6ee4a7c96c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.341284 884 tablet_service.cc:1505] Processing CreateTablet for tablet 64e2fcb966fc4a06afb65506be1711a2 (DEFAULT_TABLE table=default.Table_7beba71f16ad4551b219e1a1e8b99a62 [id=3725d264950f46eabe35b6ee4a7c96c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.341733 1182 rpcz_store.cc:276] 0202 02:52:50.310387 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:50.310860 (+ 473us) service_pool.cc:225] Handling call
0202 02:52:51.339391 (+1028531us) ts_tablet_manager.cc:1151] Deleting tablet 39eaa2848cf4450dbe563ddd3b11df06
0202 02:52:51.339405 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:51.339521 (+ 116us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1028392}
W20260202 02:52:51.343086 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 39eaa2848cf4450dbe563ddd3b11df06 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 39eaa2848cf4450dbe563ddd3b11df06 already in progress: creating tablet
I20260202 02:52:51.344430 1043 tablet_service.cc:1552] Processing DeleteTablet for tablet fea0fcb8854c4fc6acda80c8ffb8d341 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:50 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:51.344511 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64e2fcb966fc4a06afb65506be1711a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.345521 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4794a8ef6f22462d8212f3610d22acd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.347452 1175 tablet_bootstrap.cc:492] T 25a407001f964da6b51f42375136c79a P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:51.347584 1175 ts_tablet_manager.cc:1403] T 25a407001f964da6b51f42375136c79a P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.003s
I20260202 02:52:51.348273 1175 raft_consensus.cc:359] T 25a407001f964da6b51f42375136c79a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.348450 1175 raft_consensus.cc:385] T 25a407001f964da6b51f42375136c79a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:51.348500 1175 raft_consensus.cc:740] T 25a407001f964da6b51f42375136c79a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:51.348584 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64e2fcb966fc4a06afb65506be1711a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.348690 1175 consensus_queue.cc:260] T 25a407001f964da6b51f42375136c79a P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.353500 902 tablet_service.cc:1505] Processing CreateTablet for tablet 922d887f3bf2419f9306d3a2aafeb280 (DEFAULT_TABLE table=default.Table_b8aab442fd004224a0e08473c4f2b075 [id=0a41afd164554a309620558b448e0206]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.354593 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 68f6fb6be6a0414889cc1b635da35ec6 (DEFAULT_TABLE table=default.Table_de3de7c394c64794af3b8bffdc413699 [id=fcde662479a14899a24f1d7284d04bba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.354995 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68f6fb6be6a0414889cc1b635da35ec6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.355460 890 tablet_service.cc:1505] Processing CreateTablet for tablet 68f6fb6be6a0414889cc1b635da35ec6 (DEFAULT_TABLE table=default.Table_de3de7c394c64794af3b8bffdc413699 [id=fcde662479a14899a24f1d7284d04bba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.356283 1175 ts_tablet_manager.cc:1434] T 25a407001f964da6b51f42375136c79a P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260202 02:52:51.360083 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 743) took 3141 ms. Trace:
I20260202 02:52:51.360193 1882 rpcz_store.cc:276] 0202 02:52:48.218591 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:48.218661 (+ 70us) service_pool.cc:225] Handling call
0202 02:52:51.010261 (+2791600us) ts_tablet_manager.cc:1151] Deleting tablet b7880cb051c24bc998003f0effed3cf8
0202 02:52:51.010269 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:51.094355 (+ 84086us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:51.313752 (+219397us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:51.360057 (+ 46305us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2791397}
I20260202 02:52:51.360643 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 744) took 3022 ms. Trace:
I20260202 02:52:51.360711 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet b7880cb051c24bc998003f0effed3cf8 (table default.Table_a0f1f2d30d064244bec97181c11c3a06 [id=1787be853d4a40cdada2c2735cad1a01]) successfully deleted
I20260202 02:52:51.360723 1882 rpcz_store.cc:276] 0202 02:52:48.338646 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:48.343460 (+ 4814us) service_pool.cc:225] Handling call
0202 02:52:51.360501 (+3017041us) ts_tablet_manager.cc:1151] Deleting tablet 3338964dfed548f980f58147f0283691
0202 02:52:51.360511 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:51.360628 (+ 117us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3016841}
I20260202 02:52:51.360968 1882 tablet_replica.cc:333] T 550895b4010f4088b2cc34acff50e03f P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:51.361135 1882 raft_consensus.cc:2243] T 550895b4010f4088b2cc34acff50e03f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:51.361282 1882 raft_consensus.cc:2272] T 550895b4010f4088b2cc34acff50e03f P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:51.361331 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 3338964dfed548f980f58147f0283691 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3338964dfed548f980f58147f0283691 already in progress: creating tablet
I20260202 02:52:51.361819 1882 ts_tablet_manager.cc:1916] T 550895b4010f4088b2cc34acff50e03f P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:51.362478 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 1139db8ea6744e168e7b223c23665c58 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1139db8ea6744e168e7b223c23665c58 already in progress: creating tablet
W20260202 02:52:51.362874 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 87ff5fbc95b84171a94fe1a83a67a7ba because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 87ff5fbc95b84171a94fe1a83a67a7ba already in progress: creating tablet
W20260202 02:52:51.363265 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet e04960479c92479b8bb864544dd0c8ac because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e04960479c92479b8bb864544dd0c8ac already in progress: creating tablet
W20260202 02:52:51.363652 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 82680e577c504a729122cd4f6b4916a6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 82680e577c504a729122cd4f6b4916a6 already in progress: creating tablet
I20260202 02:52:51.360643 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68f6fb6be6a0414889cc1b635da35ec6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.364099 1175 tablet_bootstrap.cc:492] T 7a44585a217b48f39f247bae3ca2e255 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:51.364316 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 922d887f3bf2419f9306d3a2aafeb280. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.365659 1175 tablet_bootstrap.cc:654] T 7a44585a217b48f39f247bae3ca2e255 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:51.366346 1043 tablet_service.cc:1505] Processing CreateTablet for tablet 922d887f3bf2419f9306d3a2aafeb280 (DEFAULT_TABLE table=default.Table_b8aab442fd004224a0e08473c4f2b075 [id=0a41afd164554a309620558b448e0206]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.366751 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 922d887f3bf2419f9306d3a2aafeb280. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.369614 896 tablet_service.cc:1505] Processing CreateTablet for tablet c3ae67eb26f84587b274c1882194c3a2 (DEFAULT_TABLE table=default.Table_eaefe83cde2b48418b7a3407edb48f5e [id=8934bcbc29424ecbb25f4c550872e60d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.370055 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3ae67eb26f84587b274c1882194c3a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.371909 1031 tablet_service.cc:1505] Processing CreateTablet for tablet acc957ecd88a4e528692250cdb9d645e (DEFAULT_TABLE table=default.Table_131ee47bb13948ae8c0a3e59a88dd689 [id=099b978ef0b84e14a58e326880b58602]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.372318 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acc957ecd88a4e528692250cdb9d645e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.373382 1175 tablet_bootstrap.cc:492] T 7a44585a217b48f39f247bae3ca2e255 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:51.373556 1175 ts_tablet_manager.cc:1403] T 7a44585a217b48f39f247bae3ca2e255 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260202 02:52:51.374297 1175 raft_consensus.cc:359] T 7a44585a217b48f39f247bae3ca2e255 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.374500 1175 raft_consensus.cc:385] T 7a44585a217b48f39f247bae3ca2e255 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:51.374590 1175 raft_consensus.cc:740] T 7a44585a217b48f39f247bae3ca2e255 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:51.374847 1175 consensus_queue.cc:260] T 7a44585a217b48f39f247bae3ca2e255 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.376778 1035 tablet_service.cc:1505] Processing CreateTablet for tablet 7fea74eb4231430aaa5edc9edb5c6023 (DEFAULT_TABLE table=default.Table_fdd92a256f0648e2b27bd79fa4435d9f [id=4f6decf7e24b46f79c99c9a55dafd98f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.377218 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fea74eb4231430aaa5edc9edb5c6023. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.381083 886 tablet_service.cc:1505] Processing CreateTablet for tablet 9978c347cb604f05905eaaaeb9567f0f (DEFAULT_TABLE table=default.Table_442347079e5c4da39ef0908bcae95ae5 [id=87fd311cbeeb4e9099604b0e8442af70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.381213 901 tablet_service.cc:1505] Processing CreateTablet for tablet acc957ecd88a4e528692250cdb9d645e (DEFAULT_TABLE table=default.Table_131ee47bb13948ae8c0a3e59a88dd689 [id=099b978ef0b84e14a58e326880b58602]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.381268 898 tablet_service.cc:1505] Processing CreateTablet for tablet 7fea74eb4231430aaa5edc9edb5c6023 (DEFAULT_TABLE table=default.Table_fdd92a256f0648e2b27bd79fa4435d9f [id=4f6decf7e24b46f79c99c9a55dafd98f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.381673 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acc957ecd88a4e528692250cdb9d645e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.381875 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9978c347cb604f05905eaaaeb9567f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.382109 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fea74eb4231430aaa5edc9edb5c6023. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.388034 2858 raft_consensus.cc:493] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:51.388252 2858 raft_consensus.cc:515] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.388438 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 9978c347cb604f05905eaaaeb9567f0f (DEFAULT_TABLE table=default.Table_442347079e5c4da39ef0908bcae95ae5 [id=87fd311cbeeb4e9099604b0e8442af70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.388899 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9978c347cb604f05905eaaaeb9567f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.389029 2858 leader_election.cc:290] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:51.389609 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f1c8f7e1dc9644a6b6dd2bc264cce8ef" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:51.390120 859 leader_election.cc:343] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:51.390645 1175 ts_tablet_manager.cc:1434] T 7a44585a217b48f39f247bae3ca2e255 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.017s user 0.003s sys 0.000s
I20260202 02:52:51.391299 1175 tablet_bootstrap.cc:492] T 2616e1bbce4049dbb584550ccc150288 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:51.393615 1175 tablet_bootstrap.cc:654] T 2616e1bbce4049dbb584550ccc150288 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:51.397645 1027 tablet_service.cc:1505] Processing CreateTablet for tablet c3ae67eb26f84587b274c1882194c3a2 (DEFAULT_TABLE table=default.Table_eaefe83cde2b48418b7a3407edb48f5e [id=8934bcbc29424ecbb25f4c550872e60d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.398105 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3ae67eb26f84587b274c1882194c3a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.398401 1175 tablet_bootstrap.cc:492] T 2616e1bbce4049dbb584550ccc150288 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:51.398557 1175 ts_tablet_manager.cc:1403] T 2616e1bbce4049dbb584550ccc150288 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260202 02:52:51.400681 1175 raft_consensus.cc:359] T 2616e1bbce4049dbb584550ccc150288 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.400902 1175 raft_consensus.cc:385] T 2616e1bbce4049dbb584550ccc150288 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:51.400992 1175 raft_consensus.cc:740] T 2616e1bbce4049dbb584550ccc150288 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:51.401213 1175 consensus_queue.cc:260] T 2616e1bbce4049dbb584550ccc150288 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.402778 1175 ts_tablet_manager.cc:1434] T 2616e1bbce4049dbb584550ccc150288 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260202 02:52:51.403817 1175 tablet_bootstrap.cc:492] T 51ba0bcd498043c383c493f45ad1fe17 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:51.405035 1182 tablet_replica.cc:333] T 2f5aae089fcc492f9ce55f84674a452c P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
W20260202 02:52:51.407253 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 29c996781cb947f3a1aee5dd0a9c4828 because the tablet was not found. No further retry: Not found: Tablet not found: 29c996781cb947f3a1aee5dd0a9c4828
I20260202 02:52:51.405414 1175 tablet_bootstrap.cc:654] T 51ba0bcd498043c383c493f45ad1fe17 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:51.408418 1182 raft_consensus.cc:2243] T 2f5aae089fcc492f9ce55f84674a452c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:51.408758 1182 raft_consensus.cc:2272] T 2f5aae089fcc492f9ce55f84674a452c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:51.409850 1173 tablet_bootstrap.cc:492] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:51.409993 1173 ts_tablet_manager.cc:1403] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.104s user 0.006s sys 0.004s
I20260202 02:52:51.414515 1173 raft_consensus.cc:359] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:51.414759 1173 raft_consensus.cc:385] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:51.414842 1173 raft_consensus.cc:740] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:51.414970 1182 ts_tablet_manager.cc:1916] T 2f5aae089fcc492f9ce55f84674a452c P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:51.416837 1175 tablet_bootstrap.cc:492] T 51ba0bcd498043c383c493f45ad1fe17 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:51.417014 1175 ts_tablet_manager.cc:1403] T 51ba0bcd498043c383c493f45ad1fe17 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20260202 02:52:51.417742 1175 raft_consensus.cc:359] T 51ba0bcd498043c383c493f45ad1fe17 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.417953 1175 raft_consensus.cc:385] T 51ba0bcd498043c383c493f45ad1fe17 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:51.418035 1175 raft_consensus.cc:740] T 51ba0bcd498043c383c493f45ad1fe17 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:51.418277 1175 consensus_queue.cc:260] T 51ba0bcd498043c383c493f45ad1fe17 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.418839 1173 consensus_queue.cc:260] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:51.423583 1173 ts_tablet_manager.cc:1434] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.014s user 0.001s sys 0.001s
I20260202 02:52:51.423938 1173 tablet_bootstrap.cc:492] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:51.432204 1175 ts_tablet_manager.cc:1434] T 51ba0bcd498043c383c493f45ad1fe17 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.015s user 0.002s sys 0.000s
I20260202 02:52:51.433274 1175 tablet_bootstrap.cc:492] T 7ff6d2fa085343a6b11be0898da6233d P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:51.434460 1173 tablet_bootstrap.cc:654] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:51.437255 1175 tablet_bootstrap.cc:654] T 7ff6d2fa085343a6b11be0898da6233d P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:51.440737 1173 tablet_bootstrap.cc:492] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:51.440912 1173 ts_tablet_manager.cc:1403] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.002s
I20260202 02:52:51.441649 1173 raft_consensus.cc:359] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.442902 1182 ts_tablet_manager.cc:1929] T 2f5aae089fcc492f9ce55f84674a452c P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:51.443327 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 23dfa02219d748919e3cf24beb2095be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:50 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:51.443570 2878 raft_consensus.cc:3060] T 1d8143f529b74e6ab779dd695920413c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:51.443754 2884 raft_consensus.cc:493] T 550895b4010f4088b2cc34acff50e03f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:51.444694 753 tablet_service.cc:1505] Processing CreateTablet for tablet 4b502efa0a4c423b8ab2341931ef96aa (DEFAULT_TABLE table=default.Table_ef8313f59c0d44e3bb7ec522181f6c25 [id=db48c5322cf44770876f1d56cda035b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.445092 2794 raft_consensus.cc:493] T 7ea7a5d8a92b4ac0903483627258ae66 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:51.445830 2794 raft_consensus.cc:515] T 7ea7a5d8a92b4ac0903483627258ae66 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.446100 743 tablet_service.cc:1505] Processing CreateTablet for tablet 943835f6f43e41c99480295fbbb65adb (DEFAULT_TABLE table=default.Table_672611622d8d492f97aa14af71206575 [id=3c66c314193a4a6996d890303a273bd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.446496 2794 leader_election.cc:290] T 7ea7a5d8a92b4ac0903483627258ae66 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:51.447089 1173 raft_consensus.cc:385] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:51.447167 1173 raft_consensus.cc:740] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:51.447518 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d0ce2121d65b4a228596e0ca9018883d" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:51.447391 1173 consensus_queue.cc:260] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
W20260202 02:52:51.448315 999 leader_election.cc:343] T 7ea7a5d8a92b4ac0903483627258ae66 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 7ea7a5d8a92b4ac0903483627258ae66
I20260202 02:52:51.448817 740 tablet_service.cc:1552] Processing DeleteTablet for tablet 0aa023e74e74480498b721bbb5d15052 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:50 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:51.450855 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7ea7a5d8a92b4ac0903483627258ae66" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:51.451465 1173 ts_tablet_manager.cc:1434] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.010s user 0.000s sys 0.002s
W20260202 02:52:51.451577 998 leader_election.cc:343] T 7ea7a5d8a92b4ac0903483627258ae66 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:51.452080 998 leader_election.cc:304] T 7ea7a5d8a92b4ac0903483627258ae66 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:51.452520 2794 raft_consensus.cc:2749] T 7ea7a5d8a92b4ac0903483627258ae66 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:51.448987 765 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "939fbcaf75c7439d9208bbbf83a72717" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:51.454254 1173 tablet_bootstrap.cc:492] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:51.449206 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2b352d9a815e4bccafb7e2251500c5bc" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:51.454629 2884 raft_consensus.cc:515] T 550895b4010f4088b2cc34acff50e03f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:51.455777 998 leader_election.cc:343] T 939fbcaf75c7439d9208bbbf83a72717 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:51.456123 998 leader_election.cc:343] T 2b352d9a815e4bccafb7e2251500c5bc P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:51.456195 998 leader_election.cc:304] T 2b352d9a815e4bccafb7e2251500c5bc P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:51.453740 764 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "939fbcaf75c7439d9208bbbf83a72717" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:51.456548 2794 raft_consensus.cc:2749] T 2b352d9a815e4bccafb7e2251500c5bc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:51.457759 2860 catalog_manager.cc:1269] Loaded cluster ID: bdfd876443214aa8b701e616e981c0d4
I20260202 02:52:51.457861 2860 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8: loading cluster ID for follower catalog manager: success
I20260202 02:52:51.461427 773 raft_consensus.cc:2468] T d0ce2121d65b4a228596e0ca9018883d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:51.462667 750 tablet_service.cc:1505] Processing CreateTablet for tablet 3a15c5dfc3b24dbd8ef3ef7f3158edb2 (DEFAULT_TABLE table=default.Table_3638d0d39d6943e28507ce3f8aaf5829 [id=50e5158b6eab496e9e6693dc275b172f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.467315 2878 raft_consensus.cc:515] T 1d8143f529b74e6ab779dd695920413c P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.468111 2878 leader_election.cc:290] T 1d8143f529b74e6ab779dd695920413c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:51.468613 2878 raft_consensus.cc:493] T d0ce2121d65b4a228596e0ca9018883d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:51.468801 2878 raft_consensus.cc:515] T d0ce2121d65b4a228596e0ca9018883d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.470570 2892 raft_consensus.cc:493] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:51.470715 2892 raft_consensus.cc:515] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:51.471230 2892 leader_election.cc:290] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
W20260202 02:52:51.471724 715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 408234us
I20260202 02:52:51.474689 825 heartbeater.cc:461] Registering TS with master...
I20260202 02:52:51.474825 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1d8143f529b74e6ab779dd695920413c" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:51.475239 1064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1d8143f529b74e6ab779dd695920413c" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
W20260202 02:52:51.475842 715 leader_election.cc:343] T 1d8143f529b74e6ab779dd695920413c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:51.476087 1061 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ccd6bc8bd0504bf18d108c73685ef9ff" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
W20260202 02:52:51.476195 715 leader_election.cc:343] T 1d8143f529b74e6ab779dd695920413c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:51.476241 1061 raft_consensus.cc:3060] T ccd6bc8bd0504bf18d108c73685ef9ff P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:51.476723 1056 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "550895b4010f4088b2cc34acff50e03f" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:51.476977 2892 consensus_queue.cc:1048] T a0b5d82e5fb94b60b4aa324b978a5378 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:51.477797 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f3eebefa74584c1791d52377f5ba1290" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:51.475517 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "44c8d5b74c8e4c499f7f68e8185efc17" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:51.478283 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f5aae089fcc492f9ce55f84674a452c" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:51.478696 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b427c7e897c64977b924e7d586420e69" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:51.479506 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f1c8f7e1dc9644a6b6dd2bc264cce8ef" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:51.480067 1061 raft_consensus.cc:2468] T ccd6bc8bd0504bf18d108c73685ef9ff P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
W20260202 02:52:51.486829 998 leader_election.cc:343] T 939fbcaf75c7439d9208bbbf83a72717 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:51.487210 998 leader_election.cc:304] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:51.487526 2794 raft_consensus.cc:2804] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:51.487620 2794 raft_consensus.cc:493] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:51.487680 2794 raft_consensus.cc:3060] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:51.490792 2794 raft_consensus.cc:515] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.491840 2794 leader_election.cc:290] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:51.493265 2884 leader_election.cc:290] T 550895b4010f4088b2cc34acff50e03f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:51.498808 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d0ce2121d65b4a228596e0ca9018883d" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:51.499042 773 raft_consensus.cc:3060] T d0ce2121d65b4a228596e0ca9018883d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:51.500720 715 leader_election.cc:343] T 550895b4010f4088b2cc34acff50e03f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: Tablet not RUNNING: STOPPED
I20260202 02:52:51.501559 715 leader_election.cc:304] T ccd6bc8bd0504bf18d108c73685ef9ff P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:51.501999 2884 raft_consensus.cc:2804] T ccd6bc8bd0504bf18d108c73685ef9ff P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:51.502123 2884 raft_consensus.cc:697] T ccd6bc8bd0504bf18d108c73685ef9ff P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:51.502363 2884 consensus_queue.cc:237] T ccd6bc8bd0504bf18d108c73685ef9ff P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:51.504999 2878 leader_election.cc:290] T d0ce2121d65b4a228596e0ca9018883d P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:51.510242 773 raft_consensus.cc:2468] T d0ce2121d65b4a228596e0ca9018883d P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:51.511277 998 leader_election.cc:304] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:51.511740 2794 raft_consensus.cc:2804] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:51.511910 2794 raft_consensus.cc:697] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:51.512199 2794 consensus_queue.cc:237] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.516171 1061 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d0ce2121d65b4a228596e0ca9018883d" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:51.518872 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_7ab92cab118c400ca54a1361f373133b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:51.519601 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7ab92cab118c400ca54a1361f373133b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:51.524865 519 master_service.cc:954] ReplaceTablet: received request to replace tablet 00a8c927e0344033a774a4db2cb97d5b from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:51.525560 2911 raft_consensus.cc:493] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:51.527130 528 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 44c8d5b74c8e4c499f7f68e8185efc17 (table default.Table_f48ed1c7e02349adad7ce22c0e51ed81 [id=46b5d47377474ff9bac2acdb26b08c65]). Expected version 3 got 0
I20260202 02:52:51.527519 528 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 939fbcaf75c7439d9208bbbf83a72717 (table default.Table_12d68a317c8e4d809dbbe77c095a1f6e [id=7219522ffc3849faab94d754a42a8511]). Expected version 1 got 0
I20260202 02:52:51.531553 1031 tablet_service.cc:1552] Processing DeleteTablet for tablet 2b352d9a815e4bccafb7e2251500c5bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:22 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:51.531957 1031 tablet_service.cc:1552] Processing DeleteTablet for tablet b7880cb051c24bc998003f0effed3cf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:51.532756 1031 tablet_service.cc:1552] Processing DeleteTablet for tablet 7ea7a5d8a92b4ac0903483627258ae66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:51.533793 528 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_37ff6365170a49efb41f9a4d5db68e6a" } modify_external_catalogs: true
I20260202 02:52:51.534070 528 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_37ff6365170a49efb41f9a4d5db68e6a" } modify_external_catalogs: true
I20260202 02:52:51.535501 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet d0ce2121d65b4a228596e0ca9018883d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:51.540975 525 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 14ec41dfdf2648319ba08111c5e7c5e8 consensus state has no leader
I20260202 02:52:51.525715 2911 raft_consensus.cc:515] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:51.547348 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b502efa0a4c423b8ab2341931ef96aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.552078 2878 raft_consensus.cc:493] T 2c31b45dc8004eae993e972e9faa3ff8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:51.552258 2878 raft_consensus.cc:515] T 2c31b45dc8004eae993e972e9faa3ff8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:51.552881 2878 leader_election.cc:290] T 2c31b45dc8004eae993e972e9faa3ff8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:51.553773 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2c31b45dc8004eae993e972e9faa3ff8" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:51.553864 2878 raft_consensus.cc:493] T 1f379a29e2454f689d68f5951837512b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:51.554016 1045 raft_consensus.cc:2468] T 2c31b45dc8004eae993e972e9faa3ff8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:51.554004 2878 raft_consensus.cc:515] T 1f379a29e2454f689d68f5951837512b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.554611 2878 leader_election.cc:290] T 1f379a29e2454f689d68f5951837512b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:51.555459 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f379a29e2454f689d68f5951837512b" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:51.555634 1045 raft_consensus.cc:2468] T 1f379a29e2454f689d68f5951837512b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:51.555569 715 leader_election.cc:304] T 2c31b45dc8004eae993e972e9faa3ff8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:51.556155 715 leader_election.cc:304] T 1f379a29e2454f689d68f5951837512b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:51.556561 2892 raft_consensus.cc:2804] T 2c31b45dc8004eae993e972e9faa3ff8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:51.556635 2892 raft_consensus.cc:493] T 2c31b45dc8004eae993e972e9faa3ff8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:51.556694 2892 raft_consensus.cc:3060] T 2c31b45dc8004eae993e972e9faa3ff8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:51.556711 2911 leader_election.cc:290] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:51.556830 2911 raft_consensus.cc:2804] T 1f379a29e2454f689d68f5951837512b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:51.556882 2911 raft_consensus.cc:493] T 1f379a29e2454f689d68f5951837512b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:51.556932 2911 raft_consensus.cc:3060] T 1f379a29e2454f689d68f5951837512b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:51.559231 2911 raft_consensus.cc:515] T 1f379a29e2454f689d68f5951837512b P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.559839 2911 leader_election.cc:290] T 1f379a29e2454f689d68f5951837512b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:51.560308 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28a900bcaa8b41718a285e595886d805" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:51.560786 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f379a29e2454f689d68f5951837512b" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:51.560945 1062 raft_consensus.cc:3060] T 1f379a29e2454f689d68f5951837512b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:51.559267 2892 raft_consensus.cc:515] T 2c31b45dc8004eae993e972e9faa3ff8 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:51.563347 751 tablet_service.cc:1505] Processing CreateTablet for tablet 4794a8ef6f22462d8212f3610d22acd9 (DEFAULT_TABLE table=default.Table_a4b8204d7f474e539edf79a5e07fd706 [id=441f6f6d1057454d85f520e30439eba0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.563872 1062 raft_consensus.cc:2468] T 1f379a29e2454f689d68f5951837512b P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:51.567224 755 tablet_service.cc:1505] Processing CreateTablet for tablet 64e2fcb966fc4a06afb65506be1711a2 (DEFAULT_TABLE table=default.Table_7beba71f16ad4551b219e1a1e8b99a62 [id=3725d264950f46eabe35b6ee4a7c96c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.570693 715 leader_election.cc:304] T 1f379a29e2454f689d68f5951837512b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:51.571914 747 tablet_service.cc:1552] Processing DeleteTablet for tablet ad7c98afaa3447d1aac8a243c6a98ed6 with delete_type TABLET_DATA_DELETED (replaced by tablet d23d2b52f058471bb0da6e2608016fa5) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:51.572150 740 tablet_service.cc:1552] Processing DeleteTablet for tablet fea0fcb8854c4fc6acda80c8ffb8d341 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:50 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:51.572381 2878 raft_consensus.cc:2804] T 1f379a29e2454f689d68f5951837512b P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:51.572503 2878 raft_consensus.cc:697] T 1f379a29e2454f689d68f5951837512b P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:51.572747 2892 leader_election.cc:290] T 2c31b45dc8004eae993e972e9faa3ff8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:51.574251 743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 943835f6f43e41c99480295fbbb65adb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.574579 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64e2fcb966fc4a06afb65506be1711a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.578754 2740 raft_consensus.cc:493] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:51.578924 2740 raft_consensus.cc:515] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:51.579573 2740 leader_election.cc:290] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:51.579895 740 tablet_service.cc:1505] Processing CreateTablet for tablet 922d887f3bf2419f9306d3a2aafeb280 (DEFAULT_TABLE table=default.Table_b8aab442fd004224a0e08473c4f2b075 [id=0a41afd164554a309620558b448e0206]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.572744 2878 consensus_queue.cc:237] T 1f379a29e2454f689d68f5951837512b P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.580525 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79badde1001b432b95f58b5161ace15c" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:51.581107 998 leader_election.cc:343] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:51.585027 755 tablet_service.cc:1505] Processing CreateTablet for tablet 68f6fb6be6a0414889cc1b635da35ec6 (DEFAULT_TABLE table=default.Table_de3de7c394c64794af3b8bffdc413699 [id=fcde662479a14899a24f1d7284d04bba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.588284 747 tablet_service.cc:1505] Processing CreateTablet for tablet c3ae67eb26f84587b274c1882194c3a2 (DEFAULT_TABLE table=default.Table_eaefe83cde2b48418b7a3407edb48f5e [id=8934bcbc29424ecbb25f4c550872e60d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.590409 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2c31b45dc8004eae993e972e9faa3ff8" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:51.590786 1060 raft_consensus.cc:3060] T 2c31b45dc8004eae993e972e9faa3ff8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:51.594066 1060 raft_consensus.cc:2468] T 2c31b45dc8004eae993e972e9faa3ff8 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:51.594871 715 leader_election.cc:304] T 2c31b45dc8004eae993e972e9faa3ff8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:51.595270 2911 raft_consensus.cc:2804] T 2c31b45dc8004eae993e972e9faa3ff8 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:51.595492 2911 raft_consensus.cc:697] T 2c31b45dc8004eae993e972e9faa3ff8 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:51.604902 745 tablet_service.cc:1505] Processing CreateTablet for tablet acc957ecd88a4e528692250cdb9d645e (DEFAULT_TABLE table=default.Table_131ee47bb13948ae8c0a3e59a88dd689 [id=099b978ef0b84e14a58e326880b58602]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.469350 1175 tablet_bootstrap.cc:492] T 7ff6d2fa085343a6b11be0898da6233d P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:51.608655 523 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_96e21d4e03c941129f0581aa558481c0" } new_table_name: "default.Table_44f181eda9a64d218b9bac7f62831d07" modify_external_catalogs: true
I20260202 02:52:51.608939 1182 log.cc:1199] T 2f5aae089fcc492f9ce55f84674a452c P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/2f5aae089fcc492f9ce55f84674a452c
I20260202 02:52:51.609026 1175 ts_tablet_manager.cc:1403] T 7ff6d2fa085343a6b11be0898da6233d P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.176s user 0.004s sys 0.000s
I20260202 02:52:51.609319 1182 ts_tablet_manager.cc:1950] T 2f5aae089fcc492f9ce55f84674a452c P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:51.609833 1175 raft_consensus.cc:359] T 7ff6d2fa085343a6b11be0898da6233d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.610045 1175 raft_consensus.cc:385] T 7ff6d2fa085343a6b11be0898da6233d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:51.610110 1175 raft_consensus.cc:740] T 7ff6d2fa085343a6b11be0898da6233d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:51.610304 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_a64a2e7c03c74179a133b3bcd1da1d27"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:51.610363 1175 consensus_queue.cc:260] T 7ff6d2fa085343a6b11be0898da6233d P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.610894 1175 ts_tablet_manager.cc:1434] T 7ff6d2fa085343a6b11be0898da6233d P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260202 02:52:51.610926 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a64a2e7c03c74179a133b3bcd1da1d27 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:51.611212 1175 tablet_bootstrap.cc:492] T 528d4269f91a44679acfad16e9fbbe32 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:51.472469 2858 raft_consensus.cc:493] T 51ba0bcd498043c383c493f45ad1fe17 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:51.612462 2858 raft_consensus.cc:515] T 51ba0bcd498043c383c493f45ad1fe17 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.613214 2858 leader_election.cc:290] T 51ba0bcd498043c383c493f45ad1fe17 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:51.479990 857 leader_election.cc:343] T f3eebefa74584c1791d52377f5ba1290 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:51.614367 857 leader_election.cc:304] T f3eebefa74584c1791d52377f5ba1290 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
W20260202 02:52:51.614732 857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 134763us
I20260202 02:52:51.616096 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "44c8d5b74c8e4c499f7f68e8185efc17" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
W20260202 02:52:51.616405 857 leader_election.cc:343] T 2f5aae089fcc492f9ce55f84674a452c P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:51.616505 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 2f5aae089fcc492f9ce55f84674a452c (table default.Table_84d6693620cc468e82d5cb1ec6cc69ea [id=f6625b21b6064828b11acdfdb694cd71]) successfully deleted
I20260202 02:52:51.616535 857 leader_election.cc:304] T 2f5aae089fcc492f9ce55f84674a452c P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
W20260202 02:52:51.616707 857 raft_consensus.cc:2686] T 2f5aae089fcc492f9ce55f84674a452c P 1521a6d262b7451bbef8debe970dd6b5: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260202 02:52:51.617013 857 leader_election.cc:343] T b427c7e897c64977b924e7d586420e69 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:51.617146 913 raft_consensus.cc:2393] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f442ef85029461091de847118c34c0d in current term 1: Already voted for candidate b1f05820b6034228bdad884cc348716d in this term.
I20260202 02:52:51.575995 2918 raft_consensus.cc:493] T 7a44585a217b48f39f247bae3ca2e255 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:51.576185 2913 raft_consensus.cc:493] T 8153510d5d154c7895b1ee794585612e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:51.617504 884 tablet_service.cc:1552] Processing DeleteTablet for tablet f1c8f7e1dc9644a6b6dd2bc264cce8ef with delete_type TABLET_DATA_DELETED (replaced by tablet 21fe651f45bf43609cf972bef5eca149) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:51.618664 918 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79badde1001b432b95f58b5161ace15c" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:51.619264 857 leader_election.cc:304] T b427c7e897c64977b924e7d586420e69 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:51.619724 919 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1d8143f529b74e6ab779dd695920413c" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:51.620244 919 raft_consensus.cc:3060] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:51.620666 857 leader_election.cc:343] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:51.620800 857 leader_election.cc:304] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:51.620658 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "51ba0bcd498043c383c493f45ad1fe17" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:51.623067 919 raft_consensus.cc:2468] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:51.619746 2858 raft_consensus.cc:2749] T f3eebefa74584c1791d52377f5ba1290 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:51.623584 2858 raft_consensus.cc:2749] T b427c7e897c64977b924e7d586420e69 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:51.619688 2918 raft_consensus.cc:515] T 7a44585a217b48f39f247bae3ca2e255 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.623927 2858 raft_consensus.cc:2749] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:51.624461 1175 tablet_bootstrap.cc:654] T 528d4269f91a44679acfad16e9fbbe32 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:51.624955 2918 leader_election.cc:290] T 7a44585a217b48f39f247bae3ca2e255 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:51.619675 2913 raft_consensus.cc:515] T 8153510d5d154c7895b1ee794585612e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.627424 2913 leader_election.cc:290] T 8153510d5d154c7895b1ee794585612e P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:51.627231 859 leader_election.cc:343] T 51ba0bcd498043c383c493f45ad1fe17 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:51.634289 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8153510d5d154c7895b1ee794585612e" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:51.636472 764 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "51ba0bcd498043c383c493f45ad1fe17" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:51.636516 756 tablet_service.cc:1505] Processing CreateTablet for tablet 9978c347cb604f05905eaaaeb9567f0f (DEFAULT_TABLE table=default.Table_442347079e5c4da39ef0908bcae95ae5 [id=87fd311cbeeb4e9099604b0e8442af70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.636528 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a15c5dfc3b24dbd8ef3ef7f3158edb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.638504 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9978c347cb604f05905eaaaeb9567f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.640141 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acc957ecd88a4e528692250cdb9d645e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.640852 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 922d887f3bf2419f9306d3a2aafeb280. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.641712 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3ae67eb26f84587b274c1882194c3a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.646963 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7a44585a217b48f39f247bae3ca2e255" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:51.647435 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4794a8ef6f22462d8212f3610d22acd9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:51.651634 857 leader_election.cc:343] T 51ba0bcd498043c383c493f45ad1fe17 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:51.651810 857 leader_election.cc:304] T 51ba0bcd498043c383c493f45ad1fe17 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:51.652122 2929 raft_consensus.cc:2749] T 51ba0bcd498043c383c493f45ad1fe17 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:51.652611 999 leader_election.cc:343] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: d0ce2121d65b4a228596e0ca9018883d
I20260202 02:52:51.653134 2794 consensus_queue.cc:1048] T 44c8d5b74c8e4c499f7f68e8185efc17 P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:51.655228 764 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7a44585a217b48f39f247bae3ca2e255" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:51.655625 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8153510d5d154c7895b1ee794585612e" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:51.657423 1173 tablet_bootstrap.cc:654] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:51.660607 857 leader_election.cc:343] T 7a44585a217b48f39f247bae3ca2e255 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:51.661327 857 leader_election.cc:343] T 8153510d5d154c7895b1ee794585612e P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:51.663825 1175 tablet_bootstrap.cc:492] T 528d4269f91a44679acfad16e9fbbe32 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:51.664389 1175 ts_tablet_manager.cc:1403] T 528d4269f91a44679acfad16e9fbbe32 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.053s user 0.008s sys 0.002s
I20260202 02:52:51.665747 1175 raft_consensus.cc:359] T 528d4269f91a44679acfad16e9fbbe32 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.667698 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68f6fb6be6a0414889cc1b635da35ec6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:51.668637 520 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ec3511e5c58b4dba9e48e2b73df833a4 consensus state has no leader
I20260202 02:52:51.669212 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_ab37f999ca1f4033ac2a25bdf3f8bbf6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:51.669904 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ab37f999ca1f4033ac2a25bdf3f8bbf6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:51.668046 522 catalog_manager.cc:5654] T 939fbcaf75c7439d9208bbbf83a72717 P b1f05820b6034228bdad884cc348716d reported cstate change: term changed from 0 to 1, leader changed from <none> to b1f05820b6034228bdad884cc348716d (127.0.104.131). New cstate: current_term: 1 leader_uuid: "b1f05820b6034228bdad884cc348716d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: HEALTHY } } }
I20260202 02:52:51.676048 522 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 939fbcaf75c7439d9208bbbf83a72717 (table default.Table_12d68a317c8e4d809dbbe77c095a1f6e [id=7219522ffc3849faab94d754a42a8511]). Expected version 1 got 0
I20260202 02:52:51.676473 745 tablet_service.cc:1505] Processing CreateTablet for tablet 7fea74eb4231430aaa5edc9edb5c6023 (DEFAULT_TABLE table=default.Table_fdd92a256f0648e2b27bd79fa4435d9f [id=4f6decf7e24b46f79c99c9a55dafd98f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:51.676625 1175 raft_consensus.cc:385] T 528d4269f91a44679acfad16e9fbbe32 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:51.676739 1175 raft_consensus.cc:740] T 528d4269f91a44679acfad16e9fbbe32 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:51.676945 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fea74eb4231430aaa5edc9edb5c6023. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:51.677004 1175 consensus_queue.cc:260] T 528d4269f91a44679acfad16e9fbbe32 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.678246 1175 ts_tablet_manager.cc:1434] T 528d4269f91a44679acfad16e9fbbe32 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.013s user 0.002s sys 0.000s
W20260202 02:52:51.670878 859 leader_election.cc:343] T 8153510d5d154c7895b1ee794585612e P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:51.680876 859 leader_election.cc:304] T 8153510d5d154c7895b1ee794585612e P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:51.681185 2929 raft_consensus.cc:2749] T 8153510d5d154c7895b1ee794585612e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:51.681640 859 leader_election.cc:343] T 7a44585a217b48f39f247bae3ca2e255 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:51.681787 859 leader_election.cc:304] T 7a44585a217b48f39f247bae3ca2e255 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:51.682009 2929 raft_consensus.cc:2749] T 7a44585a217b48f39f247bae3ca2e255 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:51.685550 2929 raft_consensus.cc:493] T 2616e1bbce4049dbb584550ccc150288 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:51.685741 2929 raft_consensus.cc:515] T 2616e1bbce4049dbb584550ccc150288 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.686532 2929 leader_election.cc:290] T 2616e1bbce4049dbb584550ccc150288 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:51.692086 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2616e1bbce4049dbb584550ccc150288" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:51.609833 1182 tablet_replica.cc:333] T 79badde1001b432b95f58b5161ace15c P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:51.694960 918 raft_consensus.cc:2468] T 79badde1001b432b95f58b5161ace15c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:51.695796 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2616e1bbce4049dbb584550ccc150288" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:51.697384 1175 tablet_bootstrap.cc:492] T b4a4c75a906f40beb896bf6fff705ae6 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
W20260202 02:52:51.699047 2619 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.614s user 0.006s sys 0.028s
W20260202 02:52:51.699199 2619 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.614s user 0.006s sys 0.028s
I20260202 02:52:51.714820 1882 ts_tablet_manager.cc:1929] T 550895b4010f4088b2cc34acff50e03f P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:51.714973 1882 log.cc:1199] T 550895b4010f4088b2cc34acff50e03f P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/550895b4010f4088b2cc34acff50e03f
I20260202 02:52:51.715274 1882 ts_tablet_manager.cc:1950] T 550895b4010f4088b2cc34acff50e03f P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:51.716773 999 leader_election.cc:304] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:51.717257 2794 raft_consensus.cc:2804] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:51.717340 2794 raft_consensus.cc:493] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:51.717397 2794 raft_consensus.cc:3060] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:51.720082 1182 raft_consensus.cc:2243] T 79badde1001b432b95f58b5161ace15c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:51.720099 2794 raft_consensus.cc:515] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:51.720518 1182 raft_consensus.cc:2272] T 79badde1001b432b95f58b5161ace15c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:51.721287 1182 ts_tablet_manager.cc:1916] T 79badde1001b432b95f58b5161ace15c P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:51.722416 2794 leader_election.cc:290] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:51.720661 859 leader_election.cc:343] T 2616e1bbce4049dbb584550ccc150288 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:51.729193 764 raft_consensus.cc:3060] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:51.729506 1175 tablet_bootstrap.cc:654] T b4a4c75a906f40beb896bf6fff705ae6 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:51.731431 857 leader_election.cc:343] T 2616e1bbce4049dbb584550ccc150288 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:51.734432 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79badde1001b432b95f58b5161ace15c" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
W20260202 02:52:51.736161 998 leader_election.cc:343] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:51.741580 2740 raft_consensus.cc:493] T 3284213e4b0447b49a7718ba896d3fa8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:51.741755 2740 raft_consensus.cc:515] T 3284213e4b0447b49a7718ba896d3fa8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.742463 2740 leader_election.cc:290] T 3284213e4b0447b49a7718ba896d3fa8 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:51.744088 524 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 21fe651f45bf43609cf972bef5eca149
I20260202 02:52:51.596019 2911 consensus_queue.cc:237] T 2c31b45dc8004eae993e972e9faa3ff8 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:51.744658 524 catalog_manager.cc:6724] ReplaceTablet: tablet 21fe651f45bf43609cf972bef5eca149 deleted and replaced by tablet 165320266c8e4a258372f225d0372daa
I20260202 02:52:51.751928 2928 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92a8191cf1e14e4d99f1c378c2d3c85b)
I20260202 02:52:51.752161 2928 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } }
I20260202 02:52:51.753538 1445 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7dab2abef14445d688ce1335b1f9b2f8" candidate_term: 3 candidate_status { last_received { term: 2 index: 858 } } ignore_live_leader: false dest_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" is_pre_election: true
I20260202 02:52:51.754607 1043 tablet_service.cc:1552] Processing DeleteTablet for tablet 21fe651f45bf43609cf972bef5eca149 with delete_type TABLET_DATA_DELETED (replaced by tablet 165320266c8e4a258372f225d0372daa) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:51.755065 538 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7dab2abef14445d688ce1335b1f9b2f8" candidate_term: 3 candidate_status { last_received { term: 2 index: 858 } } ignore_live_leader: false dest_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" is_pre_election: true
I20260202 02:52:51.756187 2257 leader_election.cc:304] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7dab2abef14445d688ce1335b1f9b2f8; no voters: 92a8191cf1e14e4d99f1c378c2d3c85b, a9c3e3612bb94fb7b94c308041acfaeb
I20260202 02:52:51.757886 1173 tablet_bootstrap.cc:492] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:51.758128 1173 ts_tablet_manager.cc:1403] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.304s user 0.001s sys 0.003s
I20260202 02:52:51.761966 857 leader_election.cc:304] T 2616e1bbce4049dbb584550ccc150288 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:51.762359 2929 raft_consensus.cc:2749] T 2616e1bbce4049dbb584550ccc150288 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:51.764799 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 21fe651f45bf43609cf972bef5eca149 with delete_type TABLET_DATA_DELETED (replaced by tablet 165320266c8e4a258372f225d0372daa) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:51.768563 2929 raft_consensus.cc:493] T 25a407001f964da6b51f42375136c79a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:51.768759 2929 raft_consensus.cc:515] T 25a407001f964da6b51f42375136c79a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.769272 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 745) took 3404 ms. Trace:
I20260202 02:52:51.769338 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79badde1001b432b95f58b5161ace15c" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:51.769452 2929 leader_election.cc:290] T 25a407001f964da6b51f42375136c79a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:51.769642 1173 raft_consensus.cc:359] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.772192 2928 leader_election.cc:290] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 92a8191cf1e14e4d99f1c378c2d3c85b (127.0.104.189:46271), a9c3e3612bb94fb7b94c308041acfaeb (127.0.104.188:39077)
I20260202 02:52:51.771202 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "25a407001f964da6b51f42375136c79a" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:51.772413 2928 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260202 02:52:51.772272 1173 raft_consensus.cc:385] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:51.773104 1173 raft_consensus.cc:740] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
W20260202 02:52:51.773437 999 leader_election.cc:343] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Illegal state: Tablet not RUNNING: STOPPED
I20260202 02:52:51.774413 999 leader_election.cc:304] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
W20260202 02:52:51.775075 859 leader_election.cc:343] T 25a407001f964da6b51f42375136c79a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:51.775102 2794 raft_consensus.cc:2749] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260202 02:52:51.775417 999 leader_election.cc:343] T 3284213e4b0447b49a7718ba896d3fa8 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 3284213e4b0447b49a7718ba896d3fa8
W20260202 02:52:51.738785 778 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s user 0.015s sys 0.010s
W20260202 02:52:51.776232 778 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.539s user 0.015s sys 0.010s
I20260202 02:52:51.776475 2881 raft_consensus.cc:493] T 29c996781cb947f3a1aee5dd0a9c4828 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b1f05820b6034228bdad884cc348716d)
I20260202 02:52:51.776600 2881 raft_consensus.cc:515] T 29c996781cb947f3a1aee5dd0a9c4828 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:51.784145 1182 ts_tablet_manager.cc:1929] T 79badde1001b432b95f58b5161ace15c P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:51.784647 1182 log.cc:1199] T 79badde1001b432b95f58b5161ace15c P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/79badde1001b432b95f58b5161ace15c
I20260202 02:52:51.785012 1182 ts_tablet_manager.cc:1950] T 79badde1001b432b95f58b5161ace15c P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:51.786504 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3284213e4b0447b49a7718ba896d3fa8" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:51.787191 998 leader_election.cc:343] T 3284213e4b0447b49a7718ba896d3fa8 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:51.787345 998 leader_election.cc:304] T 3284213e4b0447b49a7718ba896d3fa8 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:51.787837 2794 raft_consensus.cc:2749] T 3284213e4b0447b49a7718ba896d3fa8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:51.789577 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_f479891b2cc74fc68dbdbd2a63e870ee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:51.790263 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f479891b2cc74fc68dbdbd2a63e870ee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:51.791767 521 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_9bc03f5de9284dd18457bca6459ca366" } new_table_name: "default.Table_2c8cb966f2d0413a8936f63472045256" modify_external_catalogs: true
W20260202 02:52:51.792029 2857 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s user 0.006s sys 0.028s
W20260202 02:52:51.792116 2857 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.566s user 0.006s sys 0.028s
I20260202 02:52:51.793143 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 79badde1001b432b95f58b5161ace15c (table default.Table_80912cc76f934f139da7f866546ae97a [id=2f3bf7e0cf4c41ac8a2554360b57214a]) successfully deleted
W20260202 02:52:51.793442 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 23dfa02219d748919e3cf24beb2095be because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 23dfa02219d748919e3cf24beb2095be already in progress: creating tablet
W20260202 02:52:51.793712 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 0da286b7f598457ca4d0254a7073cb74 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0da286b7f598457ca4d0254a7073cb74 already in progress: creating tablet
W20260202 02:52:51.796761 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 0aa023e74e74480498b721bbb5d15052 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0aa023e74e74480498b721bbb5d15052 already in progress: creating tablet
I20260202 02:52:51.796998 1182 tablet_replica.cc:333] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
W20260202 02:52:51.797066 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet fea0fcb8854c4fc6acda80c8ffb8d341 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fea0fcb8854c4fc6acda80c8ffb8d341 already in progress: creating tablet
W20260202 02:52:51.797286 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet ad7c98afaa3447d1aac8a243c6a98ed6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ad7c98afaa3447d1aac8a243c6a98ed6 already in progress: creating tablet
I20260202 02:52:51.797295 1182 raft_consensus.cc:2243] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:51.797479 1182 raft_consensus.cc:2272] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:51.798061 1182 ts_tablet_manager.cc:1916] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:51.799049 2794 raft_consensus.cc:493] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
I20260202 02:52:51.799116 524 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_0cd8f9947fbd430f8992cc1543aed9f7" } new_table_name: "default.Table_d01f7dfd99d0405a896f6fbfd581dfd2" modify_external_catalogs: true
I20260202 02:52:51.799252 2794 raft_consensus.cc:515] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:51.800395 2794 leader_election.cc:290] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:51.800730 1173 consensus_queue.cc:260] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.801618 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 07580ad1d049475a9645acfad0b1d219 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:51.801998 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 2616e1bbce4049dbb584550ccc150288 with delete_type TABLET_DATA_DELETED (replaced by tablet 8dd62de8e8234d6c9ce380b1aadc097d) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:51.802160 2257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.189:46271, user_credentials={real_user=slave}} blocked reactor thread for 46112.9us
W20260202 02:52:51.802290 999 leader_election.cc:343] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: d2d27ceea1bb4122b18f0fcf4d2d1bf4
I20260202 02:52:51.806175 764 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "25a407001f964da6b51f42375136c79a" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:51.808432 857 leader_election.cc:343] T 25a407001f964da6b51f42375136c79a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:51.808698 857 leader_election.cc:304] T 25a407001f964da6b51f42375136c79a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:51.809052 2929 raft_consensus.cc:2749] T 25a407001f964da6b51f42375136c79a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:51.809034 2794 consensus_queue.cc:1048] T 44c8d5b74c8e4c499f7f68e8185efc17 P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260202 02:52:51.809685 2633 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.724s user 0.018s sys 0.032s
W20260202 02:52:51.809801 2633 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.724s user 0.018s sys 0.032s
I20260202 02:52:51.810961 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d2d27ceea1bb4122b18f0fcf4d2d1bf4" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:51.811736 998 leader_election.cc:304] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:51.812203 2794 raft_consensus.cc:2749] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260202 02:52:51.812192 714 leader_election.cc:343] T d0ce2121d65b4a228596e0ca9018883d P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: d0ce2121d65b4a228596e0ca9018883d
I20260202 02:52:51.812369 714 leader_election.cc:304] T d0ce2121d65b4a228596e0ca9018883d P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:51.812594 2633 raft_consensus.cc:2749] T d0ce2121d65b4a228596e0ca9018883d P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:51.812654 714 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 51473.5us
W20260202 02:52:51.813004 714 leader_election.cc:343] T 550895b4010f4088b2cc34acff50e03f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 550895b4010f4088b2cc34acff50e03f
I20260202 02:52:51.814733 519 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 00a8c927e0344033a774a4db2cb97d5b
I20260202 02:52:51.815325 519 catalog_manager.cc:6724] ReplaceTablet: tablet 00a8c927e0344033a774a4db2cb97d5b deleted and replaced by tablet 9e034911b6964f1e8d6bf260f1509e95
I20260202 02:52:51.816546 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 00a8c927e0344033a774a4db2cb97d5b with delete_type TABLET_DATA_DELETED (replaced by tablet 9e034911b6964f1e8d6bf260f1509e95) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:51.820288 1043 tablet_service.cc:1552] Processing DeleteTablet for tablet 00a8c927e0344033a774a4db2cb97d5b with delete_type TABLET_DATA_DELETED (replaced by tablet 9e034911b6964f1e8d6bf260f1509e95) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:51.821517 1175 tablet_bootstrap.cc:492] T b4a4c75a906f40beb896bf6fff705ae6 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:51.821712 1175 ts_tablet_manager.cc:1403] T b4a4c75a906f40beb896bf6fff705ae6 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.124s user 0.002s sys 0.006s
I20260202 02:52:51.822579 1175 raft_consensus.cc:359] T b4a4c75a906f40beb896bf6fff705ae6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.822801 1175 raft_consensus.cc:385] T b4a4c75a906f40beb896bf6fff705ae6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:51.822875 1175 raft_consensus.cc:740] T b4a4c75a906f40beb896bf6fff705ae6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:51.823129 1175 consensus_queue.cc:260] T b4a4c75a906f40beb896bf6fff705ae6 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.769366 1882 rpcz_store.cc:276] 0202 02:52:48.364372 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:48.364435 (+ 63us) service_pool.cc:225] Handling call
0202 02:52:51.360929 (+2996494us) ts_tablet_manager.cc:1151] Deleting tablet 550895b4010f4088b2cc34acff50e03f
0202 02:52:51.360938 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:51.514494 (+153556us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:51.714680 (+200186us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:51.769244 (+ 54564us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2996257}
I20260202 02:52:51.825011 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 550895b4010f4088b2cc34acff50e03f (table default.Table_9bfaf1dc7bd8452ba14d3aa60e4160ff [id=427f58cc70ef47a6afac8de0353975f5]) successfully deleted
I20260202 02:52:51.826026 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 746) took 3458 ms. Trace:
I20260202 02:52:51.826211 1882 rpcz_store.cc:276] 0202 02:52:48.367694 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:48.367758 (+ 64us) service_pool.cc:225] Handling call
0202 02:52:51.825729 (+3457971us) ts_tablet_manager.cc:1151] Deleting tablet 118753e8cb344b47a35a745aa8fed265
0202 02:52:51.825743 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:51.826003 (+ 260us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3457645}
I20260202 02:52:51.828052 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 747) took 3441 ms. Trace:
I20260202 02:52:51.828888 1882 rpcz_store.cc:276] 0202 02:52:48.386136 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:48.386208 (+ 72us) service_pool.cc:225] Handling call
0202 02:52:51.827844 (+3441636us) ts_tablet_manager.cc:1151] Deleting tablet 48863e2d60c84460858daabb9a08dcd2
0202 02:52:51.827861 (+ 17us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:51.828018 (+ 157us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3441374}
W20260202 02:52:51.829636 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 48863e2d60c84460858daabb9a08dcd2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 48863e2d60c84460858daabb9a08dcd2 already in progress: creating tablet
I20260202 02:52:51.830199 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 748) took 3433 ms. Trace:
I20260202 02:52:51.830523 1882 rpcz_store.cc:276] 0202 02:52:48.396838 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:48.397108 (+ 270us) service_pool.cc:225] Handling call
0202 02:52:51.830038 (+3432930us) ts_tablet_manager.cc:1151] Deleting tablet 2d18485f5a4a4b11916c64a487cd28b4
0202 02:52:51.830054 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:51.830175 (+ 121us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3432672}
W20260202 02:52:51.831106 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 2d18485f5a4a4b11916c64a487cd28b4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2d18485f5a4a4b11916c64a487cd28b4 already in progress: creating tablet
I20260202 02:52:51.831609 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 749) took 3414 ms. Trace:
I20260202 02:52:51.831740 1882 rpcz_store.cc:276] 0202 02:52:48.416990 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:48.417197 (+ 207us) service_pool.cc:225] Handling call
0202 02:52:51.831441 (+3414244us) ts_tablet_manager.cc:1151] Deleting tablet f87ee8d6ff9042c08ba2e32724bea720
0202 02:52:51.831454 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:51.831588 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3414018}
W20260202 02:52:51.832350 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet f87ee8d6ff9042c08ba2e32724bea720 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f87ee8d6ff9042c08ba2e32724bea720 already in progress: creating tablet
I20260202 02:52:51.835090 1173 ts_tablet_manager.cc:1434] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.077s user 0.001s sys 0.001s
I20260202 02:52:51.840674 1175 ts_tablet_manager.cc:1434] T b4a4c75a906f40beb896bf6fff705ae6 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.019s user 0.001s sys 0.001s
I20260202 02:52:51.840840 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_20da10c887054d929c1fcc0f81006103"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:51.841117 1175 tablet_bootstrap.cc:492] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:51.842548 527 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 528d4269f91a44679acfad16e9fbbe32 (table default.Table_cdcc140c2b8d4aabae614d58873dc22a [id=c2c454eb645044839fa81b33b6460558]). Expected version 1 got 0
I20260202 02:52:51.842793 527 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 44c8d5b74c8e4c499f7f68e8185efc17 (table default.Table_f48ed1c7e02349adad7ce22c0e51ed81 [id=46b5d47377474ff9bac2acdb26b08c65]). Expected version 3 got 0
I20260202 02:52:51.843012 1175 tablet_bootstrap.cc:654] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:51.844386 527 master_service.cc:954] ReplaceTablet: received request to replace tablet 104b2c537c4f4e3da6711364cb5d2a2e from {username='slave'} at 127.0.0.1:47430
W20260202 02:52:51.848745 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_20da10c887054d929c1fcc0f81006103 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:51.851707 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 7ff6d2fa085343a6b11be0898da6233d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:51.862627 528 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 0291c3e873bb4266ab040356fe35f4e9
I20260202 02:52:51.864466 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 0291c3e873bb4266ab040356fe35f4e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:51 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:51.864857 1043 tablet_service.cc:1552] Processing DeleteTablet for tablet 0291c3e873bb4266ab040356fe35f4e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:51 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:51.865283 1173 tablet_bootstrap.cc:492] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:51.865767 1175 tablet_bootstrap.cc:492] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:51.865957 1175 ts_tablet_manager.cc:1403] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.025s user 0.000s sys 0.003s
I20260202 02:52:51.867529 1182 ts_tablet_manager.cc:1929] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:51.869727 1182 log.cc:1199] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/f1c8f7e1dc9644a6b6dd2bc264cce8ef
I20260202 02:52:51.869935 2881 leader_election.cc:290] T 29c996781cb947f3a1aee5dd0a9c4828 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:51.813133 714 leader_election.cc:304] T 550895b4010f4088b2cc34acff50e03f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:51.870527 2881 raft_consensus.cc:2749] T 550895b4010f4088b2cc34acff50e03f P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:51.870584 714 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 57585.9us
I20260202 02:52:51.870600 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "29c996781cb947f3a1aee5dd0a9c4828" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:51.870853 714 leader_election.cc:304] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:51.871196 714 leader_election.cc:304] T 1d8143f529b74e6ab779dd695920413c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:51.871443 2633 raft_consensus.cc:2804] T 1d8143f529b74e6ab779dd695920413c P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:51.871556 2633 raft_consensus.cc:697] T 1d8143f529b74e6ab779dd695920413c P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
W20260202 02:52:51.871625 714 leader_election.cc:343] T 2c31b45dc8004eae993e972e9faa3ff8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 2c31b45dc8004eae993e972e9faa3ff8
I20260202 02:52:51.871806 1182 ts_tablet_manager.cc:1950] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
W20260202 02:52:51.871865 714 leader_election.cc:343] T 1f379a29e2454f689d68f5951837512b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 1f379a29e2454f689d68f5951837512b
W20260202 02:52:51.872080 714 leader_election.cc:343] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 4 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 28a900bcaa8b41718a285e595886d805
I20260202 02:52:51.872164 714 leader_election.cc:304] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
W20260202 02:52:51.872488 714 leader_election.cc:343] T 1f379a29e2454f689d68f5951837512b P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 1f379a29e2454f689d68f5951837512b
W20260202 02:52:51.872710 714 leader_election.cc:343] T 2c31b45dc8004eae993e972e9faa3ff8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 2c31b45dc8004eae993e972e9faa3ff8
I20260202 02:52:51.873726 2857 raft_consensus.cc:2749] T 28a900bcaa8b41718a285e595886d805 P 1f442ef85029461091de847118c34c0d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260202 02:52:51.874171 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_19135ff47c4d4097abe42c140d685d9b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:51.874315 1182 tablet_replica.cc:333] T 07580ad1d049475a9645acfad0b1d219 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:51.874523 1182 raft_consensus.cc:2243] T 07580ad1d049475a9645acfad0b1d219 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:51.874680 1182 raft_consensus.cc:2272] T 07580ad1d049475a9645acfad0b1d219 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:51.874748 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_19135ff47c4d4097abe42c140d685d9b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:51.875221 1182 ts_tablet_manager.cc:1916] T 07580ad1d049475a9645acfad0b1d219 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:51.871819 2633 consensus_queue.cc:237] T 1d8143f529b74e6ab779dd695920413c P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
W20260202 02:52:51.811896 1174 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.597s user 0.020s sys 0.018s
W20260202 02:52:51.876662 1174 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.721s user 0.020s sys 0.018s
I20260202 02:52:51.878158 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet f1c8f7e1dc9644a6b6dd2bc264cce8ef (table default.Table_66adee49a1b045918360d13af6bca9a2 [id=4096475017214ffdba27d20aca4e8d64]) successfully deleted
W20260202 02:52:51.878455 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 21fe651f45bf43609cf972bef5eca149 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 21fe651f45bf43609cf972bef5eca149 already in progress: creating tablet
I20260202 02:52:51.878590 884 tablet_service.cc:1552] Processing DeleteTablet for tablet b4a4c75a906f40beb896bf6fff705ae6 with delete_type TABLET_DATA_DELETED (replaced by tablet af29c7a2ad9d4797b08052606151a7d5) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:51.880721 1173 tablet_bootstrap.cc:654] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:51.880980 2912 log.cc:927] Time spent T 29c996781cb947f3a1aee5dd0a9c4828 P 1f442ef85029461091de847118c34c0d: Append to log took a long time: real 0.135s user 0.001s sys 0.000s
W20260202 02:52:51.876924 2667 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.785s user 0.039s sys 0.063s
I20260202 02:52:51.832772 1882 tablet_replica.cc:333] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:51.885398 1175 raft_consensus.cc:359] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.885708 1175 raft_consensus.cc:385] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:51.885787 1175 raft_consensus.cc:740] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:51.886761 1175 consensus_queue.cc:260] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.889521 1175 ts_tablet_manager.cc:1434] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.023s user 0.001s sys 0.002s
I20260202 02:52:51.889889 1175 tablet_bootstrap.cc:492] T eea6bb04ddf24276b46270d009ed78b4 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:51.899677 1180 ts_tablet_manager.cc:1929] T 3d1a7578e83447669e3c3972a426ad44 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:51.899842 1180 log.cc:1199] T 3d1a7578e83447669e3c3972a426ad44 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/3d1a7578e83447669e3c3972a426ad44
I20260202 02:52:51.900187 1180 ts_tablet_manager.cc:1950] T 3d1a7578e83447669e3c3972a426ad44 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:51.894099 1173 tablet_bootstrap.cc:492] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:51.900605 1173 ts_tablet_manager.cc:1403] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.035s user 0.003s sys 0.004s
I20260202 02:52:51.901010 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 681) took 22128 ms. Trace:
I20260202 02:52:51.901099 1180 rpcz_store.cc:276] 0202 02:52:29.772293 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.772473 (+ 180us) service_pool.cc:225] Handling call
0202 02:52:50.646750 (+20874277us) ts_tablet_manager.cc:1151] Deleting tablet 3d1a7578e83447669e3c3972a426ad44
0202 02:52:50.646770 (+ 20us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:50.780676 (+133906us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:51.898669 (+1117993us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:51.900990 (+ 2321us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":20874005}
I20260202 02:52:51.901993 1173 raft_consensus.cc:359] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.902256 1173 raft_consensus.cc:385] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:51.902347 1173 raft_consensus.cc:740] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:51.902637 1173 consensus_queue.cc:260] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.905920 1882 raft_consensus.cc:2243] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:51.906170 1882 raft_consensus.cc:2272] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:51.906862 1882 ts_tablet_manager.cc:1916] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:51.907703 2667 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.802s user 0.040s sys 0.063s
I20260202 02:52:51.911494 1180 tablet_replica.cc:333] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:51.911717 1180 raft_consensus.cc:2243] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:51.912056 1180 raft_consensus.cc:2272] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:51.912657 1180 ts_tablet_manager.cc:1916] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:51.914143 1174 tablet_bootstrap.cc:492] T 939fbcaf75c7439d9208bbbf83a72717 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:51.914287 1174 ts_tablet_manager.cc:1403] T 939fbcaf75c7439d9208bbbf83a72717 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.937s user 0.020s sys 0.019s
W20260202 02:52:51.916505 523 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f30b698c177c48ee9a4e577552bf7eee consensus state has no leader
I20260202 02:52:51.918092 523 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet ee04563d3b8e4cb2b989bb8d6ee9b20e (table default.Table_62b7624afec14fb6a860d51a1a09913d [id=78feba5e47064271ac41a928580d1f02]). Expected version 2 got 0
I20260202 02:52:51.918955 523 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_62b7624afec14fb6a860d51a1a09913d" } modify_external_catalogs: true
I20260202 02:52:51.919155 523 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_62b7624afec14fb6a860d51a1a09913d" } modify_external_catalogs: true
I20260202 02:52:51.926550 1173 ts_tablet_manager.cc:1434] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.026s user 0.015s sys 0.000s
I20260202 02:52:51.926990 1173 tablet_bootstrap.cc:492] T 1d8143f529b74e6ab779dd695920413c P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:51.931802 1173 tablet_bootstrap.cc:654] T 1d8143f529b74e6ab779dd695920413c P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:51.935045 2881 raft_consensus.cc:2749] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:51.934998 1174 raft_consensus.cc:359] T 939fbcaf75c7439d9208bbbf83a72717 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:51.935168 1174 raft_consensus.cc:385] T 939fbcaf75c7439d9208bbbf83a72717 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:51.935215 1174 raft_consensus.cc:740] T 939fbcaf75c7439d9208bbbf83a72717 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:51.935398 1174 consensus_queue.cc:260] T 939fbcaf75c7439d9208bbbf83a72717 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:51.938670 1174 ts_tablet_manager.cc:1434] T 939fbcaf75c7439d9208bbbf83a72717 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.024s user 0.001s sys 0.001s
I20260202 02:52:51.939028 1174 tablet_bootstrap.cc:492] T 7709ea34429e4e91ad4922a3d73ebd14 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:51.942667 1180 ts_tablet_manager.cc:1929] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:51.943290 1180 log.cc:1199] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/d30557fbe9084cd3860820f69dda8752
I20260202 02:52:51.943742 1180 ts_tablet_manager.cc:1950] T d30557fbe9084cd3860820f69dda8752 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:51.944149 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 682) took 22171 ms. Trace:
I20260202 02:52:51.944293 1180 rpcz_store.cc:276] 0202 02:52:29.772414 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.826716 (+ 54302us) service_pool.cc:225] Handling call
0202 02:52:51.911433 (+22084717us) ts_tablet_manager.cc:1151] Deleting tablet d30557fbe9084cd3860820f69dda8752
0202 02:52:51.911444 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:51.927943 (+ 16499us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:51.942543 (+ 14600us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:51.944127 (+ 1584us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":22084461}
I20260202 02:52:51.944773 1180 tablet_replica.cc:333] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:51.945021 1180 raft_consensus.cc:2243] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:51.945407 1180 pending_rounds.cc:70] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d: Trying to abort 1 pending ops.
I20260202 02:52:51.945708 1180 pending_rounds.cc:77] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923136274223104 op_type: NO_OP noop_request { }
I20260202 02:52:51.945919 1180 raft_consensus.cc:2889] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:51.946065 1180 raft_consensus.cc:2272] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:51.947997 1174 tablet_bootstrap.cc:654] T 7709ea34429e4e91ad4922a3d73ebd14 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:51.949726 1180 ts_tablet_manager.cc:1916] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:51.953155 2313 ts_manager.cc:194] Registered new tserver with Master: 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:51.953243 715 consensus_peers.cc:597] T b7880cb051c24bc998003f0effed3cf8 P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7880cb051c24bc998003f0effed3cf8. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:51.960870 2794 raft_consensus.cc:493] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:51.961040 2794 raft_consensus.cc:515] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:51.961688 2794 leader_election.cc:290] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:51.962353 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8153510d5d154c7895b1ee794585612e" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:51.963583 998 leader_election.cc:343] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:51.978343 2860 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8: acquiring CA information for follower catalog manager: success
I20260202 02:52:51.986310 1180 ts_tablet_manager.cc:1929] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:51.986606 1180 log.cc:1199] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/fe53a8bdbd13417f9ce195c54a52a93c
I20260202 02:52:51.987030 1180 ts_tablet_manager.cc:1950] T fe53a8bdbd13417f9ce195c54a52a93c P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:50.732043 2332 rpcz_store.cc:276] 0202 02:52:30.407536 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.407613 (+ 77us) service_pool.cc:225] Handling call
0202 02:52:48.087009 (+17679396us) raft_consensus.cc:1471] Updating replica for 424 ops
0202 02:52:48.278546 (+191537us) raft_consensus.cc:1534] Early marking committed up to index 434
0202 02:52:48.346787 (+ 68241us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:48.346796 (+ 9us) write_op.cc:454] Released schema lock
0202 02:52:48.347063 (+ 267us) raft_consensus.cc:1539] Triggering prepare for 424 ops
0202 02:52:48.347484 (+ 421us) write_op.cc:270] Start()
0202 02:52:48.347540 (+ 56us) write_op.cc:276] Timestamp: P: 1770000741555262 usec, L: 0
0202 02:52:48.379724 (+ 32184us) spinlock_profiling.cc:243] Waited 27.9 ms on lock 0x60f000020228. stack: 00007f1cb1d16a86 00007f1cb1d16840 00007f1cb0cf13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f1cbacc8f78 00007f1cbdc40dc9 00007f1cbdbe7eb9 00007f1cbdbe56c7 00007f1cc1a5f21d 00007f1cbbbf8b46 00007f1cbbbf8875 00007f1cb8cd955e 00007f1cb8cd843e 00007f1cb8cde675
0202 02:52:48.379985 (+ 261us) write_op.cc:270] Start()
0202 02:52:48.380032 (+ 47us) write_op.cc:276] Timestamp: P: 1770000741555826 usec, L: 0
0202 02:52:48.380290 (+ 258us) write_op.cc:270] Start()
0202 02:52:48.380314 (+ 24us) write_op.cc:276] Timestamp: P: 1770000741556856 usec, L: 0
0202 02:52:48.380621 (+ 307us) write_op.cc:270] Start()
0202 02:52:48.380641 (+ 20us) write_op.cc:276] Timestamp: P: 1770000741557525 usec, L: 0
0202 02:52:48.380828 (+ 187us) write_op.cc:270] Start()
0202 02:52:48.381371 (+ 543us) write_op.cc:276] Timestamp: P: 1770000741558117 usec, L: 0
0202 02:52:48.381983 (+ 612us) write_op.cc:270] Start()
0202 02:52:48.382010 (+ 27us) write_op.cc:276] Timestamp: P: 1770000741558604 usec, L: 0
0202 02:52:48.382220 (+ 210us) write_op.cc:270] Start()
0202 02:52:48.382250 (+ 30us) write_op.cc:276] Timestamp: P: 1770000741559114 usec, L: 0
0202 02:52:48.430214 (+ 47964us) write_op.cc:270] Start()
0202 02:52:48.430250 (+ 36us) write_op.cc:276] Timestamp: P: 1770000741559581 usec, L: 0
0202 02:52:48.430453 (+ 203us) write_op.cc:270] Start()
0202 02:52:48.430475 (+ 22us) write_op.cc:276] Timestamp: P: 1770000741560047 usec, L: 0
0202 02:52:48.430647 (+ 172us) write_op.cc:270] Start()
0202 02:52:48.430667 (+ 20us) write_op.cc:276] Timestamp: P: 1770000741568529 usec, L: 0
0202 02:52:48.430802 (+ 135us) write_op.cc:270] Start()
0202 02:52:48.430823 (+ 21us) write_op.cc:276] Timestamp: P: 1770000741572705 usec, L: 0
0202 02:52:48.430954 (+ 131us) write_op.cc:270] Start()
0202 02:52:48.430973 (+ 19us) write_op.cc:276] Timestamp: P: 1770000741632784 usec, L: 0
0202 02:52:48.431159 (+ 186us) write_op.cc:270] Start()
0202 02:52:48.431182 (+ 23us) write_op.cc:276] Timestamp: P: 1770000741734811 usec, L: 0
0202 02:52:48.431352 (+ 170us) write_op.cc:270] Start()
0202 02:52:48.431375 (+ 23us) write_op.cc:276] Timestamp: P: 1770000741735438 usec, L: 0
0202 02:52:48.525780 (+ 94405us) write_op.cc:270] Start()
0202 02:52:48.525844 (+ 64us) write_op.cc:276] Timestamp: P: 1770000741736082 usec, L: 0
0202 02:52:48.526131 (+ 287us) write_op.cc:270] Start()
0202 02:52:48.526151 (+ 20us) write_op.cc:276] Timestamp: P: 1770000741741658 usec, L: 0
0202 02:52:48.526280 (+ 129us) write_op.cc:270] Start()
0202 02:52:48.526305 (+ 25us) write_op.cc:276] Timestamp: P: 1770000741742296 usec, L: 0
0202 02:52:48.526416 (+ 111us) write_op.cc:270] Start()
0202 02:52:48.526433 (+ 17us) write_op.cc:276] Timestamp: P: 1770000741742789 usec, L: 0
0202 02:52:48.526537 (+ 104us) write_op.cc:270] Start()
0202 02:52:48.526555 (+ 18us) write_op.cc:276] Timestamp: P: 1770000741743296 usec, L: 0
0202 02:52:48.526664 (+ 109us) write_op.cc:270] Start()
0202 02:52:48.526706 (+ 42us) write_op.cc:276] Timestamp: P: 1770000741743776 usec, L: 0
0202 02:52:48.526831 (+ 125us) write_op.cc:270] Start()
0202 02:52:48.526848 (+ 17us) write_op.cc:276] Timestamp: P: 1770000741744283 usec, L: 0
0202 02:52:48.527012 (+ 164us) write_op.cc:270] Start()
0202 02:52:48.527030 (+ 18us) write_op.cc:276] Timestamp: P: 1770000741762136 usec, L: 0
0202 02:52:48.527136 (+ 106us) write_op.cc:270] Start()
0202 02:52:48.527152 (+ 16us) write_op.cc:276] Timestamp: P: 1770000741762685 usec, L: 0
0202 02:52:48.527248 (+ 96us) write_op.cc:270] Start()
0202 02:52:48.527263 (+ 15us) write_op.cc:276] Timestamp: P: 1770000741783002 usec, L: 0
0202 02:52:48.527403 (+ 140us) write_op.cc:270] Start()
0202 02:52:48.527419 (+ 16us) write_op.cc:276] Timestamp: P: 1770000741787103 usec, L: 0
0202 02:52:48.545423 (+ 18004us) write_op.cc:270] Start()
0202 02:52:48.545460 (+ 37us) write_op.cc:276] Timestamp: P: 1770000741791231 usec, L: 0
0202 02:52:48.545720 (+ 260us) write_op.cc:270] Start()
0202 02:52:48.545743 (+ 23us) write_op.cc:276] Timestamp: P: 1770000741792203 usec, L: 0
0202 02:52:48.560006 (+ 14263us) write_op.cc:270] Start()
0202 02:52:48.560036 (+ 30us) write_op.cc:276] Timestamp: P: 1770000741801439 usec, L: 0
0202 02:52:48.560162 (+ 126us) write_op.cc:270] Start()
0202 02:52:48.560178 (+ 16us) write_op.cc:276] Timestamp: P: 1770000741856345 usec, L: 0
0202 02:52:48.560303 (+ 125us) write_op.cc:270] Start()
0202 02:52:48.560347 (+ 44us) write_op.cc:276] Timestamp: P: 1770000741858407 usec, L: 0
0202 02:52:48.560503 (+ 156us) write_op.cc:270] Start()
0202 02:52:48.560521 (+ 18us) write_op.cc:276] Timestamp: P: 1770000741864585 usec, L: 0
0202 02:52:48.560625 (+ 104us) write_op.cc:270] Start()
0202 02:52:48.560640 (+ 15us) write_op.cc:276] Timestamp: P: 1770000741927996 usec, L: 0
0202 02:52:48.560806 (+ 166us) write_op.cc:270] Start()
0202 02:52:48.560824 (+ 18us) write_op.cc:276] Timestamp: P: 1770000741929898 usec, L: 0
0202 02:52:48.560952 (+ 128us) write_op.cc:270] Start()
0202 02:52:48.560967 (+ 15us) write_op.cc:276] Timestamp: P: 1770000741946580 usec, L: 0
0202 02:52:48.561068 (+ 101us) write_op.cc:270] Start()
0202 02:52:48.561083 (+ 15us) write_op.cc:276] Timestamp: P: 1770000741953683 usec, L: 0
0202 02:52:48.561185 (+ 102us) write_op.cc:270] Start()
0202 02:52:48.561199 (+ 14us) write_op.cc:276] Timestamp: P: 1770000741963976 usec, L: 0
0202 02:52:48.561301 (+ 102us) write_op.cc:270] Start()
0202 02:52:48.561329 (+ 28us) write_op.cc:276] Timestamp: P: 1770000741985726 usec, L: 0
0202 02:52:48.561478 (+ 149us) write_op.cc:270] Start()
0202 02:52:48.561501 (+ 23us) write_op.cc:276] Timestamp: P: 1770000742787797 usec, L: 0
0202 02:52:48.561646 (+ 145us) write_op.cc:270] Start()
0202 02:52:48.561663 (+ 17us) write_op.cc:276] Timestamp: P: 1770000742801448 usec, L: 0
0202 02:52:48.561803 (+ 140us) write_op.cc:270] Start()
0202 02:52:48.561821 (+ 18us) write_op.cc:276] Timestamp: P: 1770000742803432 usec, L: 0
0202 02:52:48.561946 (+ 125us) write_op.cc:270] Start()
0202 02:52:48.561966 (+ 20us) write_op.cc:276] Timestamp: P: 1770000742804241 usec, L: 0
0202 02:52:48.562087 (+ 121us) write_op.cc:270] Start()
0202 02:52:48.562104 (+ 17us) write_op.cc:276] Timestamp: P: 1770000742804811 usec, L: 0
0202 02:52:48.562206 (+ 102us) write_op.cc:270] Start()
0202 02:52:48.562221 (+ 15us) write_op.cc:276] Timestamp: P: 1770000742805301 usec, L: 0
0202 02:52:48.562347 (+ 126us) write_op.cc:270] Start()
0202 02:52:48.562366 (+ 19us) write_op.cc:276] Timestamp: P: 1770000742853845 usec, L: 0
0202 02:52:48.562510 (+ 144us) write_op.cc:270] Start()
0202 02:52:48.562529 (+ 19us) write_op.cc:276] Timestamp: P: 1770000742869457 usec, L: 0
0202 02:52:48.562678 (+ 149us) write_op.cc:270] Start()
0202 02:52:48.562698 (+ 20us) write_op.cc:276] Timestamp: P: 1770000742896626 usec, L: 0
0202 02:52:48.562917 (+ 219us) write_op.cc:270] Start()
0202 02:52:48.562937 (+ 20us) write_op.cc:276] Timestamp: P: 1770000742909643 usec, L: 0
0202 02:52:48.563044 (+ 107us) write_op.cc:270] Start()
0202 02:52:48.563061 (+ 17us) write_op.cc:276] Timestamp: P: 1770000742926979 usec, L: 0
0202 02:52:48.563189 (+ 128us) write_op.cc:270] Start()
0202 02:52:48.563205 (+ 16us) write_op.cc:276] Timestamp: P: 1770000742940728 usec, L: 0
0202 02:52:48.563333 (+ 128us) write_op.cc:270] Start()
0202 02:52:48.563349 (+ 16us) write_op.cc:276] Timestamp: P: 1770000742954426 usec, L: 0
0202 02:52:48.618662 (+ 55313us) write_op.cc:270] Start()
0202 02:52:48.618702 (+ 40us) write_op.cc:276] Timestamp: P: 1770000742959243 usec, L: 0
0202 02:52:48.618933 (+ 231us) write_op.cc:270] Start()
0202 02:52:48.618959 (+ 26us) write_op.cc:276] Timestamp: P: 1770000742969869 usec, L: 0
0202 02:52:48.619093 (+ 134us) write_op.cc:270] Start()
0202 02:52:48.619113 (+ 20us) write_op.cc:276] Timestamp: P: 1770000742970481 usec, L: 0
0202 02:52:48.619254 (+ 141us) write_op.cc:270] Start()
0202 02:52:48.619273 (+ 19us) write_op.cc:276] Timestamp: P: 1770000742971001 usec, L: 0
0202 02:52:48.619406 (+ 133us) write_op.cc:270] Start()
0202 02:52:48.619428 (+ 22us) write_op.cc:276] Timestamp: P: 1770000742971469 usec, L: 0
0202 02:52:48.619562 (+ 134us) write_op.cc:270] Start()
0202 02:52:48.619581 (+ 19us) write_op.cc:276] Timestamp: P: 1770000742971977 usec, L: 0
0202 02:52:48.619691 (+ 110us) write_op.cc:270] Start()
0202 02:52:48.619706 (+ 15us) write_op.cc:276] Timestamp: P: 1770000742998252 usec, L: 0
0202 02:52:48.619823 (+ 117us) write_op.cc:270] Start()
0202 02:52:48.619838 (+ 15us) write_op.cc:276] Timestamp: P: 1770000743009000 usec, L: 0
0202 02:52:48.619941 (+ 103us) write_op.cc:270] Start()
0202 02:52:48.619957 (+ 16us) write_op.cc:276] Timestamp: P: 1770000743091940 usec, L: 0
0202 02:52:48.620065 (+ 108us) write_op.cc:270] Start()
0202 02:52:48.620079 (+ 14us) write_op.cc:276] Timestamp: P: 1770000743181089 usec, L: 0
0202 02:52:48.620197 (+ 118us) write_op.cc:270] Start()
0202 02:52:48.620214 (+ 17us) write_op.cc:276] Timestamp: P: 1770000743247654 usec, L: 0
0202 02:52:48.620373 (+ 159us) write_op.cc:270] Start()
0202 02:52:48.620393 (+ 20us) write_op.cc:276] Timestamp: P: 1770000743266779 usec, L: 0
0202 02:52:48.620546 (+ 153us) write_op.cc:270] Start()
0202 02:52:48.620563 (+ 17us) write_op.cc:276] Timestamp: P: 1770000743270778 usec, L: 0
0202 02:52:48.620690 (+ 127us) write_op.cc:270] Start()
0202 02:52:48.620706 (+ 16us) write_op.cc:276] Timestamp: P: 1770000743271428 usec, L: 0
0202 02:52:48.620839 (+ 133us) write_op.cc:270] Start()
0202 02:52:48.620856 (+ 17us) write_op.cc:276] Timestamp: P: 1770000743271978 usec, L: 0
0202 02:52:48.620957 (+ 101us) write_op.cc:270] Start()
0202 02:52:48.620972 (+ 15us) write_op.cc:276] Timestamp: P: 1770000743280476 usec, L: 0
0202 02:52:48.621117 (+ 145us) write_op.cc:270] Start()
0202 02:52:48.621134 (+ 17us) write_op.cc:276] Timestamp: P: 1770000743285405 usec, L: 0
0202 02:52:48.621246 (+ 112us) write_op.cc:270] Start()
0202 02:52:48.621261 (+ 15us) write_op.cc:276] Timestamp: P: 1770000743300160 usec, L: 0
0202 02:52:48.621361 (+ 100us) write_op.cc:270] Start()
0202 02:52:48.621377 (+ 16us) write_op.cc:276] Timestamp: P: 1770000743304054 usec, L: 0
0202 02:52:48.621485 (+ 108us) write_op.cc:270] Start()
0202 02:52:48.621501 (+ 16us) write_op.cc:276] Timestamp: P: 1770000743311103 usec, L: 0
0202 02:52:48.621610 (+ 109us) write_op.cc:270] Start()
0202 02:52:48.621626 (+ 16us) write_op.cc:276] Timestamp: P: 1770000743415476 usec, L: 0
0202 02:52:48.621730 (+ 104us) write_op.cc:270] Start()
0202 02:52:48.621753 (+ 23us) write_op.cc:276] Timestamp: P: 1770000743423846 usec, L: 0
0202 02:52:48.621860 (+ 107us) write_op.cc:270] Start()
0202 02:52:48.621876 (+ 16us) write_op.cc:276] Timestamp: P: 1770000743433088 usec, L: 0
0202 02:52:48.621983 (+ 107us) write_op.cc:270] Start()
0202 02:52:48.621999 (+ 16us) write_op.cc:276] Timestamp: P: 1770000743437358 usec, L: 0
0202 02:52:48.622094 (+ 95us) write_op.cc:270] Start()
0202 02:52:48.622110 (+ 16us) write_op.cc:276] Timestamp: P: 1770000743451087 usec, L: 0
0202 02:52:48.622239 (+ 129us) write_op.cc:270] Start()
0202 02:52:48.622257 (+ 18us) write_op.cc:276] Timestamp: P: 1770000743453488 usec, L: 0
0202 02:52:48.622370 (+ 113us) write_op.cc:270] Start()
0202 02:52:48.622385 (+ 15us) write_op.cc:276] Timestamp: P: 1770000743461867 usec, L: 0
0202 02:52:48.622499 (+ 114us) write_op.cc:270] Start()
0202 02:52:48.622515 (+ 16us) write_op.cc:276] Timestamp: P: 1770000743480776 usec, L: 0
0202 02:52:48.622628 (+ 113us) write_op.cc:270] Start()
0202 02:52:48.622663 (+ 35us) write_op.cc:276] Timestamp: P: 1770000743491674 usec, L: 0
0202 02:52:48.622807 (+ 144us) write_op.cc:270] Start()
0202 02:52:48.622824 (+ 17us) write_op.cc:276] Timestamp: P: 1770000743578647 usec, L: 0
0202 02:52:48.622935 (+ 111us) write_op.cc:270] Start()
0202 02:52:48.622951 (+ 16us) write_op.cc:276] Timestamp: P: 1770000743610209 usec, L: 0
0202 02:52:48.623053 (+ 102us) write_op.cc:270] Start()
0202 02:52:48.623067 (+ 14us) write_op.cc:276] Timestamp: P: 1770000743630255 usec, L: 0
0202 02:52:48.623174 (+ 107us) write_op.cc:270] Start()
0202 02:52:48.623188 (+ 14us) write_op.cc:276] Timestamp: P: 1770000743638821 usec, L: 0
0202 02:52:48.623292 (+ 104us) write_op.cc:270] Start()
0202 02:52:48.623308 (+ 16us) write_op.cc:276] Timestamp: P: 1770000743639764 usec, L: 0
0202 02:52:48.623421 (+ 113us) write_op.cc:270] Start()
0202 02:52:48.623446 (+ 25us) write_op.cc:276] Timestamp: P: 1770000743640290 usec, L: 0
0202 02:52:48.623566 (+ 120us) write_op.cc:270] Start()
0202 02:52:48.623582 (+ 16us) write_op.cc:276] Timestamp: P: 1770000743641112 usec, L: 0
0202 02:52:48.623693 (+ 111us) write_op.cc:270] Start()
0202 02:52:48.623709 (+ 16us) write_op.cc:276] Timestamp: P: 1770000743641683 usec, L: 0
0202 02:52:48.623837 (+ 128us) write_op.cc:270] Start()
0202 02:52:48.623855 (+ 18us) write_op.cc:276] Timestamp: P: 1770000743654073 usec, L: 0
0202 02:52:48.623964 (+ 109us) write_op.cc:270] Start()
0202 02:52:48.623980 (+ 16us) write_op.cc:276] Timestamp: P: 1770000743657607 usec, L: 0
0202 02:52:48.624089 (+ 109us) write_op.cc:270] Start()
0202 02:52:48.624105 (+ 16us) write_op.cc:276] Timestamp: P: 1770000743658277 usec, L: 0
0202 02:52:48.769301 (+145196us) write_op.cc:270] Start()
0202 02:52:48.769388 (+ 87us) write_op.cc:276] Timestamp: P: 1770000743658913 usec, L: 0
0202 02:52:48.769630 (+ 242us) write_op.cc:270] Start()
0202 02:52:48.769657 (+ 27us) write_op.cc:276] Timestamp: P: 1770000743660093 usec, L: 0
0202 02:52:48.769828 (+ 171us) write_op.cc:270] Start()
0202 02:52:48.769850 (+ 22us) write_op.cc:276] Timestamp: P: 1770000743662668 usec, L: 0
0202 02:52:48.770010 (+ 160us) write_op.cc:270] Start()
0202 02:52:48.770031 (+ 21us) write_op.cc:276] Timestamp: P: 1770000743823624 usec, L: 0
0202 02:52:48.770159 (+ 128us) write_op.cc:270] Start()
0202 02:52:48.770176 (+ 17us) write_op.cc:276] Timestamp: P: 1770000743824258 usec, L: 0
0202 02:52:48.770276 (+ 100us) write_op.cc:270] Start()
0202 02:52:48.770291 (+ 15us) write_op.cc:276] Timestamp: P: 1770000743832060 usec, L: 0
0202 02:52:48.770403 (+ 112us) write_op.cc:270] Start()
0202 02:52:48.770426 (+ 23us) write_op.cc:276] Timestamp: P: 1770000743842483 usec, L: 0
0202 02:52:48.770525 (+ 99us) write_op.cc:270] Start()
0202 02:52:48.770540 (+ 15us) write_op.cc:276] Timestamp: P: 1770000743867157 usec, L: 0
0202 02:52:48.770649 (+ 109us) write_op.cc:270] Start()
0202 02:52:48.770666 (+ 17us) write_op.cc:276] Timestamp: P: 1770000743868117 usec, L: 0
0202 02:52:48.770782 (+ 116us) write_op.cc:270] Start()
0202 02:52:48.770801 (+ 19us) write_op.cc:276] Timestamp: P: 1770000743868855 usec, L: 0
0202 02:52:48.770971 (+ 170us) write_op.cc:270] Start()
0202 02:52:48.770993 (+ 22us) write_op.cc:276] Timestamp: P: 1770000743884167 usec, L: 0
0202 02:52:48.771124 (+ 131us) write_op.cc:270] Start()
0202 02:52:48.771144 (+ 20us) write_op.cc:276] Timestamp: P: 1770000743889537 usec, L: 0
0202 02:52:48.880832 (+109688us) write_op.cc:270] Start()
0202 02:52:48.880875 (+ 43us) write_op.cc:276] Timestamp: P: 1770000743890210 usec, L: 0
0202 02:52:48.881064 (+ 189us) write_op.cc:270] Start()
0202 02:52:48.881082 (+ 18us) write_op.cc:276] Timestamp: P: 1770000743890764 usec, L: 0
0202 02:52:48.881196 (+ 114us) write_op.cc:270] Start()
0202 02:52:48.881212 (+ 16us) write_op.cc:276] Timestamp: P: 1770000743891412 usec, L: 0
0202 02:52:48.881337 (+ 125us) write_op.cc:270] Start()
0202 02:52:48.881356 (+ 19us) write_op.cc:276] Timestamp: P: 1770000743932335 usec, L: 0
0202 02:52:48.881484 (+ 128us) write_op.cc:270] Start()
0202 02:52:48.881503 (+ 19us) write_op.cc:276] Timestamp: P: 1770000743934256 usec, L: 0
0202 02:52:48.884099 (+ 2596us) write_op.cc:270] Start()
0202 02:52:48.884146 (+ 47us) write_op.cc:276] Timestamp: P: 1770000743945715 usec, L: 0
0202 02:52:48.911421 (+ 27275us) write_op.cc:270] Start()
0202 02:52:48.911469 (+ 48us) write_op.cc:276] Timestamp: P: 1770000743958392 usec, L: 0
0202 02:52:48.911719 (+ 250us) write_op.cc:270] Start()
0202 02:52:48.911745 (+ 26us) write_op.cc:276] Timestamp: P: 1770000743981224 usec, L: 0
0202 02:52:48.911891 (+ 146us) write_op.cc:270] Start()
0202 02:52:48.911912 (+ 21us) write_op.cc:276] Timestamp: P: 1770000744056655 usec, L: 0
0202 02:52:48.912042 (+ 130us) write_op.cc:270] Start()
0202 02:52:48.912061 (+ 19us) write_op.cc:276] Timestamp: P: 1770000744057277 usec, L: 0
0202 02:52:48.912199 (+ 138us) write_op.cc:270] Start()
0202 02:52:48.912218 (+ 19us) write_op.cc:276] Timestamp: P: 1770000744064748 usec, L: 0
0202 02:52:48.912389 (+ 171us) write_op.cc:270] Start()
0202 02:52:48.912411 (+ 22us) write_op.cc:276] Timestamp: P: 1770000744081092 usec, L: 0
0202 02:52:48.912572 (+ 161us) write_op.cc:270] Start()
0202 02:52:48.912593 (+ 21us) write_op.cc:276] Timestamp: P: 1770000744083585 usec, L: 0
0202 02:52:48.912722 (+ 129us) write_op.cc:270] Start()
0202 02:52:48.912740 (+ 18us) write_op.cc:276] Timestamp: P: 1770000744099769 usec, L: 0
0202 02:52:48.912873 (+ 133us) write_op.cc:270] Start()
0202 02:52:48.912893 (+ 20us) write_op.cc:276] Timestamp: P: 1770000744103939 usec, L: 0
0202 02:52:48.913017 (+ 124us) write_op.cc:270] Start()
0202 02:52:48.913034 (+ 17us) write_op.cc:276] Timestamp: P: 1770000744106278 usec, L: 0
0202 02:52:48.913138 (+ 104us) write_op.cc:270] Start()
0202 02:52:48.913154 (+ 16us) write_op.cc:276] Timestamp: P: 1770000744113195 usec, L: 0
0202 02:52:48.913270 (+ 116us) write_op.cc:270] Start()
0202 02:52:48.913287 (+ 17us) write_op.cc:276] Timestamp: P: 1770000744122954 usec, L: 0
0202 02:52:48.913401 (+ 114us) write_op.cc:270] Start()
0202 02:52:48.913421 (+ 20us) write_op.cc:276] Timestamp: P: 1770000744140595 usec, L: 0
0202 02:52:48.913561 (+ 140us) write_op.cc:270] Start()
0202 02:52:48.913579 (+ 18us) write_op.cc:276] Timestamp: P: 1770000744153703 usec, L: 0
0202 02:52:48.913701 (+ 122us) write_op.cc:270] Start()
0202 02:52:48.913719 (+ 18us) write_op.cc:276] Timestamp: P: 1770000744154408 usec, L: 0
0202 02:52:48.913832 (+ 113us) write_op.cc:270] Start()
0202 02:52:48.913849 (+ 17us) write_op.cc:276] Timestamp: P: 1770000744180889 usec, L: 0
0202 02:52:48.913974 (+ 125us) write_op.cc:270] Start()
0202 02:52:48.913991 (+ 17us) write_op.cc:276] Timestamp: P: 1770000744198621 usec, L: 0
0202 02:52:48.914102 (+ 111us) write_op.cc:270] Start()
0202 02:52:48.914118 (+ 16us) write_op.cc:276] Timestamp: P: 1770000744219025 usec, L: 0
0202 02:52:48.914247 (+ 129us) write_op.cc:270] Start()
0202 02:52:48.914265 (+ 18us) write_op.cc:276] Timestamp: P: 1770000744220109 usec, L: 0
0202 02:52:48.914394 (+ 129us) write_op.cc:270] Start()
0202 02:52:48.914411 (+ 17us) write_op.cc:276] Timestamp: P: 1770000744257125 usec, L: 0
0202 02:52:48.914583 (+ 172us) write_op.cc:270] Start()
0202 02:52:48.914603 (+ 20us) write_op.cc:276] Timestamp: P: 1770000744268094 usec, L: 0
0202 02:52:48.914733 (+ 130us) write_op.cc:270] Start()
0202 02:52:48.914751 (+ 18us) write_op.cc:276] Timestamp: P: 1770000744299228 usec, L: 0
0202 02:52:48.955433 (+ 40682us) write_op.cc:270] Start()
0202 02:52:48.955475 (+ 42us) write_op.cc:276] Timestamp: P: 1770000744325526 usec, L: 0
0202 02:52:48.955685 (+ 210us) write_op.cc:270] Start()
0202 02:52:48.955704 (+ 19us) write_op.cc:276] Timestamp: P: 1770000744345944 usec, L: 0
0202 02:52:48.955811 (+ 107us) write_op.cc:270] Start()
0202 02:52:48.955829 (+ 18us) write_op.cc:276] Timestamp: P: 1770000744386125 usec, L: 0
0202 02:52:48.955961 (+ 132us) write_op.cc:270] Start()
0202 02:52:48.955981 (+ 20us) write_op.cc:276] Timestamp: P: 1770000744410309 usec, L: 0
0202 02:52:48.956092 (+ 111us) write_op.cc:270] Start()
0202 02:52:48.956110 (+ 18us) write_op.cc:276] Timestamp: P: 1770000744421538 usec, L: 0
0202 02:52:48.956241 (+ 131us) write_op.cc:270] Start()
0202 02:52:48.956259 (+ 18us) write_op.cc:276] Timestamp: P: 1770000744561489 usec, L: 0
0202 02:52:48.956423 (+ 164us) write_op.cc:270] Start()
0202 02:52:48.956441 (+ 18us) write_op.cc:276] Timestamp: P: 1770000744562193 usec, L: 0
0202 02:52:48.956560 (+ 119us) write_op.cc:270] Start()
0202 02:52:48.956577 (+ 17us) write_op.cc:276] Timestamp: P: 1770000744567266 usec, L: 0
0202 02:52:48.956711 (+ 134us) write_op.cc:270] Start()
0202 02:52:48.956729 (+ 18us) write_op.cc:276] Timestamp: P: 1770000744579984 usec, L: 0
0202 02:52:48.956915 (+ 186us) write_op.cc:270] Start()
0202 02:52:48.956936 (+ 21us) write_op.cc:276] Timestamp: P: 1770000744581491 usec, L: 0
0202 02:52:48.957059 (+ 123us) write_op.cc:270] Start()
0202 02:52:48.957077 (+ 18us) write_op.cc:276] Timestamp: P: 1770000744582145 usec, L: 0
0202 02:52:48.957204 (+ 127us) write_op.cc:270] Start()
0202 02:52:48.957222 (+ 18us) write_op.cc:276] Timestamp: P: 1770000744582723 usec, L: 0
0202 02:52:48.957354 (+ 132us) write_op.cc:270] Start()
0202 02:52:48.957373 (+ 19us) write_op.cc:276] Timestamp: P: 1770000744583698 usec, L: 0
0202 02:52:48.976465 (+ 19092us) write_op.cc:270] Start()
0202 02:52:48.976508 (+ 43us) write_op.cc:276] Timestamp: P: 1770000744606959 usec, L: 0
0202 02:52:48.976736 (+ 228us) write_op.cc:270] Start()
0202 02:52:48.976760 (+ 24us) write_op.cc:276] Timestamp: P: 1770000744607755 usec, L: 0
0202 02:52:48.976894 (+ 134us) write_op.cc:270] Start()
0202 02:52:48.976915 (+ 21us) write_op.cc:276] Timestamp: P: 1770000744608477 usec, L: 0
0202 02:52:48.977050 (+ 135us) write_op.cc:270] Start()
0202 02:52:48.977070 (+ 20us) write_op.cc:276] Timestamp: P: 1770000744693091 usec, L: 0
0202 02:52:48.977226 (+ 156us) write_op.cc:270] Start()
0202 02:52:48.977254 (+ 28us) write_op.cc:276] Timestamp: P: 1770000744693801 usec, L: 0
0202 02:52:48.977403 (+ 149us) write_op.cc:270] Start()
0202 02:52:48.977421 (+ 18us) write_op.cc:276] Timestamp: P: 1770000744708900 usec, L: 0
0202 02:52:48.977553 (+ 132us) write_op.cc:270] Start()
0202 02:52:48.977574 (+ 21us) write_op.cc:276] Timestamp: P: 1770000744746650 usec, L: 0
0202 02:52:48.977706 (+ 132us) write_op.cc:270] Start()
0202 02:52:48.977725 (+ 19us) write_op.cc:276] Timestamp: P: 1770000744747712 usec, L: 0
0202 02:52:48.977890 (+ 165us) write_op.cc:270] Start()
0202 02:52:48.977912 (+ 22us) write_op.cc:276] Timestamp: P: 1770000744748727 usec, L: 0
0202 02:52:48.978034 (+ 122us) write_op.cc:270] Start()
0202 02:52:48.978051 (+ 17us) write_op.cc:276] Timestamp: P: 1770000744749589 usec, L: 0
0202 02:52:48.978160 (+ 109us) write_op.cc:270] Start()
0202 02:52:48.978176 (+ 16us) write_op.cc:276] Timestamp: P: 1770000744766244 usec, L: 0
0202 02:52:48.978286 (+ 110us) write_op.cc:270] Start()
0202 02:52:48.978302 (+ 16us) write_op.cc:276] Timestamp: P: 1770000744795872 usec, L: 0
0202 02:52:48.978413 (+ 111us) write_op.cc:270] Start()
0202 02:52:48.978429 (+ 16us) write_op.cc:276] Timestamp: P: 1770000744829804 usec, L: 0
0202 02:52:48.978562 (+ 133us) write_op.cc:270] Start()
0202 02:52:48.978580 (+ 18us) write_op.cc:276] Timestamp: P: 1770000744846249 usec, L: 0
0202 02:52:48.978694 (+ 114us) write_op.cc:270] Start()
0202 02:52:48.978713 (+ 19us) write_op.cc:276] Timestamp: P: 1770000744859488 usec, L: 0
0202 02:52:48.978845 (+ 132us) write_op.cc:270] Start()
0202 02:52:48.978866 (+ 21us) write_op.cc:276] Timestamp: P: 1770000744881272 usec, L: 0
0202 02:52:49.060850 (+ 81984us) write_op.cc:270] Start()
0202 02:52:49.060893 (+ 43us) write_op.cc:276] Timestamp: P: 1770000744905499 usec, L: 0
0202 02:52:49.061682 (+ 789us) write_op.cc:270] Start()
0202 02:52:49.061715 (+ 33us) write_op.cc:276] Timestamp: P: 1770000744919276 usec, L: 0
0202 02:52:49.061896 (+ 181us) write_op.cc:270] Start()
0202 02:52:49.061940 (+ 44us) write_op.cc:276] Timestamp: P: 1770000744937759 usec, L: 0
0202 02:52:49.062080 (+ 140us) write_op.cc:270] Start()
0202 02:52:49.062100 (+ 20us) write_op.cc:276] Timestamp: P: 1770000744954342 usec, L: 0
0202 02:52:49.062218 (+ 118us) write_op.cc:270] Start()
0202 02:52:49.062239 (+ 21us) write_op.cc:276] Timestamp: P: 1770000744990024 usec, L: 0
0202 02:52:49.062384 (+ 145us) write_op.cc:270] Start()
0202 02:52:49.062403 (+ 19us) write_op.cc:276] Timestamp: P: 1770000745021745 usec, L: 0
0202 02:52:49.062545 (+ 142us) write_op.cc:270] Start()
0202 02:52:49.062565 (+ 20us) write_op.cc:276] Timestamp: P: 1770000745026901 usec, L: 0
0202 02:52:49.062708 (+ 143us) write_op.cc:270] Start()
0202 02:52:49.062767 (+ 59us) write_op.cc:276] Timestamp: P: 1770000745028260 usec, L: 0
0202 02:52:49.062930 (+ 163us) write_op.cc:270] Start()
0202 02:52:49.062947 (+ 17us) write_op.cc:276] Timestamp: P: 1770000745029369 usec, L: 0
0202 02:52:49.063046 (+ 99us) write_op.cc:270] Start()
0202 02:52:49.063060 (+ 14us) write_op.cc:276] Timestamp: P: 1770000745030390 usec, L: 0
0202 02:52:49.094014 (+ 30954us) write_op.cc:270] Start()
0202 02:52:49.094065 (+ 51us) write_op.cc:276] Timestamp: P: 1770000745084944 usec, L: 0
0202 02:52:49.094633 (+ 568us) write_op.cc:270] Start()
0202 02:52:49.094669 (+ 36us) write_op.cc:276] Timestamp: P: 1770000745088363 usec, L: 0
0202 02:52:49.094868 (+ 199us) write_op.cc:270] Start()
0202 02:52:49.094891 (+ 23us) write_op.cc:276] Timestamp: P: 1770000745094051 usec, L: 0
0202 02:52:49.095056 (+ 165us) write_op.cc:270] Start()
0202 02:52:49.095078 (+ 22us) write_op.cc:276] Timestamp: P: 1770000745095134 usec, L: 0
0202 02:52:49.095208 (+ 130us) write_op.cc:270] Start()
0202 02:52:49.095228 (+ 20us) write_op.cc:276] Timestamp: P: 1770000745096280 usec, L: 0
0202 02:52:49.095369 (+ 141us) write_op.cc:270] Start()
0202 02:52:49.095389 (+ 20us) write_op.cc:276] Timestamp: P: 1770000745097874 usec, L: 0
0202 02:52:49.095524 (+ 135us) write_op.cc:270] Start()
0202 02:52:49.095559 (+ 35us) write_op.cc:276] Timestamp: P: 1770000745099306 usec, L: 0
0202 02:52:49.095708 (+ 149us) write_op.cc:270] Start()
0202 02:52:49.095727 (+ 19us) write_op.cc:276] Timestamp: P: 1770000745120922 usec, L: 0
0202 02:52:49.095895 (+ 168us) write_op.cc:270] Start()
0202 02:52:49.095917 (+ 22us) write_op.cc:276] Timestamp: P: 1770000745121829 usec, L: 0
0202 02:52:49.096054 (+ 137us) write_op.cc:270] Start()
0202 02:52:49.096072 (+ 18us) write_op.cc:276] Timestamp: P: 1770000745172419 usec, L: 0
0202 02:52:49.096197 (+ 125us) write_op.cc:270] Start()
0202 02:52:49.096217 (+ 20us) write_op.cc:276] Timestamp: P: 1770000745235079 usec, L: 0
0202 02:52:49.115980 (+ 19763us) write_op.cc:270] Start()
0202 02:52:49.116027 (+ 47us) write_op.cc:276] Timestamp: P: 1770000745235942 usec, L: 0
0202 02:52:49.120809 (+ 4782us) write_op.cc:270] Start()
0202 02:52:49.120850 (+ 41us) write_op.cc:276] Timestamp: P: 1770000745236541 usec, L: 0
0202 02:52:49.121647 (+ 797us) write_op.cc:270] Start()
0202 02:52:49.121682 (+ 35us) write_op.cc:276] Timestamp: P: 1770000745237116 usec, L: 0
0202 02:52:49.121860 (+ 178us) write_op.cc:270] Start()
0202 02:52:49.121881 (+ 21us) write_op.cc:276] Timestamp: P: 1770000745273623 usec, L: 0
0202 02:52:49.122018 (+ 137us) write_op.cc:270] Start()
0202 02:52:49.122036 (+ 18us) write_op.cc:276] Timestamp: P: 1770000745275501 usec, L: 0
0202 02:52:49.122154 (+ 118us) write_op.cc:270] Start()
0202 02:52:49.122172 (+ 18us) write_op.cc:276] Timestamp: P: 1770000745276643 usec, L: 0
0202 02:52:49.122313 (+ 141us) write_op.cc:270] Start()
0202 02:52:49.122332 (+ 19us) write_op.cc:276] Timestamp: P: 1770000745277608 usec, L: 0
0202 02:52:49.122465 (+ 133us) write_op.cc:270] Start()
0202 02:52:49.122484 (+ 19us) write_op.cc:276] Timestamp: P: 1770000745279317 usec, L: 0
0202 02:52:49.122619 (+ 135us) write_op.cc:270] Start()
0202 02:52:49.122638 (+ 19us) write_op.cc:276] Timestamp: P: 1770000745279985 usec, L: 0
0202 02:52:49.122757 (+ 119us) write_op.cc:270] Start()
0202 02:52:49.122774 (+ 17us) write_op.cc:276] Timestamp: P: 1770000745340715 usec, L: 0
0202 02:52:49.122905 (+ 131us) write_op.cc:270] Start()
0202 02:52:49.122923 (+ 18us) write_op.cc:276] Timestamp: P: 1770000745376657 usec, L: 0
0202 02:52:49.123042 (+ 119us) write_op.cc:270] Start()
0202 02:52:49.123058 (+ 16us) write_op.cc:276] Timestamp: P: 1770000745377783 usec, L: 0
0202 02:52:49.123177 (+ 119us) write_op.cc:270] Start()
0202 02:52:49.123194 (+ 17us) write_op.cc:276] Timestamp: P: 1770000745378370 usec, L: 0
0202 02:52:49.123309 (+ 115us) wr
I20260202 02:52:51.990710 2929 raft_consensus.cc:493] T 528d4269f91a44679acfad16e9fbbe32 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:51.991453 1175 tablet_bootstrap.cc:654] T eea6bb04ddf24276b46270d009ed78b4 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:51.991452 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8153510d5d154c7895b1ee794585612e" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:51.997069 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 683) took 22169 ms. Trace:
I20260202 02:52:51.996917 2638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } } }
I20260202 02:52:51.997263 2638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b [sys.catalog]: This master's current role is: LEADER
I20260202 02:52:51.997383 1180 rpcz_store.cc:276] 0202 02:52:29.827144 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.827186 (+ 42us) service_pool.cc:225] Handling call
0202 02:52:51.944706 (+22117520us) ts_tablet_manager.cc:1151] Deleting tablet fe53a8bdbd13417f9ce195c54a52a93c
0202 02:52:51.944718 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:51.961097 (+ 16379us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:51.986182 (+ 25085us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:51.996422 (+ 10240us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":22117391}
I20260202 02:52:51.997850 1180 tablet_replica.cc:333] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:51.998239 1180 raft_consensus.cc:2243] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:51.998652 1180 pending_rounds.cc:70] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d: Trying to abort 1 pending ops.
I20260202 02:52:51.998740 1180 pending_rounds.cc:77] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923070696534016 op_type: NO_OP noop_request { }
I20260202 02:52:51.998876 1180 raft_consensus.cc:2889] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:51.998981 1180 raft_consensus.cc:2272] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:51.999644 1180 ts_tablet_manager.cc:1916] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:52.007812 2929 raft_consensus.cc:515] T 528d4269f91a44679acfad16e9fbbe32 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.009117 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "528d4269f91a44679acfad16e9fbbe32" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:52.011893 2633 raft_consensus.cc:493] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:52.012081 2633 raft_consensus.cc:515] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:52.012770 2633 leader_election.cc:290] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:52.013410 914 raft_consensus.cc:2468] T 8153510d5d154c7895b1ee794585612e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:52.013937 999 leader_election.cc:304] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:52.014362 2740 raft_consensus.cc:2804] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:52.014431 2740 raft_consensus.cc:493] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:52.014482 2740 raft_consensus.cc:3060] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:52.015468 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "78f122342dd144b48de89902faa0f94f" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:52.015669 1059 raft_consensus.cc:2468] T 78f122342dd144b48de89902faa0f94f P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 2.
I20260202 02:52:52.017062 2740 raft_consensus.cc:515] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.017668 2740 leader_election.cc:290] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:52.017689 715 leader_election.cc:304] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:52.017987 2633 raft_consensus.cc:2804] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260202 02:52:52.018062 2633 raft_consensus.cc:493] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:52.018124 2633 raft_consensus.cc:3060] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Advancing to term 3
I20260202 02:52:52.018427 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8153510d5d154c7895b1ee794585612e" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
W20260202 02:52:52.018900 998 leader_election.cc:343] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:52.021428 859 leader_election.cc:343] T 528d4269f91a44679acfad16e9fbbe32 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:52.022277 2929 leader_election.cc:290] T 528d4269f91a44679acfad16e9fbbe32 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:52.024050 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8153510d5d154c7895b1ee794585612e" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:52.024318 914 raft_consensus.cc:3060] T 8153510d5d154c7895b1ee794585612e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:52.020680 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "528d4269f91a44679acfad16e9fbbe32" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:52.025928 857 leader_election.cc:343] T 528d4269f91a44679acfad16e9fbbe32 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:52.026079 857 leader_election.cc:304] T 528d4269f91a44679acfad16e9fbbe32 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:52.027948 1174 tablet_bootstrap.cc:492] T 7709ea34429e4e91ad4922a3d73ebd14 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:52.028151 1174 ts_tablet_manager.cc:1403] T 7709ea34429e4e91ad4922a3d73ebd14 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.089s user 0.003s sys 0.008s
I20260202 02:52:52.029242 2633 raft_consensus.cc:515] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:52.029754 1174 raft_consensus.cc:359] T 7709ea34429e4e91ad4922a3d73ebd14 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.029982 1174 raft_consensus.cc:385] T 7709ea34429e4e91ad4922a3d73ebd14 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:52.030133 1174 raft_consensus.cc:740] T 7709ea34429e4e91ad4922a3d73ebd14 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:52.030412 2633 leader_election.cc:290] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 3 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:52.030480 2929 raft_consensus.cc:2749] T 528d4269f91a44679acfad16e9fbbe32 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:52.030514 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "78f122342dd144b48de89902faa0f94f" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:52.030700 1059 raft_consensus.cc:3060] T 78f122342dd144b48de89902faa0f94f P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Advancing to term 3
I20260202 02:52:52.031152 914 raft_consensus.cc:2468] T 8153510d5d154c7895b1ee794585612e P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:52.031605 999 leader_election.cc:304] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:52.031998 2929 raft_consensus.cc:493] T b4a4c75a906f40beb896bf6fff705ae6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:52.032132 2929 raft_consensus.cc:515] T b4a4c75a906f40beb896bf6fff705ae6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.032797 2929 leader_election.cc:290] T b4a4c75a906f40beb896bf6fff705ae6 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:52.033131 2794 raft_consensus.cc:2804] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:52.033301 1059 raft_consensus.cc:2468] T 78f122342dd144b48de89902faa0f94f P b1f05820b6034228bdad884cc348716d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 3.
I20260202 02:52:52.033789 715 leader_election.cc:304] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:52.033761 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b4a4c75a906f40beb896bf6fff705ae6" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:52.034310 859 leader_election.cc:343] T b4a4c75a906f40beb896bf6fff705ae6 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:52.034925 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b4a4c75a906f40beb896bf6fff705ae6" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:52.035432 857 leader_election.cc:343] T b4a4c75a906f40beb896bf6fff705ae6 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:52.035564 857 leader_election.cc:304] T b4a4c75a906f40beb896bf6fff705ae6 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:52.035651 1182 ts_tablet_manager.cc:1929] T 07580ad1d049475a9645acfad0b1d219 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:52.035789 1182 log.cc:1199] T 07580ad1d049475a9645acfad0b1d219 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/07580ad1d049475a9645acfad0b1d219
I20260202 02:52:52.030417 1174 consensus_queue.cc:260] T 7709ea34429e4e91ad4922a3d73ebd14 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.036222 1182 ts_tablet_manager.cc:1950] T 07580ad1d049475a9645acfad0b1d219 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:52.036499 2929 raft_consensus.cc:2749] T b4a4c75a906f40beb896bf6fff705ae6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:52.036690 1182 tablet_replica.cc:333] T 2616e1bbce4049dbb584550ccc150288 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:52.036904 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 07580ad1d049475a9645acfad0b1d219 (table default.Table_444485558ccc4c6ba771ab686bc9e1c1 [id=c4207a758ebb4784a67c662cfa4333d2]) successfully deleted
I20260202 02:52:52.037293 1182 raft_consensus.cc:2243] T 2616e1bbce4049dbb584550ccc150288 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:52.037492 1182 raft_consensus.cc:2272] T 2616e1bbce4049dbb584550ccc150288 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:52.038071 1182 ts_tablet_manager.cc:1916] T 2616e1bbce4049dbb584550ccc150288 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:52.048489 2881 raft_consensus.cc:2804] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [term 3 FOLLOWER]: Leader election won for term 3
I20260202 02:52:52.048676 2881 raft_consensus.cc:697] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [term 3 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:52.048985 2881 consensus_queue.cc:237] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:52.055238 1174 ts_tablet_manager.cc:1434] T 7709ea34429e4e91ad4922a3d73ebd14 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.027s user 0.002s sys 0.000s
I20260202 02:52:52.060567 1174 tablet_bootstrap.cc:492] T c8a273ac8cd344719a48fd81945bed0a P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:52.063195 1174 tablet_bootstrap.cc:654] T c8a273ac8cd344719a48fd81945bed0a P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:52.070024 1043 tablet_service.cc:1552] Processing DeleteTablet for tablet 2f5aae089fcc492f9ce55f84674a452c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.070245 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 79badde1001b432b95f58b5161ace15c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.068126 755 tablet_service.cc:1552] Processing DeleteTablet for tablet 21fe651f45bf43609cf972bef5eca149 with delete_type TABLET_DATA_DELETED (replaced by tablet 165320266c8e4a258372f225d0372daa) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.068486 745 tablet_service.cc:1552] Processing DeleteTablet for tablet 00a8c927e0344033a774a4db2cb97d5b with delete_type TABLET_DATA_DELETED (replaced by tablet 9e034911b6964f1e8d6bf260f1509e95) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.070950 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 3d1a7578e83447669e3c3972a426ad44 (table default.Table_a5c327c53c1141b79b4274480bbc8f11 [id=97a0437f66864a7291b966c9b72c0eb7]) successfully deleted
I20260202 02:52:52.068753 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 0291c3e873bb4266ab040356fe35f4e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:51 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.071147 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet d30557fbe9084cd3860820f69dda8752 (table default.Table_0bf3a06700844105a02b98edd499e75a [id=966fa513d6e947cdb895edc05d26ff65]) successfully deleted
I20260202 02:52:52.071310 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet fe53a8bdbd13417f9ce195c54a52a93c (table default.Table_935434504bf249f8842274aa67486d8c [id=4f19275f416a425c9b9d100b2a83870a]) successfully deleted
I20260202 02:52:52.071306 919 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "78f122342dd144b48de89902faa0f94f" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:52.071565 919 raft_consensus.cc:2468] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 2.
I20260202 02:52:52.071599 915 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "78f122342dd144b48de89902faa0f94f" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:52.071795 915 raft_consensus.cc:3060] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Advancing to term 3
W20260202 02:52:52.069679 714 leader_election.cc:343] T 29c996781cb947f3a1aee5dd0a9c4828 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 29c996781cb947f3a1aee5dd0a9c4828
I20260202 02:52:52.072894 714 leader_election.cc:304] T 29c996781cb947f3a1aee5dd0a9c4828 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:52.073154 2878 raft_consensus.cc:2749] T 29c996781cb947f3a1aee5dd0a9c4828 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260202 02:52:52.071615 1182 ts_tablet_manager.cc:1929] T 2616e1bbce4049dbb584550ccc150288 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:52.073597 1182 log.cc:1199] T 2616e1bbce4049dbb584550ccc150288 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/2616e1bbce4049dbb584550ccc150288
I20260202 02:52:52.073791 1180 ts_tablet_manager.cc:1929] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:52.073885 1180 log.cc:1199] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/06b80a1c9251420cbc16fc95ae98aca9
I20260202 02:52:52.073930 1182 ts_tablet_manager.cc:1950] T 2616e1bbce4049dbb584550ccc150288 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:52.074203 1180 ts_tablet_manager.cc:1950] T 06b80a1c9251420cbc16fc95ae98aca9 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:52.074904 915 raft_consensus.cc:2468] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 3.
I20260202 02:52:52.074931 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 684) took 22247 ms. Trace:
I20260202 02:52:52.075008 1180 rpcz_store.cc:276] 0202 02:52:29.827445 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.827485 (+ 40us) service_pool.cc:225] Handling call
0202 02:52:51.997783 (+22170298us) ts_tablet_manager.cc:1151] Deleting tablet 06b80a1c9251420cbc16fc95ae98aca9
0202 02:52:51.997796 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:52.056857 (+ 59061us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:52.073669 (+ 16812us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:52.074912 (+ 1243us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":22170137}
I20260202 02:52:52.075367 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 685) took 22247 ms. Trace:
I20260202 02:52:52.075767 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 2616e1bbce4049dbb584550ccc150288 (table default.Table_4fb44d5cbe71439ba073bd75711e77e0 [id=8c2d0f27f5d04a98bfad01483c783ecb]) successfully deleted
I20260202 02:52:52.076151 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 06b80a1c9251420cbc16fc95ae98aca9 (table default.Table_84d6693620cc468e82d5cb1ec6cc69ea [id=f6625b21b6064828b11acdfdb694cd71]) successfully deleted
W20260202 02:52:52.076538 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 00a8c927e0344033a774a4db2cb97d5b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 00a8c927e0344033a774a4db2cb97d5b already in progress: creating tablet
I20260202 02:52:52.079192 2878 consensus_queue.cc:1048] T 474df41e4fe24a818fa45a4ce84d5474 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:52.081068 1175 tablet_bootstrap.cc:492] T eea6bb04ddf24276b46270d009ed78b4 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:52.081243 1175 ts_tablet_manager.cc:1403] T eea6bb04ddf24276b46270d009ed78b4 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.191s user 0.001s sys 0.004s
I20260202 02:52:52.082052 1175 raft_consensus.cc:359] T eea6bb04ddf24276b46270d009ed78b4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.082252 1175 raft_consensus.cc:385] T eea6bb04ddf24276b46270d009ed78b4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:52.082324 1175 raft_consensus.cc:740] T eea6bb04ddf24276b46270d009ed78b4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:52.082561 1175 consensus_queue.cc:260] T eea6bb04ddf24276b46270d009ed78b4 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.083093 1175 ts_tablet_manager.cc:1434] T eea6bb04ddf24276b46270d009ed78b4 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260202 02:52:52.085171 2948 raft_consensus.cc:493] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:52.085394 2948 raft_consensus.cc:515] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.087244 1175 tablet_bootstrap.cc:492] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
W20260202 02:52:52.087218 999 leader_election.cc:343] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: da5435f969074364bc8f8a6ddc5e4f8a
I20260202 02:52:52.089346 1182 tablet_replica.cc:333] T 7ff6d2fa085343a6b11be0898da6233d P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:52.089601 1182 raft_consensus.cc:2243] T 7ff6d2fa085343a6b11be0898da6233d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:52.089784 1182 raft_consensus.cc:2272] T 7ff6d2fa085343a6b11be0898da6233d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:52.090067 2948 leader_election.cc:290] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:52.090394 1182 ts_tablet_manager.cc:1916] T 7ff6d2fa085343a6b11be0898da6233d P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:52.090617 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "da5435f969074364bc8f8a6ddc5e4f8a" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:52.090853 766 raft_consensus.cc:2468] T da5435f969074364bc8f8a6ddc5e4f8a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:52.091284 1175 tablet_bootstrap.cc:654] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:52.091295 998 leader_election.cc:304] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:52.091769 2948 raft_consensus.cc:2804] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:52.091902 2948 raft_consensus.cc:493] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:52.092033 2948 raft_consensus.cc:3060] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:52.093786 2929 raft_consensus.cc:493] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:52.093969 2929 raft_consensus.cc:515] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.094626 2929 leader_election.cc:290] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:52.094744 2948 raft_consensus.cc:515] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.095285 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7709ea34429e4e91ad4922a3d73ebd14" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:52.079205 1180 rpcz_store.cc:276] 0202 02:52:29.827747 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.827785 (+ 38us) service_pool.cc:225] Handling call
0202 02:52:52.075255 (+22247470us) ts_tablet_manager.cc:1151] Deleting tablet 67495e8e525542fb976f675b71512ed2
0202 02:52:52.075265 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:52.075354 (+ 89us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":22247347}
W20260202 02:52:52.096182 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 67495e8e525542fb976f675b71512ed2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 67495e8e525542fb976f675b71512ed2 already in progress: creating tablet
W20260202 02:52:52.096261 859 leader_election.cc:343] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:52.096482 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 686) took 22268 ms. Trace:
I20260202 02:52:52.096575 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "da5435f969074364bc8f8a6ddc5e4f8a" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:52.096563 1180 rpcz_store.cc:276] 0202 02:52:29.828045 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.828086 (+ 41us) service_pool.cc:225] Handling call
0202 02:52:52.096348 (+22268262us) ts_tablet_manager.cc:1151] Deleting tablet 8dd62de8e8234d6c9ce380b1aadc097d
0202 02:52:52.096361 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:52.096462 (+ 101us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":22268131}
I20260202 02:52:52.096740 766 raft_consensus.cc:3060] T da5435f969074364bc8f8a6ddc5e4f8a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:52.096974 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 8dd62de8e8234d6c9ce380b1aadc097d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8dd62de8e8234d6c9ce380b1aadc097d already in progress: creating tablet
I20260202 02:52:52.097131 1175 tablet_bootstrap.cc:492] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:52.097270 1175 ts_tablet_manager.cc:1403] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.002s
I20260202 02:52:52.097705 2948 leader_election.cc:290] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:52.099072 1175 raft_consensus.cc:359] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.099249 1175 raft_consensus.cc:385] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:52.099323 1175 raft_consensus.cc:740] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:52.099318 766 raft_consensus.cc:2468] T da5435f969074364bc8f8a6ddc5e4f8a P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:52.099797 998 leader_election.cc:304] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:52.100080 2948 raft_consensus.cc:2804] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:52.100278 2948 raft_consensus.cc:697] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:52.100416 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 687) took 22272 ms. Trace:
I20260202 02:52:52.100518 1180 rpcz_store.cc:276] 0202 02:52:29.828315 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.855131 (+ 26816us) service_pool.cc:225] Handling call
0202 02:52:52.100247 (+22245116us) ts_tablet_manager.cc:1151] Deleting tablet 1d98c7a46139413b8f0d3697b74b1e52
0202 02:52:52.100261 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:52.100395 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":22244911}
I20260202 02:52:52.100579 2948 consensus_queue.cc:237] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.100988 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 688) took 22243 ms. Trace:
I20260202 02:52:52.101056 1180 rpcz_store.cc:276] 0202 02:52:29.857215 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.857265 (+ 50us) service_pool.cc:225] Handling call
0202 02:52:52.100884 (+22243619us) ts_tablet_manager.cc:1151] Deleting tablet 78a9a483b95c4bb5a5ab69abc899837b
0202 02:52:52.100894 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:52.100973 (+ 79us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":22243422}
I20260202 02:52:52.101389 1180 tablet_replica.cc:333] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:52.101575 1180 raft_consensus.cc:2243] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:52.101877 1180 pending_rounds.cc:70] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d: Trying to abort 1 pending ops.
I20260202 02:52:52.101922 1180 pending_rounds.cc:77] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923147383300096 op_type: NO_OP noop_request { }
I20260202 02:52:52.102001 1180 raft_consensus.cc:2889] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:52.102077 1180 raft_consensus.cc:2272] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:52.102630 1180 ts_tablet_manager.cc:1916] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:52.103735 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7709ea34429e4e91ad4922a3d73ebd14" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:52.103832 2878 raft_consensus.cc:493] T 939fbcaf75c7439d9208bbbf83a72717 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:52.103932 766 raft_consensus.cc:2468] T 7709ea34429e4e91ad4922a3d73ebd14 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
I20260202 02:52:52.103945 2878 raft_consensus.cc:515] T 939fbcaf75c7439d9208bbbf83a72717 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:52.104173 1175 consensus_queue.cc:260] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.104575 2878 leader_election.cc:290] T 939fbcaf75c7439d9208bbbf83a72717 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:52.104684 857 leader_election.cc:304] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
W20260202 02:52:52.104730 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 1d98c7a46139413b8f0d3697b74b1e52 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1d98c7a46139413b8f0d3697b74b1e52 already in progress: creating tablet
W20260202 02:52:52.105005 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 78a9a483b95c4bb5a5ab69abc899837b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 78a9a483b95c4bb5a5ab69abc899837b already in progress: creating tablet
I20260202 02:52:52.105196 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "939fbcaf75c7439d9208bbbf83a72717" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:52.106890 2929 raft_consensus.cc:2804] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:52.107033 2929 raft_consensus.cc:493] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:52.107160 2929 raft_consensus.cc:3060] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:52.107440 999 leader_election.cc:343] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: da5435f969074364bc8f8a6ddc5e4f8a
I20260202 02:52:52.110728 2929 raft_consensus.cc:515] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.111444 2929 leader_election.cc:290] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:52.111876 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7709ea34429e4e91ad4922a3d73ebd14" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:52.112002 1175 ts_tablet_manager.cc:1434] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.015s user 0.002s sys 0.000s
I20260202 02:52:52.112273 915 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "939fbcaf75c7439d9208bbbf83a72717" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:52.112385 1175 tablet_bootstrap.cc:492] T dc90eeb79ded4739883ad98928476680 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:52.112545 915 raft_consensus.cc:2393] T 939fbcaf75c7439d9208bbbf83a72717 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f442ef85029461091de847118c34c0d in current term 1: Already voted for candidate b1f05820b6034228bdad884cc348716d in this term.
W20260202 02:52:52.112506 859 leader_election.cc:343] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:52.112567 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7709ea34429e4e91ad4922a3d73ebd14" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:52.112792 766 raft_consensus.cc:3060] T 7709ea34429e4e91ad4922a3d73ebd14 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:52.111300 1180 ts_tablet_manager.cc:1929] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:52.113119 1180 log.cc:1199] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/26eeef6eebbf4f2cb44f69255fb37594
I20260202 02:52:52.113502 1180 ts_tablet_manager.cc:1950] T 26eeef6eebbf4f2cb44f69255fb37594 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:52.114121 1175 tablet_bootstrap.cc:654] T dc90eeb79ded4739883ad98928476680 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:52.114413 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 689) took 22207 ms. Trace:
I20260202 02:52:52.114545 1180 rpcz_store.cc:276] 0202 02:52:29.907329 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.907389 (+ 60us) service_pool.cc:225] Handling call
0202 02:52:52.101345 (+22193956us) ts_tablet_manager.cc:1151] Deleting tablet 26eeef6eebbf4f2cb44f69255fb37594
0202 02:52:52.101353 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:52.108097 (+ 6744us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:52.111188 (+ 3091us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:52.114390 (+ 3202us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":21865372}
I20260202 02:52:52.115163 1180 tablet_replica.cc:333] T be993c1512b04df4b92cab3195241d84 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:52.115155 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 26eeef6eebbf4f2cb44f69255fb37594 (table default.Table_684386396f2a4be68080349404089bf8 [id=7fbb33c105f149ef98505dcf622930c9]) successfully deleted
I20260202 02:52:52.115429 766 raft_consensus.cc:2468] T 7709ea34429e4e91ad4922a3d73ebd14 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
I20260202 02:52:52.115815 714 leader_election.cc:304] T 939fbcaf75c7439d9208bbbf83a72717 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:52.116218 2878 raft_consensus.cc:3060] T 939fbcaf75c7439d9208bbbf83a72717 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:52.117511 2794 raft_consensus.cc:697] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:52.117830 2794 consensus_queue.cc:237] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.118685 2878 raft_consensus.cc:2749] T 939fbcaf75c7439d9208bbbf83a72717 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:52.115442 1180 raft_consensus.cc:2243] T be993c1512b04df4b92cab3195241d84 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:52.119630 1180 raft_consensus.cc:2272] T be993c1512b04df4b92cab3195241d84 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:52.120314 1180 ts_tablet_manager.cc:1916] T be993c1512b04df4b92cab3195241d84 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:52.128875 857 leader_election.cc:304] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:52.129509 2929 raft_consensus.cc:493] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b1f05820b6034228bdad884cc348716d)
I20260202 02:52:52.129683 2929 raft_consensus.cc:515] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:52.130348 2929 leader_election.cc:290] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:52.130511 2929 raft_consensus.cc:2804] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:52.130635 2929 raft_consensus.cc:697] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:52.130679 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f6746ccbf37e477196a9706396d1e3fd" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:52.130892 1046 raft_consensus.cc:2468] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 2.
I20260202 02:52:52.130899 2929 consensus_queue.cc:237] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.131502 859 leader_election.cc:304] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:52.126873 1180 ts_tablet_manager.cc:1929] T be993c1512b04df4b92cab3195241d84 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260202 02:52:52.132146 714 consensus_peers.cc:597] T 474df41e4fe24a818fa45a4ce84d5474 P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 474df41e4fe24a818fa45a4ce84d5474. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:52.133797 1180 log.cc:1199] T be993c1512b04df4b92cab3195241d84 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/be993c1512b04df4b92cab3195241d84
I20260202 02:52:52.134232 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f6746ccbf37e477196a9706396d1e3fd" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:52.134464 766 raft_consensus.cc:2393] T f6746ccbf37e477196a9706396d1e3fd P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1521a6d262b7451bbef8debe970dd6b5 in current term 2: Already voted for candidate 1f442ef85029461091de847118c34c0d in this term.
W20260202 02:52:52.140113 524 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 16b2d80274a54f1193a3f1746c278252 consensus state has no leader
I20260202 02:52:52.140677 527 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 104b2c537c4f4e3da6711364cb5d2a2e
I20260202 02:52:52.142803 2925 raft_consensus.cc:2804] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260202 02:52:52.142966 2925 raft_consensus.cc:493] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b1f05820b6034228bdad884cc348716d)
I20260202 02:52:52.143080 2925 raft_consensus.cc:3060] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:52.145023 527 catalog_manager.cc:6724] ReplaceTablet: tablet 104b2c537c4f4e3da6711364cb5d2a2e deleted and replaced by tablet 9d548ecfbeff4c3ba2981d656b4fa92a
I20260202 02:52:52.145967 2925 raft_consensus.cc:515] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:52.146361 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 104b2c537c4f4e3da6711364cb5d2a2e with delete_type TABLET_DATA_DELETED (replaced by tablet 9d548ecfbeff4c3ba2981d656b4fa92a) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.146613 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 104b2c537c4f4e3da6711364cb5d2a2e with delete_type TABLET_DATA_DELETED (replaced by tablet 9d548ecfbeff4c3ba2981d656b4fa92a) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.148909 1174 tablet_bootstrap.cc:492] T c8a273ac8cd344719a48fd81945bed0a P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:52.149053 1174 ts_tablet_manager.cc:1403] T c8a273ac8cd344719a48fd81945bed0a P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.089s user 0.005s sys 0.006s
I20260202 02:52:52.149679 1174 raft_consensus.cc:359] T c8a273ac8cd344719a48fd81945bed0a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.149830 1174 raft_consensus.cc:385] T c8a273ac8cd344719a48fd81945bed0a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:52.149902 1174 raft_consensus.cc:740] T c8a273ac8cd344719a48fd81945bed0a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:52.150100 1174 consensus_queue.cc:260] T c8a273ac8cd344719a48fd81945bed0a P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.150514 1174 ts_tablet_manager.cc:1434] T c8a273ac8cd344719a48fd81945bed0a P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260202 02:52:52.150805 1174 tablet_bootstrap.cc:492] T 79badde1001b432b95f58b5161ace15c P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:52.152648 1174 tablet_bootstrap.cc:654] T 79badde1001b432b95f58b5161ace15c P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:52.153110 1882 ts_tablet_manager.cc:1929] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260202 02:52:52.153235 1882 log.cc:1199] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/d2d27ceea1bb4122b18f0fcf4d2d1bf4
I20260202 02:52:52.153625 1882 ts_tablet_manager.cc:1950] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:52.153517 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 104b2c537c4f4e3da6711364cb5d2a2e with delete_type TABLET_DATA_DELETED (replaced by tablet 9d548ecfbeff4c3ba2981d656b4fa92a) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.154496 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 751) took 3718 ms. Trace:
I20260202 02:52:52.154600 1882 rpcz_store.cc:276] 0202 02:52:48.436197 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:48.436266 (+ 69us) service_pool.cc:225] Handling call
0202 02:52:51.832707 (+3396441us) ts_tablet_manager.cc:1151] Deleting tablet d2d27ceea1bb4122b18f0fcf4d2d1bf4
0202 02:52:51.832720 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:51.966945 (+134225us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:52.152872 (+185927us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:52.154477 (+ 1605us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3396156}
I20260202 02:52:52.155045 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 752) took 3662 ms. Trace:
I20260202 02:52:52.155092 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet d2d27ceea1bb4122b18f0fcf4d2d1bf4 (table default.Table_10aa0dafec0547069eb92ecf12b3c780 [id=1f9c48c8e2484856a66a0719268f22ae]) successfully deleted
I20260202 02:52:52.155114 1882 rpcz_store.cc:276] 0202 02:52:48.492294 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:48.492371 (+ 77us) service_pool.cc:225] Handling call
0202 02:52:52.154916 (+3662545us) ts_tablet_manager.cc:1151] Deleting tablet e07e87c0182b46a49b8ca64c072f4210
0202 02:52:52.154926 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:52.155031 (+ 105us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3662310}
I20260202 02:52:52.155413 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 753) took 3620 ms. Trace:
W20260202 02:52:52.155530 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet e07e87c0182b46a49b8ca64c072f4210 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e07e87c0182b46a49b8ca64c072f4210 already in progress: creating tablet
I20260202 02:52:52.155475 1882 rpcz_store.cc:276] 0202 02:52:48.534816 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:48.534886 (+ 70us) service_pool.cc:225] Handling call
0202 02:52:52.155313 (+3620427us) ts_tablet_manager.cc:1151] Deleting tablet d594295561ff4b01bf13451533a5743c
0202 02:52:52.155321 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:52.155399 (+ 78us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3620258}
I20260202 02:52:52.155782 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 759) took 3125 ms. Trace:
W20260202 02:52:52.155897 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet d594295561ff4b01bf13451533a5743c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d594295561ff4b01bf13451533a5743c already in progress: creating tablet
I20260202 02:52:52.155845 1882 rpcz_store.cc:276] 0202 02:52:49.030691 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:49.030751 (+ 60us) service_pool.cc:225] Handling call
0202 02:52:52.155688 (+3124937us) ts_tablet_manager.cc:1151] Deleting tablet 7ac33134670747ee860b6d39dd7db547
0202 02:52:52.155696 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:52.155767 (+ 71us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3124751}
I20260202 02:52:52.156147 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 760) took 3125 ms. Trace:
W20260202 02:52:52.156239 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 7ac33134670747ee860b6d39dd7db547 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7ac33134670747ee860b6d39dd7db547 already in progress: creating tablet
I20260202 02:52:52.156877 2925 leader_election.cc:290] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:52.157485 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f6746ccbf37e477196a9706396d1e3fd" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:52.157727 766 raft_consensus.cc:2393] T f6746ccbf37e477196a9706396d1e3fd P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1521a6d262b7451bbef8debe970dd6b5 in current term 2: Already voted for candidate 1f442ef85029461091de847118c34c0d in this term.
I20260202 02:52:52.156817 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f6746ccbf37e477196a9706396d1e3fd" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:52.161144 1059 raft_consensus.cc:2468] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 2.
I20260202 02:52:52.163794 520 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_7beba71f16ad4551b219e1a1e8b99a62" } modify_external_catalogs: true
I20260202 02:52:52.163967 528 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_8ce461ec04534b9cb8878004a0cc3bf5" } new_table_name: "default.Table_6ab16c2d02d84dbdb460ee7a7aeec421" modify_external_catalogs: true
I20260202 02:52:52.164021 520 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_7beba71f16ad4551b219e1a1e8b99a62" } modify_external_catalogs: true
I20260202 02:52:52.164556 859 leader_election.cc:304] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:52.164842 526 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_cba18d80d56e482f84815b720dc2a83f" } modify_external_catalogs: true
I20260202 02:52:52.165067 526 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_cba18d80d56e482f84815b720dc2a83f" } modify_external_catalogs: true
I20260202 02:52:52.165987 522 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_c8fe5bcb63164a1980d102e0374c1d0e" } modify_external_catalogs: true
I20260202 02:52:52.166154 522 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_c8fe5bcb63164a1980d102e0374c1d0e" } modify_external_catalogs: true
I20260202 02:52:52.166592 2925 raft_consensus.cc:2804] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Leader election won for term 2
I20260202 02:52:52.166769 2925 raft_consensus.cc:697] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [term 2 LEADER]: Becoming Leader. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:52.167045 2925 consensus_queue.cc:237] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:52.167354 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_9584e635380e470f877ad287cbad051d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:52.167949 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9584e635380e470f877ad287cbad051d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:52.169270 1175 tablet_bootstrap.cc:492] T dc90eeb79ded4739883ad98928476680 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:52.169433 1175 ts_tablet_manager.cc:1403] T dc90eeb79ded4739883ad98928476680 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.057s user 0.003s sys 0.000s
I20260202 02:52:52.170302 1175 raft_consensus.cc:359] T dc90eeb79ded4739883ad98928476680 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.170512 1175 raft_consensus.cc:385] T dc90eeb79ded4739883ad98928476680 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:52.170593 1175 raft_consensus.cc:740] T dc90eeb79ded4739883ad98928476680 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:52.170817 1175 consensus_queue.cc:260] T dc90eeb79ded4739883ad98928476680 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.175714 1180 ts_tablet_manager.cc:1950] T be993c1512b04df4b92cab3195241d84 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:52.176587 524 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet 44c8d5b74c8e4c499f7f68e8185efc17 (table default.Table_f48ed1c7e02349adad7ce22c0e51ed81 [id=46b5d47377474ff9bac2acdb26b08c65]). Expected version 3 got 0
I20260202 02:52:52.176664 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 690) took 22268 ms. Trace:
I20260202 02:52:52.176756 1180 rpcz_store.cc:276] 0202 02:52:29.907869 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:29.907940 (+ 71us) service_pool.cc:225] Handling call
0202 02:52:52.115107 (+22207167us) ts_tablet_manager.cc:1151] Deleting tablet be993c1512b04df4b92cab3195241d84
0202 02:52:52.115118 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:52.123468 (+ 8350us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:52.126737 (+ 3269us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:52.176641 (+ 49904us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":21877915}
I20260202 02:52:52.177874 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet be993c1512b04df4b92cab3195241d84 (table default.Table_dd517ae94bf54267bc3a03dee4896708 [id=22f1ff6e13a84a72820c887dd3107f49]) successfully deleted
I20260202 02:52:52.180143 1173 tablet_bootstrap.cc:492] T 1d8143f529b74e6ab779dd695920413c P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:52.180311 1173 ts_tablet_manager.cc:1403] T 1d8143f529b74e6ab779dd695920413c P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.253s user 0.016s sys 0.024s
I20260202 02:52:52.180665 1174 tablet_bootstrap.cc:492] T 79badde1001b432b95f58b5161ace15c P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:52.180820 1174 ts_tablet_manager.cc:1403] T 79badde1001b432b95f58b5161ace15c P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.030s user 0.005s sys 0.001s
I20260202 02:52:52.181131 1173 raft_consensus.cc:359] T 1d8143f529b74e6ab779dd695920413c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.181332 1173 raft_consensus.cc:385] T 1d8143f529b74e6ab779dd695920413c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:52.181385 1173 raft_consensus.cc:740] T 1d8143f529b74e6ab779dd695920413c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:52.181560 1174 raft_consensus.cc:359] T 79badde1001b432b95f58b5161ace15c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:52.181730 1174 raft_consensus.cc:385] T 79badde1001b432b95f58b5161ace15c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:52.181802 1174 raft_consensus.cc:740] T 79badde1001b432b95f58b5161ace15c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:52.181860 1173 consensus_queue.cc:260] T 1d8143f529b74e6ab779dd695920413c P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.182035 1174 consensus_queue.cc:260] T 79badde1001b432b95f58b5161ace15c P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
W20260202 02:52:52.182613 521 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f66be6c9ca98432cb0b18b6cbf1d8a0f consensus state has no leader
I20260202 02:52:52.182636 1174 ts_tablet_manager.cc:1434] T 79badde1001b432b95f58b5161ace15c P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:52.182904 1174 tablet_bootstrap.cc:492] T 587cfd0ae62f4cfd937faf6890e78f4a P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:52.182971 521 master_service.cc:954] ReplaceTablet: received request to replace tablet af29c7a2ad9d4797b08052606151a7d5 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:52.183434 751 tablet_service.cc:1552] Processing DeleteTablet for tablet d0ce2121d65b4a228596e0ca9018883d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.183838 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 93f958924885457ab15f2aac5509e0aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:25 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.184278 745 tablet_service.cc:1552] Processing DeleteTablet for tablet 1f379a29e2454f689d68f5951837512b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.184303 751 tablet_service.cc:1552] Processing DeleteTablet for tablet d30557fbe9084cd3860820f69dda8752 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.184628 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 26eeef6eebbf4f2cb44f69255fb37594 with delete_type TABLET_DATA_DELETED (replaced by tablet dc4c3f52f4f44a84b0d0e3c9a4a49f09) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.184814 1174 tablet_bootstrap.cc:654] T 587cfd0ae62f4cfd937faf6890e78f4a P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:52.185087 745 tablet_service.cc:1552] Processing DeleteTablet for tablet 06b80a1c9251420cbc16fc95ae98aca9 with delete_type TABLET_DATA_DELETED (replaced by tablet 2f5aae089fcc492f9ce55f84674a452c) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.185084 751 tablet_service.cc:1552] Processing DeleteTablet for tablet fe53a8bdbd13417f9ce195c54a52a93c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:21 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.185835 524 master_service.cc:954] ReplaceTablet: received request to replace tablet 41a4143247ed4eed98defbdfe2895372 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:52.186039 1180 tablet_replica.cc:333] T 75569511179b41038e521fa3346a92ae P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:52.186257 1180 raft_consensus.cc:2243] T 75569511179b41038e521fa3346a92ae P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:52.186440 1180 raft_consensus.cc:2272] T 75569511179b41038e521fa3346a92ae P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:52.190623 523 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet ee04563d3b8e4cb2b989bb8d6ee9b20e
I20260202 02:52:52.194416 884 tablet_service.cc:1552] Processing DeleteTablet for tablet ee04563d3b8e4cb2b989bb8d6ee9b20e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:51 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.195217 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_0990710bca2c4106a2a43ad8ca140c7c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:52.195899 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0990710bca2c4106a2a43ad8ca140c7c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:52.196192 2944 raft_consensus.cc:493] T 75a24a1bf54d4c799938a5723ac6fa99 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:52.196388 2944 raft_consensus.cc:515] T 75a24a1bf54d4c799938a5723ac6fa99 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.197171 2944 leader_election.cc:290] T 75a24a1bf54d4c799938a5723ac6fa99 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:52.198618 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet ee04563d3b8e4cb2b989bb8d6ee9b20e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:51 UTC) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:52.199456 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet 99e0b5e9a9874aec886155e0d614dec2 (table default.Table_a8c4f67a58b74bf8bb3716174ed5654f [id=e7975854ed2f4810a2d438b590267584]) was not created within the allowed timeout. Replacing with a new tablet d677b534cba94354b6d0379a4efc4432
I20260202 02:52:52.199894 751 tablet_service.cc:1552] Processing DeleteTablet for tablet ee04563d3b8e4cb2b989bb8d6ee9b20e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:51 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.200156 1180 ts_tablet_manager.cc:1916] T 75569511179b41038e521fa3346a92ae P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:52.200403 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet f3eebefa74584c1791d52377f5ba1290 (table default.Table_37b8a9e1666a4ddbaf5a13d828e9702c [id=e0cea4989f66443faf074eacf434658c]) was not created within the allowed timeout. Replacing with a new tablet 1f171e07223b4895873a513ee05c8f21
I20260202 02:52:52.202636 2925 raft_consensus.cc:493] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:52.202904 2925 raft_consensus.cc:515] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.203891 2925 leader_election.cc:290] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:52.205874 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "75a24a1bf54d4c799938a5723ac6fa99" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:52.206466 715 leader_election.cc:343] T 75a24a1bf54d4c799938a5723ac6fa99 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:52.206837 2948 raft_consensus.cc:493] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:52.206976 2948 raft_consensus.cc:515] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.207571 2948 leader_election.cc:290] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:52.208132 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "16b47a2831f54d809ed0b16deaec7e3d" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:52.208417 766 raft_consensus.cc:2468] T 16b47a2831f54d809ed0b16deaec7e3d P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 2.
I20260202 02:52:52.208895 857 leader_election.cc:304] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters:
I20260202 02:52:52.209403 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "16b47a2831f54d809ed0b16deaec7e3d" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:52.209847 766 raft_consensus.cc:2468] T 16b47a2831f54d809ed0b16deaec7e3d P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 2.
I20260202 02:52:52.209599 1174 tablet_bootstrap.cc:492] T 587cfd0ae62f4cfd937faf6890e78f4a P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:52.210279 1174 ts_tablet_manager.cc:1403] T 587cfd0ae62f4cfd937faf6890e78f4a P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.003s
I20260202 02:52:52.210745 998 leader_election.cc:304] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:52.211014 2929 raft_consensus.cc:2804] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260202 02:52:52.211126 2929 raft_consensus.cc:493] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:52.211390 2948 raft_consensus.cc:2804] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260202 02:52:52.211513 1174 raft_consensus.cc:359] T 587cfd0ae62f4cfd937faf6890e78f4a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.211501 2948 raft_consensus.cc:493] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:52.211215 2929 raft_consensus.cc:3060] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Advancing to term 3
I20260202 02:52:52.211694 1174 raft_consensus.cc:385] T 587cfd0ae62f4cfd937faf6890e78f4a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:52.211773 1174 raft_consensus.cc:740] T 587cfd0ae62f4cfd937faf6890e78f4a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:52.211687 2948 raft_consensus.cc:3060] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Advancing to term 3
I20260202 02:52:52.211838 915 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "16b47a2831f54d809ed0b16deaec7e3d" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:52.212018 1174 consensus_queue.cc:260] T 587cfd0ae62f4cfd937faf6890e78f4a P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.212641 1174 ts_tablet_manager.cc:1434] T 587cfd0ae62f4cfd937faf6890e78f4a P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260202 02:52:52.212885 714 leader_election.cc:343] T 75a24a1bf54d4c799938a5723ac6fa99 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 75a24a1bf54d4c799938a5723ac6fa99
I20260202 02:52:52.212960 1174 tablet_bootstrap.cc:492] T 2f5aae089fcc492f9ce55f84674a452c P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:52.213016 714 leader_election.cc:304] T 75a24a1bf54d4c799938a5723ac6fa99 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:52.213379 2944 raft_consensus.cc:2749] T 75a24a1bf54d4c799938a5723ac6fa99 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:52.214466 2948 raft_consensus.cc:515] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.214624 1174 tablet_bootstrap.cc:654] T 2f5aae089fcc492f9ce55f84674a452c P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:52.216670 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "16b47a2831f54d809ed0b16deaec7e3d" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:52.216986 1059 raft_consensus.cc:2393] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1521a6d262b7451bbef8debe970dd6b5 in current term 3: Already voted for candidate b1f05820b6034228bdad884cc348716d in this term.
I20260202 02:52:52.216346 2929 raft_consensus.cc:515] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.217837 915 raft_consensus.cc:2393] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1f05820b6034228bdad884cc348716d in current term 3: Already voted for candidate 1521a6d262b7451bbef8debe970dd6b5 in this term.
I20260202 02:52:52.216527 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "16b47a2831f54d809ed0b16deaec7e3d" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:52.218441 913 raft_consensus.cc:2393] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b1f05820b6034228bdad884cc348716d in current term 3: Already voted for candidate 1521a6d262b7451bbef8debe970dd6b5 in this term.
I20260202 02:52:52.218761 2929 leader_election.cc:290] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 3 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:52.220556 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "16b47a2831f54d809ed0b16deaec7e3d" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:52.220798 778 raft_consensus.cc:3060] T 16b47a2831f54d809ed0b16deaec7e3d P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Advancing to term 3
W20260202 02:52:52.222864 528 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 06016cee61b64c59b2f01e8f01e445cc consensus state has no leader
I20260202 02:52:52.223309 778 raft_consensus.cc:2468] T 16b47a2831f54d809ed0b16deaec7e3d P 1f442ef85029461091de847118c34c0d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 3.
I20260202 02:52:52.223398 2929 raft_consensus.cc:493] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:52.223559 2929 raft_consensus.cc:515] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.223950 522 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet 9e034911b6964f1e8d6bf260f1509e95
I20260202 02:52:52.224205 2929 leader_election.cc:290] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:52.224524 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "16b47a2831f54d809ed0b16deaec7e3d" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:52.224746 1046 raft_consensus.cc:2393] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1521a6d262b7451bbef8debe970dd6b5 in current term 3: Already voted for candidate b1f05820b6034228bdad884cc348716d in this term.
I20260202 02:52:52.225437 998 leader_election.cc:304] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:52.225916 522 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_6eea808cf1ea46998a599819ee2cbd5c" } new_table_name: "default.Table_f36b7c02df0d47c8afd267579fb3607e" modify_external_catalogs: true
I20260202 02:52:52.226150 2944 consensus_queue.cc:1048] T 65bd1c1a382e42e4a68077264563c55b P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:52.227563 2740 raft_consensus.cc:2804] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [term 3 FOLLOWER]: Leader election won for term 3
I20260202 02:52:52.230476 519 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_0dd06fff80bc41c78d6f707fc8c0bee3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:52.231179 519 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0dd06fff80bc41c78d6f707fc8c0bee3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:52.232197 764 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ee04563d3b8e4cb2b989bb8d6ee9b20e" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:52.234026 857 leader_election.cc:343] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:52.200783 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet 8153510d5d154c7895b1ee794585612e (table default.Table_0e7ceba4956f408c8a9fc0f61fa67c90 [id=c0c6499675894350ac008b534d030c93]) was not created within the allowed timeout. Replacing with a new tablet 9f805fa060d54d7e9067b5120f3d28ad
W20260202 02:52:52.235179 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet c8a273ac8cd344719a48fd81945bed0a (table default.Table_5a703dc84eb34674bdc5793ac7295098 [id=62c3c39fb73249349cee49b86d535612]) was not created within the allowed timeout. Replacing with a new tablet ae9a02dd8c3647f19abe28647ee2b3ce
I20260202 02:52:52.235265 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "16b47a2831f54d809ed0b16deaec7e3d" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:52.235523 778 raft_consensus.cc:2393] T 16b47a2831f54d809ed0b16deaec7e3d P 1f442ef85029461091de847118c34c0d [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1521a6d262b7451bbef8debe970dd6b5 in current term 3: Already voted for candidate b1f05820b6034228bdad884cc348716d in this term.
W20260202 02:52:52.235796 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet f5727f9b9bbb4e77abfef065c51606a1 (table default.Table_4e6c3886f58b49698a00d45b957f4c9a [id=b5cc03b77f414304b29c5fb4bb7dc3ee]) was not created within the allowed timeout. Replacing with a new tablet fdc72096ee4d440789899e6e71580746
W20260202 02:52:52.236176 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet eea6bb04ddf24276b46270d009ed78b4 (table default.Table_1366b68811c14d28b1785266d6f65a3a [id=b7139f5062824d66af2dbd62469a9c03]) was not created within the allowed timeout. Replacing with a new tablet 54591d3d809648de930f7b4221c94e49
I20260202 02:52:52.236233 857 leader_election.cc:304] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:52.236524 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ee04563d3b8e4cb2b989bb8d6ee9b20e" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:52.236696 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet b427c7e897c64977b924e7d586420e69 (table default.Table_616a8ade4b5e481b8d4906b7a2a026d0 [id=98ee1a7044cd46d8b39dfda755ed8a7d]) was not created within the allowed timeout. Replacing with a new tablet 71dd6890e0e948d09d102296d1bd6464
W20260202 02:52:52.237039 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet 8b0368e03f644f1db28dbb57c95c48c1 (table default.Table_f8d8615b34774b738faeb83039f9601a [id=8882e1fbac2949838c99adb1290f273e]) was not created within the allowed timeout. Replacing with a new tablet d0e00bed25e740f5b39e023f9ed61d5a
I20260202 02:52:52.237167 2929 raft_consensus.cc:2749] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260202 02:52:52.237293 859 leader_election.cc:343] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:52.237356 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet 7709ea34429e4e91ad4922a3d73ebd14 (table default.Table_a9d14eea240d4bca96de4ea77b5b31f2 [id=c6e5418bf48e43e2adb6d1b60445f0b9]) was not created within the allowed timeout. Replacing with a new tablet 22c31519d7e642118ff1218e2a1b7621
I20260202 02:52:52.237428 859 leader_election.cc:304] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
W20260202 02:52:52.237624 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet 528d4269f91a44679acfad16e9fbbe32 (table default.Table_cdcc140c2b8d4aabae614d58873dc22a [id=c2c454eb645044839fa81b33b6460558]) was not created within the allowed timeout. Replacing with a new tablet 6f2739c054ce4e6b87634eced940d548
W20260202 02:52:52.237931 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet 1d8143f529b74e6ab779dd695920413c (table default.Table_33fc32ec03094e3c8fe78de4f6a26777 [id=85340c1716bd4235bdad7042b53ce58c]) was not created within the allowed timeout. Replacing with a new tablet 84d68594516e4be4a3a1f4ebe9e217a5
W20260202 02:52:52.238193 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet 51ba0bcd498043c383c493f45ad1fe17 (table default.Table_619d2fcf726a46bfb5e939bbe6f55612 [id=038692c44a0043bab69ab29f77aa13be]) was not created within the allowed timeout. Replacing with a new tablet dac0f74c3d3f410f8222764ba9cd7f98
W20260202 02:52:52.238451 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet 7a44585a217b48f39f247bae3ca2e255 (table default.Table_2ee0b536f96e44e4a3a4a9439c7bccc9 [id=d95486b11c494863be7380146ebcf0c8]) was not created within the allowed timeout. Replacing with a new tablet d8d706b1bf49494c871aee3ddc16e40c
W20260202 02:52:52.238751 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet 25a407001f964da6b51f42375136c79a (table default.Table_48226e7c77da41cdba56e04221eb9689 [id=05e52ce8e94e4975b51849059b904272]) was not created within the allowed timeout. Replacing with a new tablet b3e095157aec4096a2ea5db96872ae12
W20260202 02:52:52.239156 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet e2edc6660b9a41e39e9529627fafcb03 (table default.Table_9d4fbf619567432a88aa9a0c1b19cb3e [id=fff85ebd1bd1415c932f7c9c5a0da42a]) was not created within the allowed timeout. Replacing with a new tablet 5692ca1e36894c3194f0fcff18d014ba
I20260202 02:52:52.156216 1882 rpcz_store.cc:276] 0202 02:52:49.030810 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:49.030972 (+ 162us) service_pool.cc:225] Handling call
0202 02:52:52.156032 (+3125060us) ts_tablet_manager.cc:1151] Deleting tablet 8a206159afa84ae19f8c7763a2fe43d4
0202 02:52:52.156040 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:52.156133 (+ 93us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3124925}
W20260202 02:52:52.239897 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 8a206159afa84ae19f8c7763a2fe43d4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8a206159afa84ae19f8c7763a2fe43d4 already in progress: creating tablet
I20260202 02:52:52.240494 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 761) took 3209 ms. Trace:
I20260202 02:52:52.240659 1882 rpcz_store.cc:276] 0202 02:52:49.030898 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:49.031135 (+ 237us) service_pool.cc:225] Handling call
0202 02:52:52.240266 (+3209131us) ts_tablet_manager.cc:1151] Deleting tablet 9d0212e64d934968bdbdc627c69874b1
0202 02:52:52.240279 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:52.240471 (+ 192us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3209011}
W20260202 02:52:52.241196 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 9d0212e64d934968bdbdc627c69874b1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9d0212e64d934968bdbdc627c69874b1 already in progress: creating tablet
I20260202 02:52:52.241611 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 762) took 3210 ms. Trace:
I20260202 02:52:52.241724 1882 rpcz_store.cc:276] 0202 02:52:49.030977 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:49.031275 (+ 298us) service_pool.cc:225] Handling call
0202 02:52:52.241483 (+3210208us) ts_tablet_manager.cc:1151] Deleting tablet d1342810762e41588ea00ea02acad3e2
0202 02:52:52.241493 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:52.241596 (+ 103us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3210096}
W20260202 02:52:52.242182 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet d1342810762e41588ea00ea02acad3e2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d1342810762e41588ea00ea02acad3e2 already in progress: creating tablet
I20260202 02:52:52.242514 1882 tablet_replica.cc:333] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:52.242741 1882 raft_consensus.cc:2243] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d [term 2 LEADER]: Raft consensus shutting down.
I20260202 02:52:52.243105 1882 raft_consensus.cc:2272] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:52.243815 1882 ts_tablet_manager.cc:1916] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:52.248917 2948 leader_election.cc:290] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 3 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:52.249629 2925 raft_consensus.cc:2749] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:52.249843 2925 raft_consensus.cc:493] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:52.249967 2925 raft_consensus.cc:515] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:52.250581 1174 tablet_bootstrap.cc:492] T 2f5aae089fcc492f9ce55f84674a452c P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:52.250644 2925 leader_election.cc:290] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:52.250749 1174 ts_tablet_manager.cc:1403] T 2f5aae089fcc492f9ce55f84674a452c P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.038s user 0.004s sys 0.003s
I20260202 02:52:52.251435 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b76d1837813c46e8a0925b49939d8884" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:52.251255 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b76d1837813c46e8a0925b49939d8884" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:52.251571 1174 raft_consensus.cc:359] T 2f5aae089fcc492f9ce55f84674a452c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:52.251751 766 raft_consensus.cc:2468] T b76d1837813c46e8a0925b49939d8884 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 2.
I20260202 02:52:52.251758 1174 raft_consensus.cc:385] T 2f5aae089fcc492f9ce55f84674a452c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:52.251808 1059 raft_consensus.cc:2468] T b76d1837813c46e8a0925b49939d8884 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
I20260202 02:52:52.251904 1174 raft_consensus.cc:740] T 2f5aae089fcc492f9ce55f84674a452c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:52.252163 1174 consensus_queue.cc:260] T 2f5aae089fcc492f9ce55f84674a452c P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:52.252580 857 leader_election.cc:304] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters:
I20260202 02:52:52.252871 1174 ts_tablet_manager.cc:1434] T 2f5aae089fcc492f9ce55f84674a452c P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260202 02:52:52.252866 2925 raft_consensus.cc:2804] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260202 02:52:52.253053 2925 raft_consensus.cc:493] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:52.253228 1174 tablet_bootstrap.cc:492] T ac2ab12f5040488aa306cb69b107b485 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:52.253213 2925 raft_consensus.cc:3060] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Advancing to term 3
I20260202 02:52:52.254849 1174 tablet_bootstrap.cc:654] T ac2ab12f5040488aa306cb69b107b485 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:52.261453 2925 raft_consensus.cc:515] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:52.262441 1173 ts_tablet_manager.cc:1434] T 1d8143f529b74e6ab779dd695920413c P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.082s user 0.000s sys 0.002s
I20260202 02:52:52.262807 1173 tablet_bootstrap.cc:492] T 75a24a1bf54d4c799938a5723ac6fa99 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:52.264489 1173 tablet_bootstrap.cc:654] T 75a24a1bf54d4c799938a5723ac6fa99 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:52.269408 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b76d1837813c46e8a0925b49939d8884" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:52.269775 1059 raft_consensus.cc:3060] T b76d1837813c46e8a0925b49939d8884 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Advancing to term 3
I20260202 02:52:52.273202 2948 raft_consensus.cc:493] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:52.273494 1174 tablet_bootstrap.cc:492] T ac2ab12f5040488aa306cb69b107b485 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:52.273662 1174 ts_tablet_manager.cc:1403] T ac2ab12f5040488aa306cb69b107b485 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.005s
I20260202 02:52:52.274181 2948 raft_consensus.cc:515] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:52.274452 1174 raft_consensus.cc:359] T ac2ab12f5040488aa306cb69b107b485 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.274628 1174 raft_consensus.cc:385] T ac2ab12f5040488aa306cb69b107b485 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:52.274708 1174 raft_consensus.cc:740] T ac2ab12f5040488aa306cb69b107b485 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:52.274092 1059 raft_consensus.cc:2468] T b76d1837813c46e8a0925b49939d8884 P b1f05820b6034228bdad884cc348716d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 3.
I20260202 02:52:52.274917 1174 consensus_queue.cc:260] T ac2ab12f5040488aa306cb69b107b485 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.275147 1182 ts_tablet_manager.cc:1929] T 7ff6d2fa085343a6b11be0898da6233d P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:52.275326 1182 log.cc:1199] T 7ff6d2fa085343a6b11be0898da6233d P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/7ff6d2fa085343a6b11be0898da6233d
I20260202 02:52:52.275542 1174 ts_tablet_manager.cc:1434] T ac2ab12f5040488aa306cb69b107b485 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260202 02:52:52.275858 1182 ts_tablet_manager.cc:1950] T 7ff6d2fa085343a6b11be0898da6233d P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:52.276044 2944 raft_consensus.cc:493] T 587cfd0ae62f4cfd937faf6890e78f4a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:52.276187 2944 raft_consensus.cc:515] T 587cfd0ae62f4cfd937faf6890e78f4a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.276798 2944 leader_election.cc:290] T 587cfd0ae62f4cfd937faf6890e78f4a P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:52.277745 714 leader_election.cc:343] T 587cfd0ae62f4cfd937faf6890e78f4a P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 587cfd0ae62f4cfd937faf6890e78f4a
W20260202 02:52:52.278016 514 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 06016cee61b64c59b2f01e8f01e445cc (table default.Table_6ab16c2d02d84dbdb460ee7a7aeec421 [id=2a20bf11ada9426cb8c580e2fa143d71], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 06016cee61b64c59b2f01e8f01e445cc consensus state has no leader
I20260202 02:52:52.279827 525 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_58d515e7e01b4109b06e1e027ec60e3a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:52.280506 525 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_58d515e7e01b4109b06e1e027ec60e3a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:52.283979 999 leader_election.cc:343] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 2f5aae089fcc492f9ce55f84674a452c
I20260202 02:52:52.284579 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f5aae089fcc492f9ce55f84674a452c" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:52.284866 766 raft_consensus.cc:2468] T 2f5aae089fcc492f9ce55f84674a452c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:52.285310 998 leader_election.cc:304] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:52.286445 2794 raft_consensus.cc:2804] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:52.286594 2794 raft_consensus.cc:493] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:52.286715 2794 raft_consensus.cc:3060] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:52.289106 714 consensus_peers.cc:597] T 65bd1c1a382e42e4a68077264563c55b P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 65bd1c1a382e42e4a68077264563c55b. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:52.290314 2794 raft_consensus.cc:515] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:52.291621 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f5aae089fcc492f9ce55f84674a452c" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:52.291955 766 raft_consensus.cc:3060] T 2f5aae089fcc492f9ce55f84674a452c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:52.292479 2948 leader_election.cc:290] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:52.294682 766 raft_consensus.cc:2468] T 2f5aae089fcc492f9ce55f84674a452c P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:52.296226 998 leader_election.cc:304] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:52.297187 2794 leader_election.cc:290] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:52.297608 999 leader_election.cc:343] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 2f5aae089fcc492f9ce55f84674a452c
I20260202 02:52:52.298043 2948 raft_consensus.cc:2804] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:52.298149 2948 raft_consensus.cc:697] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:52.298020 2794 consensus_queue.cc:1048] T 939fbcaf75c7439d9208bbbf83a72717 P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:52.298380 2948 consensus_queue.cc:237] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:52.300109 2740 raft_consensus.cc:697] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [term 3 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:52.300592 2740 consensus_queue.cc:237] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.302465 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "587cfd0ae62f4cfd937faf6890e78f4a" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:52.302668 1059 raft_consensus.cc:2468] T 587cfd0ae62f4cfd937faf6890e78f4a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:52.303159 1174 tablet_bootstrap.cc:492] T 2b352d9a815e4bccafb7e2251500c5bc P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:52.303205 715 leader_election.cc:304] T 587cfd0ae62f4cfd937faf6890e78f4a P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:52.303665 2911 raft_consensus.cc:2804] T 587cfd0ae62f4cfd937faf6890e78f4a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:52.303764 2911 raft_consensus.cc:493] T 587cfd0ae62f4cfd937faf6890e78f4a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:52.303870 2911 raft_consensus.cc:3060] T 587cfd0ae62f4cfd937faf6890e78f4a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:52.307524 2911 raft_consensus.cc:515] T 587cfd0ae62f4cfd937faf6890e78f4a P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.308772 2911 leader_election.cc:290] T 587cfd0ae62f4cfd937faf6890e78f4a P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:52.309789 714 leader_election.cc:343] T 587cfd0ae62f4cfd937faf6890e78f4a P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 587cfd0ae62f4cfd937faf6890e78f4a
I20260202 02:52:52.275336 859 leader_election.cc:304] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:52.313263 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "587cfd0ae62f4cfd937faf6890e78f4a" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:52.313482 1059 raft_consensus.cc:3060] T 587cfd0ae62f4cfd937faf6890e78f4a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:52.313521 2929 raft_consensus.cc:2804] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260202 02:52:52.313819 2929 raft_consensus.cc:697] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [term 3 LEADER]: Becoming Leader. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:52.314713 2925 leader_election.cc:290] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 3 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:52.315546 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b76d1837813c46e8a0925b49939d8884" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:52.315837 766 raft_consensus.cc:3060] T b76d1837813c46e8a0925b49939d8884 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Advancing to term 3
I20260202 02:52:52.316083 1059 raft_consensus.cc:2468] T 587cfd0ae62f4cfd937faf6890e78f4a P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:52.316511 715 leader_election.cc:304] T 587cfd0ae62f4cfd937faf6890e78f4a P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:52.316749 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 7ff6d2fa085343a6b11be0898da6233d (table default.Table_bcf2aa761e104360bb4a1251204a6608 [id=5eaf1a210fa44066b81bb5b01db4d7cf]) successfully deleted
I20260202 02:52:52.318167 2633 raft_consensus.cc:2804] T 587cfd0ae62f4cfd937faf6890e78f4a P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:52.318307 2633 raft_consensus.cc:697] T 587cfd0ae62f4cfd937faf6890e78f4a P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:52.318555 2633 consensus_queue.cc:237] T 587cfd0ae62f4cfd937faf6890e78f4a P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
W20260202 02:52:52.319027 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 0291c3e873bb4266ab040356fe35f4e9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0291c3e873bb4266ab040356fe35f4e9 already in progress: creating tablet
W20260202 02:52:52.320768 522 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 11610f2d5220411aa28b47d2dd724f00 consensus state has no leader
I20260202 02:52:52.324234 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_f73b956d6f8b45c39a325ae61487bc80"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:52.324463 714 consensus_peers.cc:597] T ccd6bc8bd0504bf18d108c73685ef9ff P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ccd6bc8bd0504bf18d108c73685ef9ff. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:52.324911 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f73b956d6f8b45c39a325ae61487bc80 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:52.326552 519 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_6f5ad3d161b6498ebccd807c6d76430f" } modify_external_catalogs: true
I20260202 02:52:52.326776 519 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_6f5ad3d161b6498ebccd807c6d76430f" } modify_external_catalogs: true
I20260202 02:52:52.328240 1182 tablet_replica.cc:333] T b4a4c75a906f40beb896bf6fff705ae6 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:52.329708 1182 raft_consensus.cc:2243] T b4a4c75a906f40beb896bf6fff705ae6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:52.330471 1182 raft_consensus.cc:2272] T b4a4c75a906f40beb896bf6fff705ae6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:52.330711 525 master_service.cc:954] ReplaceTablet: received request to replace tablet fa57a267b0484262ac31039862f40a76 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:52.331518 1182 ts_tablet_manager.cc:1916] T b4a4c75a906f40beb896bf6fff705ae6 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:52.332473 522 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_6e6ab095d9a6497d8a89bcc865b6f42f" } new_table_name: "default.Table_bb379cca6ffa4183bea51c1e561862e7" modify_external_catalogs: true
W20260202 02:52:52.338768 522 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 34a01a4ce1be4ad08d911763de158073 consensus state has no leader
I20260202 02:52:52.339094 766 raft_consensus.cc:2468] T b76d1837813c46e8a0925b49939d8884 P 1f442ef85029461091de847118c34c0d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 3.
I20260202 02:52:52.339284 522 master_service.cc:954] ReplaceTablet: received request to replace tablet b9fc33e8efc04734a951f7fcf9048142 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:52.340065 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_ee948408dac543078c1e7453d5930acc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:52.340951 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ee948408dac543078c1e7453d5930acc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:52.341168 2968 raft_consensus.cc:493] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:52.342383 2968 raft_consensus.cc:515] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.342734 1174 tablet_bootstrap.cc:654] T 2b352d9a815e4bccafb7e2251500c5bc P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:52.333436 1175 ts_tablet_manager.cc:1434] T dc90eeb79ded4739883ad98928476680 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.164s user 0.007s sys 0.009s
I20260202 02:52:52.346601 1175 tablet_bootstrap.cc:492] T 5e20842de8cb4355b732685605c88834 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:52.348474 1175 tablet_bootstrap.cc:654] T 5e20842de8cb4355b732685605c88834 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:52.314133 2929 consensus_queue.cc:237] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:52.344511 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e2edc6660b9a41e39e9529627fafcb03" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:52.349465 913 raft_consensus.cc:2468] T e2edc6660b9a41e39e9529627fafcb03 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:52.349888 999 leader_election.cc:304] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:52.351564 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_bc3f75a4a12c4f92a4bbf19395b7800a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:52.352267 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bc3f75a4a12c4f92a4bbf19395b7800a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:52.357111 2860 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260202 02:52:52.360734 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_768da74ea5aa47e187113ae83e1a5b72"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:52.361653 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_768da74ea5aa47e187113ae83e1a5b72 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:52.364589 1174 tablet_bootstrap.cc:492] T 2b352d9a815e4bccafb7e2251500c5bc P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:52.364527 2911 consensus_queue.cc:1048] T ccd6bc8bd0504bf18d108c73685ef9ff P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:52.364760 1174 ts_tablet_manager.cc:1403] T 2b352d9a815e4bccafb7e2251500c5bc P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.062s user 0.007s sys 0.003s
I20260202 02:52:52.369278 1174 raft_consensus.cc:359] T 2b352d9a815e4bccafb7e2251500c5bc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.369519 1174 raft_consensus.cc:385] T 2b352d9a815e4bccafb7e2251500c5bc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:52.369601 1174 raft_consensus.cc:740] T 2b352d9a815e4bccafb7e2251500c5bc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:52.369971 522 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_69b88340567c4887947ed8458ffd1cba" } new_table_name: "default.Table_4f72bc6302d84055ba28584ec5e3ace9" modify_external_catalogs: true
I20260202 02:52:52.370744 1882 ts_tablet_manager.cc:1929] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260202 02:52:52.370874 1882 log.cc:1199] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/26b890ed87c3471b9fc0b172c90e0a11
I20260202 02:52:52.371248 1882 ts_tablet_manager.cc:1950] T 26b890ed87c3471b9fc0b172c90e0a11 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:52.371147 1174 consensus_queue.cc:260] T 2b352d9a815e4bccafb7e2251500c5bc P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.372099 1174 ts_tablet_manager.cc:1434] T 2b352d9a815e4bccafb7e2251500c5bc P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260202 02:52:52.372089 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 763) took 3296 ms. Trace:
I20260202 02:52:52.372174 1882 rpcz_store.cc:276] 0202 02:52:49.076076 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:49.076135 (+ 59us) service_pool.cc:225] Handling call
0202 02:52:52.242465 (+3166330us) ts_tablet_manager.cc:1151] Deleting tablet 26b890ed87c3471b9fc0b172c90e0a11
0202 02:52:52.242475 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:52.246926 (+ 4451us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:52.370613 (+123687us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:52.372066 (+ 1453us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3166145}
I20260202 02:52:52.372442 1174 tablet_bootstrap.cc:492] T 0138ba7326674741b0729324735f1de7 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:52.372673 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 26b890ed87c3471b9fc0b172c90e0a11 (table default.Table_044ef6a2b1af4e3cab3c8cf00505c6a6 [id=d2e4220a0cbf443e8380433da4f064f8]) successfully deleted
I20260202 02:52:52.374030 1174 tablet_bootstrap.cc:654] T 0138ba7326674741b0729324735f1de7 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:52.378635 2980 raft_consensus.cc:2804] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:52.380259 1174 tablet_bootstrap.cc:492] T 0138ba7326674741b0729324735f1de7 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:52.380484 2980 raft_consensus.cc:493] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:52.380769 2980 raft_consensus.cc:3060] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:52.381825 2968 leader_election.cc:290] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:52.387969 1174 ts_tablet_manager.cc:1403] T 0138ba7326674741b0729324735f1de7 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.001s
I20260202 02:52:52.389027 1174 raft_consensus.cc:359] T 0138ba7326674741b0729324735f1de7 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.389225 1174 raft_consensus.cc:385] T 0138ba7326674741b0729324735f1de7 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:52.389286 1174 raft_consensus.cc:740] T 0138ba7326674741b0729324735f1de7 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:52.382536 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e2edc6660b9a41e39e9529627fafcb03" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:52.392946 1174 consensus_queue.cc:260] T 0138ba7326674741b0729324735f1de7 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
W20260202 02:52:52.393136 998 leader_election.cc:343] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:52.396716 1180 ts_tablet_manager.cc:1929] T 75569511179b41038e521fa3346a92ae P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:52.396988 1180 log.cc:1199] T 75569511179b41038e521fa3346a92ae P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/75569511179b41038e521fa3346a92ae
I20260202 02:52:52.397212 1174 ts_tablet_manager.cc:1434] T 0138ba7326674741b0729324735f1de7 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260202 02:52:52.397451 1180 ts_tablet_manager.cc:1950] T 75569511179b41038e521fa3346a92ae P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:52.397569 1174 tablet_bootstrap.cc:492] T 3284213e4b0447b49a7718ba896d3fa8 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:52.398321 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 691) took 21854 ms. Trace:
I20260202 02:52:52.400024 2980 raft_consensus.cc:515] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.402683 2980 leader_election.cc:290] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:52.407567 999 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 57703.6us
I20260202 02:52:52.408596 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e2edc6660b9a41e39e9529627fafcb03" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:52.408814 913 raft_consensus.cc:3060] T e2edc6660b9a41e39e9529627fafcb03 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:52.411612 913 raft_consensus.cc:2468] T e2edc6660b9a41e39e9529627fafcb03 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:52.417536 2929 raft_consensus.cc:493] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:52.418025 2964 raft_consensus.cc:493] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b1f05820b6034228bdad884cc348716d)
I20260202 02:52:52.418202 2964 raft_consensus.cc:515] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:52.412529 999 leader_election.cc:304] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:52.418809 2794 raft_consensus.cc:2804] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:52.418995 2794 raft_consensus.cc:697] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:52.419275 2794 consensus_queue.cc:237] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.420197 2964 leader_election.cc:290] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:52.420506 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cb634d4bae554ef3b39eeaa9958adc6c" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:52.420689 1046 raft_consensus.cc:2468] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 2.
I20260202 02:52:52.421212 859 leader_election.cc:304] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:52.421555 2964 raft_consensus.cc:2804] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260202 02:52:52.421634 2964 raft_consensus.cc:493] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b1f05820b6034228bdad884cc348716d)
I20260202 02:52:52.421705 2964 raft_consensus.cc:3060] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:52.399276 1174 tablet_bootstrap.cc:654] T 3284213e4b0447b49a7718ba896d3fa8 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:52.401911 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e2edc6660b9a41e39e9529627fafcb03" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:52.419312 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cb634d4bae554ef3b39eeaa9958adc6c" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:52.426198 778 raft_consensus.cc:2393] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1521a6d262b7451bbef8debe970dd6b5 in current term 2: Already voted for candidate 1f442ef85029461091de847118c34c0d in this term.
I20260202 02:52:52.432873 2794 raft_consensus.cc:493] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:52.433050 2794 raft_consensus.cc:515] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:52.433965 2794 leader_election.cc:290] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:52.435137 999 leader_election.cc:343] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 25765fb687294e9db1289f76530dd877
I20260202 02:52:52.435572 1175 tablet_bootstrap.cc:492] T 5e20842de8cb4355b732685605c88834 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:52.435724 1175 ts_tablet_manager.cc:1403] T 5e20842de8cb4355b732685605c88834 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.089s user 0.002s sys 0.006s
I20260202 02:52:52.436578 1175 raft_consensus.cc:359] T 5e20842de8cb4355b732685605c88834 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:52.436785 1175 raft_consensus.cc:385] T 5e20842de8cb4355b732685605c88834 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:52.436832 1175 raft_consensus.cc:740] T 5e20842de8cb4355b732685605c88834 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:52.437047 1175 consensus_queue.cc:260] T 5e20842de8cb4355b732685605c88834 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:52.437580 1175 ts_tablet_manager.cc:1434] T 5e20842de8cb4355b732685605c88834 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:52.437801 2929 raft_consensus.cc:515] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.439205 2929 leader_election.cc:290] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:52.439457 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c8a273ac8cd344719a48fd81945bed0a" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:52.440192 859 leader_election.cc:343] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:52.453239 2964 raft_consensus.cc:515] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:52.454355 2964 leader_election.cc:290] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:52.454445 764 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cb634d4bae554ef3b39eeaa9958adc6c" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:52.454640 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cb634d4bae554ef3b39eeaa9958adc6c" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:52.457484 1059 raft_consensus.cc:2468] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 2.
I20260202 02:52:52.462906 1175 tablet_bootstrap.cc:492] T f5727f9b9bbb4e77abfef065c51606a1 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:52.467797 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c8a273ac8cd344719a48fd81945bed0a" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:52.468278 1180 rpcz_store.cc:276] 0202 02:52:30.543770 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.548478 (+ 4708us) service_pool.cc:225] Handling call
0202 02:52:52.185987 (+21637509us) ts_tablet_manager.cc:1151] Deleting tablet 75569511179b41038e521fa3346a92ae
0202 02:52:52.185998 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:52.269510 (+ 83512us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:52.396545 (+127035us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:52.398268 (+ 1723us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":21637332}
I20260202 02:52:52.469321 764 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "25765fb687294e9db1289f76530dd877" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:52.469432 1175 tablet_bootstrap.cc:654] T f5727f9b9bbb4e77abfef065c51606a1 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:52.469568 764 raft_consensus.cc:2468] T 25765fb687294e9db1289f76530dd877 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:52.468919 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 75569511179b41038e521fa3346a92ae (table default.Table_6da292362a6c43738dbeb895750b918a [id=6d50800a3c8642c0992dd1a7f38cfd30]) successfully deleted
I20260202 02:52:52.473186 1180 tablet_replica.cc:333] T fe9ea4c0502f4976b440465992f2ea2c P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:52.473459 1180 raft_consensus.cc:2243] T fe9ea4c0502f4976b440465992f2ea2c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:52.473661 1180 raft_consensus.cc:2272] T fe9ea4c0502f4976b440465992f2ea2c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:52.474273 1180 ts_tablet_manager.cc:1916] T fe9ea4c0502f4976b440465992f2ea2c P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:52.477329 766 raft_consensus.cc:2468] T c8a273ac8cd344719a48fd81945bed0a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
I20260202 02:52:52.477912 857 leader_election.cc:304] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:52.482295 2964 raft_consensus.cc:2804] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:52.482409 2964 raft_consensus.cc:493] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:52.482475 2964 raft_consensus.cc:3060] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:52.489814 2964 raft_consensus.cc:515] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.490468 2964 leader_election.cc:290] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:52.491959 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c8a273ac8cd344719a48fd81945bed0a" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:52.492436 766 raft_consensus.cc:3060] T c8a273ac8cd344719a48fd81945bed0a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:52.493734 526 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 27158aa96e2742d5b0b2c3fa9531178d
I20260202 02:52:52.494453 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_b6f0e40295b247c493d61b5369e218f3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:52.495000 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b6f0e40295b247c493d61b5369e218f3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:52.496584 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 27158aa96e2742d5b0b2c3fa9531178d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.501493 766 raft_consensus.cc:2468] T c8a273ac8cd344719a48fd81945bed0a P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
I20260202 02:52:52.501513 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 27158aa96e2742d5b0b2c3fa9531178d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.502409 857 leader_election.cc:304] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters:
I20260202 02:52:52.503340 1174 tablet_bootstrap.cc:492] T 3284213e4b0447b49a7718ba896d3fa8 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:52.503495 1174 ts_tablet_manager.cc:1403] T 3284213e4b0447b49a7718ba896d3fa8 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.106s user 0.003s sys 0.006s
I20260202 02:52:52.504447 1174 raft_consensus.cc:359] T 3284213e4b0447b49a7718ba896d3fa8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.504710 1174 raft_consensus.cc:385] T 3284213e4b0447b49a7718ba896d3fa8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:52.504839 1174 raft_consensus.cc:740] T 3284213e4b0447b49a7718ba896d3fa8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:52.505147 1174 consensus_queue.cc:260] T 3284213e4b0447b49a7718ba896d3fa8 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.505681 1174 ts_tablet_manager.cc:1434] T 3284213e4b0447b49a7718ba896d3fa8 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:52.505795 1180 ts_tablet_manager.cc:1929] T fe9ea4c0502f4976b440465992f2ea2c P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:52.505973 1180 log.cc:1199] T fe9ea4c0502f4976b440465992f2ea2c P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/fe9ea4c0502f4976b440465992f2ea2c
I20260202 02:52:52.506369 1180 ts_tablet_manager.cc:1950] T fe9ea4c0502f4976b440465992f2ea2c P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:52.509371 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 692) took 21965 ms. Trace:
I20260202 02:52:52.509855 1180 rpcz_store.cc:276] 0202 02:52:30.543884 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.552376 (+ 8492us) service_pool.cc:225] Handling call
0202 02:52:52.468609 (+21916233us) ts_tablet_manager.cc:1151] Deleting tablet fe9ea4c0502f4976b440465992f2ea2c
0202 02:52:52.468621 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:52.479347 (+ 10726us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:52.498161 (+ 18814us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:52.509320 (+ 11159us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":21895295}
I20260202 02:52:52.510474 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet fe9ea4c0502f4976b440465992f2ea2c (table default.Table_9181c8b857b540ff846c397f9f1a5484 [id=d2fb3868962547c48e29f6353e5747a9]) successfully deleted
I20260202 02:52:52.509778 1174 tablet_bootstrap.cc:492] T 7ea7a5d8a92b4ac0903483627258ae66 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:52.460562 1882 tablet_replica.cc:333] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:52.512207 1882 raft_consensus.cc:2243] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [term 3 FOLLOWER]: Raft consensus shutting down.
W20260202 02:52:52.462292 998 leader_election.cc:343] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:52.467054 1173 tablet_bootstrap.cc:492] T 75a24a1bf54d4c799938a5723ac6fa99 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:52.491305 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c8a273ac8cd344719a48fd81945bed0a" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:52.497298 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 27158aa96e2742d5b0b2c3fa9531178d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:52.512552 998 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 50273.6us
I20260202 02:52:52.513080 1173 ts_tablet_manager.cc:1403] T 75a24a1bf54d4c799938a5723ac6fa99 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.250s user 0.000s sys 0.006s
I20260202 02:52:52.515951 1882 raft_consensus.cc:2272] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:52.516585 1882 ts_tablet_manager.cc:1916] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:52.503715 2964 raft_consensus.cc:2804] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:52.518151 2964 raft_consensus.cc:697] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:52.518424 2964 consensus_queue.cc:237] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.518765 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 719) took 21972 ms. Trace:
I20260202 02:52:52.519415 1180 rpcz_store.cc:276] 0202 02:52:30.546204 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:30.683129 (+136925us) service_pool.cc:225] Handling call
0202 02:52:52.518113 (+21834984us) ts_tablet_manager.cc:1151] Deleting tablet 91a604fb877a48479914a5b60144a22d
0202 02:52:52.518125 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:52.518712 (+ 587us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":16745053}
W20260202 02:52:52.520013 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 91a604fb877a48479914a5b60144a22d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 91a604fb877a48479914a5b60144a22d already in progress: creating tablet
I20260202 02:52:52.520561 998 leader_election.cc:304] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:52.521001 2794 consensus_queue.cc:1048] T 939fbcaf75c7439d9208bbbf83a72717 P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:52.519331 1174 tablet_bootstrap.cc:654] T 7ea7a5d8a92b4ac0903483627258ae66 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:52.521432 2794 raft_consensus.cc:2804] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260202 02:52:52.521512 2794 raft_consensus.cc:493] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:52.521579 2794 raft_consensus.cc:3060] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Advancing to term 3
I20260202 02:52:52.521543 1180 tablet_replica.cc:333] T d8418a9dd0944456b4d4820dde0cde02 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:52.524724 1180 raft_consensus.cc:2243] T d8418a9dd0944456b4d4820dde0cde02 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:52.524219 2794 raft_consensus.cc:515] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:52.526436 2881 raft_consensus.cc:493] T 79badde1001b432b95f58b5161ace15c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:52.527323 2881 raft_consensus.cc:515] T 79badde1001b432b95f58b5161ace15c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:52.527948 2881 leader_election.cc:290] T 79badde1001b432b95f58b5161ace15c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
W20260202 02:52:52.512971 859 leader_election.cc:343] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:52.528262 764 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "25765fb687294e9db1289f76530dd877" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:52.528482 764 raft_consensus.cc:3060] T 25765fb687294e9db1289f76530dd877 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Advancing to term 3
W20260202 02:52:52.528477 999 leader_election.cc:343] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 3 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 25765fb687294e9db1289f76530dd877
I20260202 02:52:52.528421 1173 raft_consensus.cc:359] T 75a24a1bf54d4c799938a5723ac6fa99 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.528632 1173 raft_consensus.cc:385] T 75a24a1bf54d4c799938a5723ac6fa99 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:52.528685 1173 raft_consensus.cc:740] T 75a24a1bf54d4c799938a5723ac6fa99 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:52.528858 1173 consensus_queue.cc:260] T 75a24a1bf54d4c799938a5723ac6fa99 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.529912 2794 leader_election.cc:290] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 3 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:52.531203 764 raft_consensus.cc:2468] T 25765fb687294e9db1289f76530dd877 P 1f442ef85029461091de847118c34c0d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 3.
I20260202 02:52:52.532918 2944 raft_consensus.cc:493] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b1f05820b6034228bdad884cc348716d)
I20260202 02:52:52.533056 2944 raft_consensus.cc:515] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.533627 2944 leader_election.cc:290] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:52.533562 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_7f1d13e7a79e4a86ad883eb6e5d214e0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:52.533901 2911 raft_consensus.cc:493] T 73f8db8f30f14d369d2227c9be7c4647 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
I20260202 02:52:52.534009 2911 raft_consensus.cc:515] T 73f8db8f30f14d369d2227c9be7c4647 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:52.534157 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7f1d13e7a79e4a86ad883eb6e5d214e0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:52.534235 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79badde1001b432b95f58b5161ace15c" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:52.534438 1046 raft_consensus.cc:2393] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f442ef85029461091de847118c34c0d in current term 1: Already voted for candidate b1f05820b6034228bdad884cc348716d in this term.
I20260202 02:52:52.534456 2911 leader_election.cc:290] T 73f8db8f30f14d369d2227c9be7c4647 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:52.535293 1180 raft_consensus.cc:2272] T d8418a9dd0944456b4d4820dde0cde02 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:52.535962 1180 ts_tablet_manager.cc:1916] T d8418a9dd0944456b4d4820dde0cde02 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:52.536638 1054 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "73f8db8f30f14d369d2227c9be7c4647" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:52.537791 714 leader_election.cc:343] T 79badde1001b432b95f58b5161ace15c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 79badde1001b432b95f58b5161ace15c
I20260202 02:52:52.537935 714 leader_election.cc:304] T 79badde1001b432b95f58b5161ace15c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
W20260202 02:52:52.538344 714 leader_election.cc:343] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 874ca4ba4ebc4721bb4b60cba8e01841
W20260202 02:52:52.538625 715 leader_election.cc:343] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Not found: Tablet not found: 874ca4ba4ebc4721bb4b60cba8e01841
I20260202 02:52:52.538698 715 leader_election.cc:304] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:52.539139 2881 raft_consensus.cc:3060] T 79badde1001b432b95f58b5161ace15c P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:52.541729 2881 raft_consensus.cc:2749] T 79badde1001b432b95f58b5161ace15c P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:52.541927 2881 raft_consensus.cc:2749] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260202 02:52:52.542557 526 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_8ffb80597a3c44f3ad9abbfffddda115" } new_table_name: "default.Table_09d660d79a7f40a19ce5c0628d0e9ee8" modify_external_catalogs: true
I20260202 02:52:52.545732 998 leader_election.cc:304] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:52.547159 1173 ts_tablet_manager.cc:1434] T 75a24a1bf54d4c799938a5723ac6fa99 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.034s user 0.001s sys 0.000s
W20260202 02:52:52.547551 714 leader_election.cc:343] T 73f8db8f30f14d369d2227c9be7c4647 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 73f8db8f30f14d369d2227c9be7c4647
I20260202 02:52:52.547668 714 leader_election.cc:304] T 73f8db8f30f14d369d2227c9be7c4647 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
W20260202 02:52:52.550978 526 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6b2a86a251da4196ba21848166b29de1 consensus state has no leader
I20260202 02:52:52.551342 526 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_f73b956d6f8b45c39a325ae61487bc80" } modify_external_catalogs: true
I20260202 02:52:52.551494 526 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_f73b956d6f8b45c39a325ae61487bc80" } modify_external_catalogs: true
I20260202 02:52:52.551579 1173 tablet_bootstrap.cc:492] T 8b0368e03f644f1db28dbb57c95c48c1 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
W20260202 02:52:52.553355 999 consensus_peers.cc:597] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d0ce2121d65b4a228596e0ca9018883d. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:52.557760 1173 tablet_bootstrap.cc:654] T 8b0368e03f644f1db28dbb57c95c48c1 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:52.558634 2881 raft_consensus.cc:2749] T 73f8db8f30f14d369d2227c9be7c4647 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260202 02:52:52.547654 2794 raft_consensus.cc:2804] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [term 3 FOLLOWER]: Leader election won for term 3
I20260202 02:52:52.559808 2794 raft_consensus.cc:697] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [term 3 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:52.560070 2794 consensus_queue.cc:237] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:52.561928 526 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet d29eb5955d4e48a4893a128aa539b8eb
I20260202 02:52:52.562242 526 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_15cd35fa6b4341fab4701f7afb43bd74" } new_table_name: "default.Table_6e5bb72ecb274cad99b5745622019ebd" modify_external_catalogs: true
I20260202 02:52:52.511945 1182 ts_tablet_manager.cc:1929] T b4a4c75a906f40beb896bf6fff705ae6 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:52.566403 1182 log.cc:1199] T b4a4c75a906f40beb896bf6fff705ae6 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/b4a4c75a906f40beb896bf6fff705ae6
I20260202 02:52:52.566772 1182 ts_tablet_manager.cc:1950] T b4a4c75a906f40beb896bf6fff705ae6 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:52.567283 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet b4a4c75a906f40beb896bf6fff705ae6 (table default.Table_6806575803c4468fa7b7ef990004b446 [id=944fbe1acd894017bae3c54f135ee9b0]) successfully deleted
I20260202 02:52:52.567699 1180 ts_tablet_manager.cc:1929] T d8418a9dd0944456b4d4820dde0cde02 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:52.567854 1180 log.cc:1199] T d8418a9dd0944456b4d4820dde0cde02 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/d8418a9dd0944456b4d4820dde0cde02
W20260202 02:52:52.568229 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 104b2c537c4f4e3da6711364cb5d2a2e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 104b2c537c4f4e3da6711364cb5d2a2e already in progress: creating tablet
I20260202 02:52:52.568370 1180 ts_tablet_manager.cc:1950] T d8418a9dd0944456b4d4820dde0cde02 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:52.568482 1182 tablet_replica.cc:333] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:52.568697 1182 raft_consensus.cc:2243] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:52.568859 1182 raft_consensus.cc:2272] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:52.569273 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 720) took 16794 ms. Trace:
I20260202 02:52:52.569370 1180 rpcz_store.cc:276] 0202 02:52:35.775161 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:35.775699 (+ 538us) service_pool.cc:225] Handling call
0202 02:52:52.520406 (+16744707us) ts_tablet_manager.cc:1151] Deleting tablet d8418a9dd0944456b4d4820dde0cde02
0202 02:52:52.520417 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:52.538904 (+ 18487us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:52.550927 (+ 12023us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:52.569250 (+ 18323us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":16744540}
I20260202 02:52:52.569842 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet d8418a9dd0944456b4d4820dde0cde02 (table default.Table_38201dffd3a349508fdedab6e81e4965 [id=e620930eae8e4707b93e7ab16369e13d]) successfully deleted
W20260202 02:52:52.572412 526 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6167abf7b93b48f090ff56a57a49fe41 consensus state has no leader
I20260202 02:52:52.572522 1180 tablet_replica.cc:333] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:52.572728 1180 raft_consensus.cc:2243] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:52.572925 1180 raft_consensus.cc:2272] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:52.572960 526 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_6cf38c67711747acb1b5be35632b7a68" } modify_external_catalogs: true
I20260202 02:52:52.573133 526 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_6cf38c67711747acb1b5be35632b7a68" } modify_external_catalogs: true
I20260202 02:52:52.573976 1180 ts_tablet_manager.cc:1916] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:52.575536 714 consensus_peers.cc:597] T 1f379a29e2454f689d68f5951837512b P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1f379a29e2454f689d68f5951837512b. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:52.579883 1173 tablet_bootstrap.cc:492] T 8b0368e03f644f1db28dbb57c95c48c1 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:52.582317 859 leader_election.cc:304] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:52.582831 2929 raft_consensus.cc:2804] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Leader election won for term 2
I20260202 02:52:52.582957 2929 raft_consensus.cc:697] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [term 2 LEADER]: Becoming Leader. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:52.583194 2929 consensus_queue.cc:237] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:52.591796 524 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 41a4143247ed4eed98defbdfe2895372
I20260202 02:52:52.592298 524 catalog_manager.cc:6724] ReplaceTablet: tablet 41a4143247ed4eed98defbdfe2895372 deleted and replaced by tablet 3a4534ce9acb4e0ebb202e7695bbf4d5
I20260202 02:52:52.593294 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 41a4143247ed4eed98defbdfe2895372 with delete_type TABLET_DATA_DELETED (replaced by tablet 3a4534ce9acb4e0ebb202e7695bbf4d5) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.593284 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_92f45fe4aded4c79816e651a2be7ff27"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:52.593974 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_92f45fe4aded4c79816e651a2be7ff27 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:52.595505 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 41a4143247ed4eed98defbdfe2895372 with delete_type TABLET_DATA_DELETED (replaced by tablet 3a4534ce9acb4e0ebb202e7695bbf4d5) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.599007 1180 ts_tablet_manager.cc:1929] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260202 02:52:52.596118 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 41a4143247ed4eed98defbdfe2895372 with delete_type TABLET_DATA_DELETED (replaced by tablet 3a4534ce9acb4e0ebb202e7695bbf4d5) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.602322 1180 log.cc:1199] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/91ec01d25f8f441cbe3783cacbee51db
I20260202 02:52:52.602734 1180 ts_tablet_manager.cc:1950] T 91ec01d25f8f441cbe3783cacbee51db P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:52.603363 1173 ts_tablet_manager.cc:1403] T 8b0368e03f644f1db28dbb57c95c48c1 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.052s user 0.003s sys 0.003s
I20260202 02:52:52.603644 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 721) took 16828 ms. Trace:
I20260202 02:52:52.603790 1180 rpcz_store.cc:276] 0202 02:52:35.775271 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:35.775893 (+ 622us) service_pool.cc:225] Handling call
0202 02:52:52.572463 (+16796570us) ts_tablet_manager.cc:1151] Deleting tablet 91ec01d25f8f441cbe3783cacbee51db
0202 02:52:52.572476 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:52.581579 (+ 9103us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:52.598876 (+ 17297us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:52.603624 (+ 4748us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":16796446}
I20260202 02:52:52.604161 524 master_service.cc:954] ReplaceTablet: received request to replace tablet b00b4a0b85374bfe83928162c4da6479 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:52.604146 1173 raft_consensus.cc:359] T 8b0368e03f644f1db28dbb57c95c48c1 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.604499 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 91ec01d25f8f441cbe3783cacbee51db (table default.Table_23afb316ffac47aea4db07bda15a956b [id=255f4d94812f45d0b4423150098460ff]) successfully deleted
I20260202 02:52:52.604861 1180 tablet_replica.cc:333] T fbcb4a522df648ba9542250450c08f1b P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:52.607101 1180 raft_consensus.cc:2243] T fbcb4a522df648ba9542250450c08f1b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:52.607357 1180 raft_consensus.cc:2272] T fbcb4a522df648ba9542250450c08f1b P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:52.602962 1182 ts_tablet_manager.cc:1916] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:52.610160 1180 ts_tablet_manager.cc:1916] T fbcb4a522df648ba9542250450c08f1b P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:52.611214 527 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 44c8d5b74c8e4c499f7f68e8185efc17 (table default.Table_f48ed1c7e02349adad7ce22c0e51ed81 [id=46b5d47377474ff9bac2acdb26b08c65]). Expected version 3 got 0
I20260202 02:52:52.612394 1173 raft_consensus.cc:385] T 8b0368e03f644f1db28dbb57c95c48c1 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:52.612505 1173 raft_consensus.cc:740] T 8b0368e03f644f1db28dbb57c95c48c1 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:52.612787 1173 consensus_queue.cc:260] T 8b0368e03f644f1db28dbb57c95c48c1 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.613963 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_399d6088949d4b3488bdde3ee5dcf41f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:52.614532 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_399d6088949d4b3488bdde3ee5dcf41f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:52.615808 1173 ts_tablet_manager.cc:1434] T 8b0368e03f644f1db28dbb57c95c48c1 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.012s user 0.002s sys 0.000s
I20260202 02:52:52.616381 1173 tablet_bootstrap.cc:492] T b427c7e897c64977b924e7d586420e69 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:52.617856 1173 tablet_bootstrap.cc:654] T b427c7e897c64977b924e7d586420e69 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:52.618875 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet da5435f969074364bc8f8a6ddc5e4f8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.619834 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 26b890ed87c3471b9fc0b172c90e0a11 with delete_type TABLET_DATA_DELETED (replaced by tablet e5ab9384afe14e4a875be18ffdb863c3) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.620395 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 8153510d5d154c7895b1ee794585612e with delete_type TABLET_DATA_DELETED (Replaced by 9f805fa060d54d7e9067b5120f3d28ad at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.621999 1174 tablet_bootstrap.cc:492] T 7ea7a5d8a92b4ac0903483627258ae66 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:52.622195 1174 ts_tablet_manager.cc:1403] T 7ea7a5d8a92b4ac0903483627258ae66 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.113s user 0.018s sys 0.001s
I20260202 02:52:52.622915 1174 raft_consensus.cc:359] T 7ea7a5d8a92b4ac0903483627258ae66 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.623121 1174 raft_consensus.cc:385] T 7ea7a5d8a92b4ac0903483627258ae66 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:52.623188 1174 raft_consensus.cc:740] T 7ea7a5d8a92b4ac0903483627258ae66 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:52.623528 1174 consensus_queue.cc:260] T 7ea7a5d8a92b4ac0903483627258ae66 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.624017 1174 ts_tablet_manager.cc:1434] T 7ea7a5d8a92b4ac0903483627258ae66 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:52.626510 1174 tablet_bootstrap.cc:492] T 8b0368e03f644f1db28dbb57c95c48c1 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:52.626507 1882 ts_tablet_manager.cc:1929] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.4
I20260202 02:52:52.626741 1882 log.cc:1199] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/28a900bcaa8b41718a285e595886d805
I20260202 02:52:52.627204 1882 ts_tablet_manager.cc:1950] T 28a900bcaa8b41718a285e595886d805 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:52.627986 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 764) took 3546 ms. Trace:
I20260202 02:52:52.628540 1882 rpcz_store.cc:276] 0202 02:52:49.081820 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:49.081898 (+ 78us) service_pool.cc:225] Handling call
0202 02:52:52.460488 (+3378590us) ts_tablet_manager.cc:1151] Deleting tablet 28a900bcaa8b41718a285e595886d805
0202 02:52:52.460505 (+ 17us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:52.519520 (+ 59015us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:52.626390 (+106870us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:52.627967 (+ 1577us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3378350}
I20260202 02:52:52.629081 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 28a900bcaa8b41718a285e595886d805 (table default.Table_911bb209b4554ee8b9bc09edaf488d63 [id=cd07f9746ec94d9ebe956a7ca0d19b1c]) successfully deleted
I20260202 02:52:52.634999 1174 tablet_bootstrap.cc:654] T 8b0368e03f644f1db28dbb57c95c48c1 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:52.636358 520 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 64e2fcb966fc4a06afb65506be1711a2
I20260202 02:52:52.636457 2633 consensus_queue.cc:1048] T 1f379a29e2454f689d68f5951837512b P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:52.636940 520 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:47430 (request call id 1015) took 1256 ms. Trace:
I20260202 02:52:52.637061 520 rpcz_store.cc:276] 0202 02:52:51.380506 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.064237 (+683731us) service_pool.cc:225] Handling call
0202 02:52:52.163682 (+ 99445us) spinlock_profiling.cc:243] Waited 98.8 ms on lock 0x616000041254. stack: 00007fc023f2da86 00007fc023f2d840 00007fc022f083ad 0000000000984f94 0000000000984f5d 00007fc0332576bd 00007fc0332577b3 00007fc02fe1e521 00007fc037ae96fd 00007fc037c76d86 00007fc02e776486 00007fc02e7761b5 00007fc02aef055e 00007fc02aeef43e 00007fc02aef5675 00007fc02aef5f9f
0202 02:52:52.164109 (+ 427us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0202 02:52:52.164277 (+ 168us) catalog_manager.cc:2815] Modifying in-memory table state
0202 02:52:52.164375 (+ 98us) catalog_manager.cc:2824] Locking tablets
0202 02:52:52.608027 (+443652us) catalog_manager.cc:2838] Removing table and tablets from system table
0202 02:52:52.636143 (+ 28116us) catalog_manager.cc:2856] Removing table from by-name map
0202 02:52:52.636236 (+ 93us) catalog_manager.cc:2872] Committing in-memory state
0202 02:52:52.636275 (+ 39us) catalog_manager.cc:2876] Aborting table tasks
0202 02:52:52.636899 (+ 624us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0202 02:52:52.620866 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0202 02:52:52.620992 (+ 126us) write_op.cc:432] Acquiring schema lock in shared mode
0202 02:52:52.620999 (+ 7us) write_op.cc:435] Acquired schema lock
0202 02:52:52.621002 (+ 3us) tablet.cc:660] Decoding operations
0202 02:52:52.621124 (+ 122us) write_op.cc:620] Acquiring the partition lock for write op
0202 02:52:52.621138 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0202 02:52:52.621143 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
0202 02:52:52.621252 (+ 109us) tablet.cc:699] Row locks acquired
0202 02:52:52.621255 (+ 3us) write_op.cc:260] PREPARE: finished
0202 02:52:52.621299 (+ 44us) write_op.cc:270] Start()
0202 02:52:52.621338 (+ 39us) write_op.cc:276] Timestamp: P: 1770000772621243 usec, L: 0
0202 02:52:52.621342 (+ 4us) op_driver.cc:348] REPLICATION: starting
0202 02:52:52.621547 (+ 205us) log.cc:844] Serialized 731 byte log entry
0202 02:52:52.635516 (+ 13969us) op_driver.cc:464] REPLICATION: finished
0202 02:52:52.635596 (+ 80us) write_op.cc:301] APPLY: starting
0202 02:52:52.635614 (+ 18us) tablet.cc:1365] starting BulkCheckPresence
0202 02:52:52.635666 (+ 52us) tablet.cc:1368] finished BulkCheckPresence
0202 02:52:52.635695 (+ 29us) tablet.cc:1370] starting ApplyRowOperation cycle
0202 02:52:52.635770 (+ 75us) tablet.cc:1381] finished ApplyRowOperation cycle
0202 02:52:52.635816 (+ 46us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0202 02:52:52.635824 (+ 8us) write_op.cc:312] APPLY: finished
0202 02:52:52.635915 (+ 91us) log.cc:844] Serialized 29 byte log entry
0202 02:52:52.635993 (+ 78us) write_op.cc:489] Releasing partition, row and schema locks
0202 02:52:52.636012 (+ 19us) write_op.cc:454] Released schema lock
0202 02:52:52.636019 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":227288448,"child_traces":[["op",{"apply.queue_time_us":30,"apply.run_cpu_time_us":481,"apply.run_wall_time_us":481,"num_ops":2,"prepare.queue_time_us":12073,"prepare.run_cpu_time_us":802,"prepare.run_wall_time_us":803,"raft.queue_time_us":1008,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":264,"replication_time_us":14137}]]}
I20260202 02:52:52.639489 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 64e2fcb966fc4a06afb65506be1711a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.639863 1173 tablet_bootstrap.cc:492] T b427c7e897c64977b924e7d586420e69 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:52.640089 1173 ts_tablet_manager.cc:1403] T b427c7e897c64977b924e7d586420e69 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.024s user 0.006s sys 0.000s
I20260202 02:52:52.640790 1173 raft_consensus.cc:359] T b427c7e897c64977b924e7d586420e69 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.640950 1173 raft_consensus.cc:385] T b427c7e897c64977b924e7d586420e69 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:52.641001 1173 raft_consensus.cc:740] T b427c7e897c64977b924e7d586420e69 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:52.641191 1173 consensus_queue.cc:260] T b427c7e897c64977b924e7d586420e69 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.641647 1173 ts_tablet_manager.cc:1434] T b427c7e897c64977b924e7d586420e69 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260202 02:52:52.641933 1173 tablet_bootstrap.cc:492] T 99e0b5e9a9874aec886155e0d614dec2 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:52.643244 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_bdbef281942540c7bf6d06c6c66b2acb" } new_table_name: "default.Table_44e1b09767d14dc899a9d956b5db9b02" modify_external_catalogs: true
I20260202 02:52:52.644970 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 64e2fcb966fc4a06afb65506be1711a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.643914 1173 tablet_bootstrap.cc:654] T 99e0b5e9a9874aec886155e0d614dec2 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:52.647250 528 catalog_manager.cc:5654] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 reported cstate change: term changed from 1 to 2, leader changed from b1f05820b6034228bdad884cc348716d (127.0.104.131) to 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130). New cstate: current_term: 2 leader_uuid: "1521a6d262b7451bbef8debe970dd6b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: HEALTHY } } }
I20260202 02:52:52.648522 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_b64e770cf31548baa0b24b084ea9f939"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:52.649117 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b64e770cf31548baa0b24b084ea9f939 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:52.649380 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 99e0b5e9a9874aec886155e0d614dec2
I20260202 02:52:52.650239 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 99e0b5e9a9874aec886155e0d614dec2 with delete_type TABLET_DATA_DELETED (Replaced by d677b534cba94354b6d0379a4efc4432 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.650507 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 99e0b5e9a9874aec886155e0d614dec2 with delete_type TABLET_DATA_DELETED (Replaced by d677b534cba94354b6d0379a4efc4432 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.650797 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet f3eebefa74584c1791d52377f5ba1290
I20260202 02:52:52.651461 884 tablet_service.cc:1552] Processing DeleteTablet for tablet f3eebefa74584c1791d52377f5ba1290 with delete_type TABLET_DATA_DELETED (Replaced by 1f171e07223b4895873a513ee05c8f21 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.651796 751 tablet_service.cc:1552] Processing DeleteTablet for tablet f3eebefa74584c1791d52377f5ba1290 with delete_type TABLET_DATA_DELETED (Replaced by 1f171e07223b4895873a513ee05c8f21 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.653172 1180 ts_tablet_manager.cc:1929] T fbcb4a522df648ba9542250450c08f1b P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:52.653931 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 8153510d5d154c7895b1ee794585612e
I20260202 02:52:52.653693 2633 consensus_queue.cc:1048] T 2c31b45dc8004eae993e972e9faa3ff8 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:52.654518 1174 tablet_bootstrap.cc:492] T 8b0368e03f644f1db28dbb57c95c48c1 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:52.655364 1174 ts_tablet_manager.cc:1403] T 8b0368e03f644f1db28dbb57c95c48c1 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.002s
I20260202 02:52:52.656139 1174 raft_consensus.cc:359] T 8b0368e03f644f1db28dbb57c95c48c1 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.656400 1174 raft_consensus.cc:385] T 8b0368e03f644f1db28dbb57c95c48c1 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:52.656400 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 64e2fcb966fc4a06afb65506be1711a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.656513 1174 raft_consensus.cc:740] T 8b0368e03f644f1db28dbb57c95c48c1 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:52.656562 1173 tablet_bootstrap.cc:492] T 99e0b5e9a9874aec886155e0d614dec2 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:52.656675 1173 ts_tablet_manager.cc:1403] T 99e0b5e9a9874aec886155e0d614dec2 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.001s
I20260202 02:52:52.656798 1174 consensus_queue.cc:260] T 8b0368e03f644f1db28dbb57c95c48c1 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.657938 1174 ts_tablet_manager.cc:1434] T 8b0368e03f644f1db28dbb57c95c48c1 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
W20260202 02:52:52.657914 1026 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:49890: Illegal state: replica b1f05820b6034228bdad884cc348716d is not leader of this config: current role FOLLOWER
I20260202 02:52:52.658073 1180 log.cc:1199] T fbcb4a522df648ba9542250450c08f1b P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/fbcb4a522df648ba9542250450c08f1b
I20260202 02:52:52.658648 1174 tablet_bootstrap.cc:492] T 8153510d5d154c7895b1ee794585612e P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:52.659559 1173 raft_consensus.cc:359] T 99e0b5e9a9874aec886155e0d614dec2 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.659744 1173 raft_consensus.cc:385] T 99e0b5e9a9874aec886155e0d614dec2 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:52.659816 1173 raft_consensus.cc:740] T 99e0b5e9a9874aec886155e0d614dec2 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:52.660028 1173 consensus_queue.cc:260] T 99e0b5e9a9874aec886155e0d614dec2 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.660679 1173 ts_tablet_manager.cc:1434] T 99e0b5e9a9874aec886155e0d614dec2 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260202 02:52:52.661463 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 99e0b5e9a9874aec886155e0d614dec2 with delete_type TABLET_DATA_DELETED (Replaced by d677b534cba94354b6d0379a4efc4432 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.661542 1173 tablet_bootstrap.cc:492] T 07580ad1d049475a9645acfad0b1d219 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:52.662781 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet f3eebefa74584c1791d52377f5ba1290 with delete_type TABLET_DATA_DELETED (Replaced by 1f171e07223b4895873a513ee05c8f21 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.663095 1180 ts_tablet_manager.cc:1950] T fbcb4a522df648ba9542250450c08f1b P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
W20260202 02:52:52.663405 515 catalog_manager.cc:5064] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): alter failed for tablet f6746ccbf37e477196a9706396d1e3fd (table default.Table_4f72bc6302d84055ba28584ec5e3ace9 [id=0c8385b71ceb44199472e1ce640b54e0]): Illegal state: replica b1f05820b6034228bdad884cc348716d is not leader of this config: current role FOLLOWER
I20260202 02:52:52.663626 1174 tablet_bootstrap.cc:654] T 8153510d5d154c7895b1ee794585612e P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:52.665207 526 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 898778f8f0b548ec89cb1417d6f07926
I20260202 02:52:52.665659 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 722) took 16890 ms. Trace:
I20260202 02:52:52.665741 1180 rpcz_store.cc:276] 0202 02:52:35.775346 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:35.776018 (+ 672us) service_pool.cc:225] Handling call
0202 02:52:52.604805 (+16828787us) ts_tablet_manager.cc:1151] Deleting tablet fbcb4a522df648ba9542250450c08f1b
0202 02:52:52.604816 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:52.629887 (+ 25071us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:52.653045 (+ 23158us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:52.665637 (+ 12592us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":16828676}
I20260202 02:52:52.666139 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 898778f8f0b548ec89cb1417d6f07926 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.666370 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet fbcb4a522df648ba9542250450c08f1b (table default.Table_d01f7dfd99d0405a896f6fbfd581dfd2 [id=163eefc3d38a4b28bce88b948cc5df3c]) successfully deleted
I20260202 02:52:52.667361 1173 tablet_bootstrap.cc:654] T 07580ad1d049475a9645acfad0b1d219 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:52.666121 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 723) took 16890 ms. Trace:
I20260202 02:52:52.671114 521 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet af29c7a2ad9d4797b08052606151a7d5
I20260202 02:52:52.671065 1180 rpcz_store.cc:276] 0202 02:52:35.775420 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:35.776127 (+ 707us) service_pool.cc:225] Handling call
0202 02:52:52.666000 (+16889873us) ts_tablet_manager.cc:1151] Deleting tablet f1fadf2cb5be4ec8980871fc96ee9730
0202 02:52:52.666009 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:52.666105 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":16889772}
I20260202 02:52:52.671677 521 catalog_manager.cc:6724] ReplaceTablet: tablet af29c7a2ad9d4797b08052606151a7d5 deleted and replaced by tablet c8606fb177db49c7b2e2b16b18d4725c
W20260202 02:52:52.671799 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet f1fadf2cb5be4ec8980871fc96ee9730 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f1fadf2cb5be4ec8980871fc96ee9730 already in progress: creating tablet
I20260202 02:52:52.672261 751 tablet_service.cc:1552] Processing DeleteTablet for tablet af29c7a2ad9d4797b08052606151a7d5 with delete_type TABLET_DATA_DELETED (replaced by tablet c8606fb177db49c7b2e2b16b18d4725c) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.672794 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 898778f8f0b548ec89cb1417d6f07926 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.673009 896 tablet_service.cc:1552] Processing DeleteTablet for tablet af29c7a2ad9d4797b08052606151a7d5 with delete_type TABLET_DATA_DELETED (replaced by tablet c8606fb177db49c7b2e2b16b18d4725c) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.673061 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 724) took 4732 ms. Trace:
I20260202 02:52:52.673132 1180 rpcz_store.cc:276] 0202 02:52:47.940928 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:47.952253 (+ 11325us) service_pool.cc:225] Handling call
0202 02:52:52.672916 (+4720663us) ts_tablet_manager.cc:1151] Deleting tablet 3dcf4f794ec14b2f9d181d015fc437fb
0202 02:52:52.672929 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:52.673041 (+ 112us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4620873}
I20260202 02:52:52.673321 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_7b218d827e024dee86865d1c48ab6cf9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:52.673589 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 3dcf4f794ec14b2f9d181d015fc437fb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3dcf4f794ec14b2f9d181d015fc437fb already in progress: creating tablet
W20260202 02:52:52.673952 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7b218d827e024dee86865d1c48ab6cf9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:52.674060 1173 tablet_bootstrap.cc:492] T 07580ad1d049475a9645acfad0b1d219 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:52.674194 1173 ts_tablet_manager.cc:1403] T 07580ad1d049475a9645acfad0b1d219 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.002s
I20260202 02:52:52.674283 1175 tablet_bootstrap.cc:492] T f5727f9b9bbb4e77abfef065c51606a1 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:52.674400 1175 ts_tablet_manager.cc:1403] T f5727f9b9bbb4e77abfef065c51606a1 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.212s user 0.002s sys 0.000s
I20260202 02:52:52.674878 1173 raft_consensus.cc:359] T 07580ad1d049475a9645acfad0b1d219 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.675056 1173 raft_consensus.cc:385] T 07580ad1d049475a9645acfad0b1d219 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:52.674997 1175 raft_consensus.cc:359] T f5727f9b9bbb4e77abfef065c51606a1 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:52.675112 1173 raft_consensus.cc:740] T 07580ad1d049475a9645acfad0b1d219 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:52.675415 1882 tablet_replica.cc:333] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:52.675568 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_b5055a016bd5483980012ad46c8c68c9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:52.676081 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b5055a016bd5483980012ad46c8c68c9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:52.676592 524 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet b00b4a0b85374bfe83928162c4da6479
I20260202 02:52:52.677150 524 catalog_manager.cc:6724] ReplaceTablet: tablet b00b4a0b85374bfe83928162c4da6479 deleted and replaced by tablet 4e348f6a09cf4e40b7debeafcbae83ee
I20260202 02:52:52.677522 524 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_874748aec6a14c11b3787312a348ef91" } new_table_name: "default.Table_e699c7601f354d13907893985186dc4d" modify_external_catalogs: true
I20260202 02:52:52.677601 751 tablet_service.cc:1552] Processing DeleteTablet for tablet b00b4a0b85374bfe83928162c4da6479 with delete_type TABLET_DATA_DELETED (replaced by tablet 4e348f6a09cf4e40b7debeafcbae83ee) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.675343 1173 consensus_queue.cc:260] T 07580ad1d049475a9645acfad0b1d219 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.677973 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 898778f8f0b548ec89cb1417d6f07926 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.677762 896 tablet_service.cc:1552] Processing DeleteTablet for tablet b00b4a0b85374bfe83928162c4da6479 with delete_type TABLET_DATA_DELETED (replaced by tablet 4e348f6a09cf4e40b7debeafcbae83ee) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.679315 1173 ts_tablet_manager.cc:1434] T 07580ad1d049475a9645acfad0b1d219 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260202 02:52:52.679615 1173 tablet_bootstrap.cc:492] T 0138ba7326674741b0729324735f1de7 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:52.680088 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet af29c7a2ad9d4797b08052606151a7d5 with delete_type TABLET_DATA_DELETED (replaced by tablet c8606fb177db49c7b2e2b16b18d4725c) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.680159 1882 raft_consensus.cc:2243] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:52.680389 1882 raft_consensus.cc:2272] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:52.681186 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet b00b4a0b85374bfe83928162c4da6479 with delete_type TABLET_DATA_DELETED (replaced by tablet 4e348f6a09cf4e40b7debeafcbae83ee) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.679515 1175 raft_consensus.cc:385] T f5727f9b9bbb4e77abfef065c51606a1 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:52.681303 1173 tablet_bootstrap.cc:654] T 0138ba7326674741b0729324735f1de7 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:52.681353 1175 raft_consensus.cc:740] T f5727f9b9bbb4e77abfef065c51606a1 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:52.681865 1180 tablet_replica.cc:333] T 75a24a1bf54d4c799938a5723ac6fa99 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:52.682209 1175 consensus_queue.cc:260] T f5727f9b9bbb4e77abfef065c51606a1 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:52.686396 2794 raft_consensus.cc:493] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
I20260202 02:52:52.686887 2794 raft_consensus.cc:515] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
W20260202 02:52:52.690414 999 leader_election.cc:343] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 8bb14bf551f441dcbaaff7057b51b2be
I20260202 02:52:52.690717 2794 leader_election.cc:290] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:52.691067 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8bb14bf551f441dcbaaff7057b51b2be" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:52.695428 998 leader_election.cc:304] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:52.696133 2794 raft_consensus.cc:2749] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260202 02:52:52.699433 1180 raft_consensus.cc:2243] T 75a24a1bf54d4c799938a5723ac6fa99 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:52.701273 1882 ts_tablet_manager.cc:1916] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:52.704083 896 tablet_service.cc:1552] Processing DeleteTablet for tablet 8153510d5d154c7895b1ee794585612e with delete_type TABLET_DATA_DELETED (Replaced by 9f805fa060d54d7e9067b5120f3d28ad at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.706259 523 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet 939fbcaf75c7439d9208bbbf83a72717 (table default.Table_12d68a317c8e4d809dbbe77c095a1f6e [id=7219522ffc3849faab94d754a42a8511]). Expected version 1 got 0
I20260202 02:52:52.706480 523 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet 44c8d5b74c8e4c499f7f68e8185efc17 (table default.Table_f48ed1c7e02349adad7ce22c0e51ed81 [id=46b5d47377474ff9bac2acdb26b08c65]). Expected version 3 got 0
I20260202 02:52:52.706676 523 catalog_manager.cc:5654] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "1f442ef85029461091de847118c34c0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } }
I20260202 02:52:52.709988 884 tablet_service.cc:1552] Processing DeleteTablet for tablet eea6bb04ddf24276b46270d009ed78b4 with delete_type TABLET_DATA_DELETED (Replaced by 54591d3d809648de930f7b4221c94e49 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.710441 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 7709ea34429e4e91ad4922a3d73ebd14 with delete_type TABLET_DATA_DELETED (Replaced by 22c31519d7e642118ff1218e2a1b7621 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.710371 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_4d795c06694e40cf9bf6b637784d3ea4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:52.710469 890 tablet_service.cc:1552] Processing DeleteTablet for tablet dc90eeb79ded4739883ad98928476680 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.710608 896 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a3dac5cd1ce47488a50b4c32eca67c7 with delete_type TABLET_DATA_DELETED (replaced by tablet 365417e050304073b5de887082fc699b) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.711048 1180 raft_consensus.cc:2272] T 75a24a1bf54d4c799938a5723ac6fa99 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:52.711077 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4d795c06694e40cf9bf6b637784d3ea4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:52.711645 1180 ts_tablet_manager.cc:1916] T 75a24a1bf54d4c799938a5723ac6fa99 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:52.711838 2633 raft_consensus.cc:493] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:52.711982 2633 raft_consensus.cc:515] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:52.712695 2633 leader_election.cc:290] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:52.713513 915 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e1284fde67bc4d65b067a54c416ccc33" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:52.713781 1182 ts_tablet_manager.cc:1929] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:52.713879 1182 log.cc:1199] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/ee04563d3b8e4cb2b989bb8d6ee9b20e
I20260202 02:52:52.714010 2633 raft_consensus.cc:493] T 8bb14bf551f441dcbaaff7057b51b2be P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
I20260202 02:52:52.714154 1182 ts_tablet_manager.cc:1950] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:52.714133 2633 raft_consensus.cc:515] T 8bb14bf551f441dcbaaff7057b51b2be P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:52.714787 2633 leader_election.cc:290] T 8bb14bf551f441dcbaaff7057b51b2be P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:52.716521 714 leader_election.cc:343] T 8bb14bf551f441dcbaaff7057b51b2be P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 8bb14bf551f441dcbaaff7057b51b2be
I20260202 02:52:52.718366 1175 ts_tablet_manager.cc:1434] T f5727f9b9bbb4e77abfef065c51606a1 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.044s user 0.001s sys 0.001s
I20260202 02:52:52.718910 1175 tablet_bootstrap.cc:492] T 9a1d7d173c41462c8e32a95b494bb708 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:52.721118 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 8153510d5d154c7895b1ee794585612e with delete_type TABLET_DATA_DELETED (Replaced by 9f805fa060d54d7e9067b5120f3d28ad at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.722057 1174 tablet_bootstrap.cc:492] T 8153510d5d154c7895b1ee794585612e P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:52.722222 1174 ts_tablet_manager.cc:1403] T 8153510d5d154c7895b1ee794585612e P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.064s user 0.002s sys 0.001s
I20260202 02:52:52.722991 1174 raft_consensus.cc:359] T 8153510d5d154c7895b1ee794585612e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.723167 1174 raft_consensus.cc:385] T 8153510d5d154c7895b1ee794585612e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:52.723215 1174 raft_consensus.cc:740] T 8153510d5d154c7895b1ee794585612e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:52.723456 1174 consensus_queue.cc:260] T 8153510d5d154c7895b1ee794585612e P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
W20260202 02:52:52.723780 714 consensus_peers.cc:597] T 2c31b45dc8004eae993e972e9faa3ff8 P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2c31b45dc8004eae993e972e9faa3ff8. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:52.724099 1174 ts_tablet_manager.cc:1434] T 8153510d5d154c7895b1ee794585612e P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:52.724563 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet c8a273ac8cd344719a48fd81945bed0a
I20260202 02:52:52.725549 1174 tablet_bootstrap.cc:492] T 51ba0bcd498043c383c493f45ad1fe17 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:52.727233 1182 tablet_replica.cc:333] T 99e0b5e9a9874aec886155e0d614dec2 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:52.705124 2794 consensus_queue.cc:1048] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:52.720204 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8bb14bf551f441dcbaaff7057b51b2be" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:52.727396 1060 raft_consensus.cc:2468] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:52.727829 715 leader_election.cc:304] T 8bb14bf551f441dcbaaff7057b51b2be P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:52.728364 2881 raft_consensus.cc:2804] T 8bb14bf551f441dcbaaff7057b51b2be P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260202 02:52:52.728446 2881 raft_consensus.cc:493] T 8bb14bf551f441dcbaaff7057b51b2be P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
I20260202 02:52:52.728525 2881 raft_consensus.cc:3060] T 8bb14bf551f441dcbaaff7057b51b2be P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:52.729959 1182 raft_consensus.cc:2243] T 99e0b5e9a9874aec886155e0d614dec2 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:52.730214 1182 raft_consensus.cc:2272] T 99e0b5e9a9874aec886155e0d614dec2 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:52.731099 2881 raft_consensus.cc:515] T 8bb14bf551f441dcbaaff7057b51b2be P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:52.731320 1173 tablet_bootstrap.cc:492] T 0138ba7326674741b0729324735f1de7 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:52.720050 1059 raft_consensus.cc:3060] T 1d8143f529b74e6ab779dd695920413c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:52.724972 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e1284fde67bc4d65b067a54c416ccc33" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:52.731455 1173 ts_tablet_manager.cc:1403] T 0138ba7326674741b0729324735f1de7 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.052s user 0.021s sys 0.021s
I20260202 02:52:52.731542 1046 raft_consensus.cc:2468] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 2.
I20260202 02:52:52.731741 2881 leader_election.cc:290] T 8bb14bf551f441dcbaaff7057b51b2be P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:52.731957 2881 raft_consensus.cc:493] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
I20260202 02:52:52.732059 2881 raft_consensus.cc:515] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:52.732136 2878 consensus_queue.cc:1048] T 1d8143f529b74e6ab779dd695920413c P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:52.732553 2881 leader_election.cc:290] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:52.733023 1173 raft_consensus.cc:359] T 0138ba7326674741b0729324735f1de7 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.733178 1173 raft_consensus.cc:385] T 0138ba7326674741b0729324735f1de7 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:52.733222 1173 raft_consensus.cc:740] T 0138ba7326674741b0729324735f1de7 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
W20260202 02:52:52.733419 714 leader_election.cc:343] T 8bb14bf551f441dcbaaff7057b51b2be P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 8bb14bf551f441dcbaaff7057b51b2be
I20260202 02:52:52.733632 1173 consensus_queue.cc:260] T 0138ba7326674741b0729324735f1de7 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.734254 1173 ts_tablet_manager.cc:1434] T 0138ba7326674741b0729324735f1de7 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260202 02:52:52.734534 1173 tablet_bootstrap.cc:492] T c8a273ac8cd344719a48fd81945bed0a P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
W20260202 02:52:52.734520 714 leader_election.cc:343] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: d2d27ceea1bb4122b18f0fcf4d2d1bf4
I20260202 02:52:52.734807 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 8153510d5d154c7895b1ee794585612e with delete_type TABLET_DATA_DELETED (Replaced by 9f805fa060d54d7e9067b5120f3d28ad at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.735023 715 leader_election.cc:304] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:52.735425 2881 raft_consensus.cc:2804] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260202 02:52:52.735491 2881 raft_consensus.cc:493] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:52.735546 2881 raft_consensus.cc:3060] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Advancing to term 3
I20260202 02:52:52.736109 1173 tablet_bootstrap.cc:654] T c8a273ac8cd344719a48fd81945bed0a P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:52.738070 2881 raft_consensus.cc:515] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:52.738955 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8bb14bf551f441dcbaaff7057b51b2be" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:52.739143 1046 raft_consensus.cc:3060] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:52.741782 1046 raft_consensus.cc:2468] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 2.
I20260202 02:52:52.743384 715 leader_election.cc:304] T 8bb14bf551f441dcbaaff7057b51b2be P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:52.743937 2878 raft_consensus.cc:2804] T 8bb14bf551f441dcbaaff7057b51b2be P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Leader election won for term 2
I20260202 02:52:52.743995 1173 tablet_bootstrap.cc:492] T c8a273ac8cd344719a48fd81945bed0a P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:52.744063 2878 raft_consensus.cc:697] T 8bb14bf551f441dcbaaff7057b51b2be P 1f442ef85029461091de847118c34c0d [term 2 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:52.744112 1173 ts_tablet_manager.cc:1403] T c8a273ac8cd344719a48fd81945bed0a P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260202 02:52:52.744280 2878 consensus_queue.cc:237] T 8bb14bf551f441dcbaaff7057b51b2be P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:52.744639 2881 leader_election.cc:290] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 3 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:52.744675 919 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e1284fde67bc4d65b067a54c416ccc33" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
W20260202 02:52:52.745357 715 leader_election.cc:343] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Not found: Tablet not found: d2d27ceea1bb4122b18f0fcf4d2d1bf4
I20260202 02:52:52.745445 1182 ts_tablet_manager.cc:1916] T 99e0b5e9a9874aec886155e0d614dec2 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:52.745481 715 leader_election.cc:304] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:52.745395 1173 raft_consensus.cc:359] T c8a273ac8cd344719a48fd81945bed0a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.745678 2944 raft_consensus.cc:2749] T d2d27ceea1bb4122b18f0fcf4d2d1bf4 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260202 02:52:52.745783 1173 raft_consensus.cc:385] T c8a273ac8cd344719a48fd81945bed0a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:52.745885 1173 raft_consensus.cc:740] T c8a273ac8cd344719a48fd81945bed0a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:52.746202 1173 consensus_queue.cc:260] T c8a273ac8cd344719a48fd81945bed0a P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.746469 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e1284fde67bc4d65b067a54c416ccc33" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:52.746628 1046 raft_consensus.cc:3060] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Advancing to term 3
I20260202 02:52:52.746953 1173 ts_tablet_manager.cc:1434] T c8a273ac8cd344719a48fd81945bed0a P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260202 02:52:52.747252 1173 tablet_bootstrap.cc:492] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:52.749643 1173 tablet_bootstrap.cc:654] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:52.752187 1046 raft_consensus.cc:2468] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 3.
I20260202 02:52:52.752948 715 leader_election.cc:304] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:52.752911 2944 consensus_queue.cc:1048] T 1d8143f529b74e6ab779dd695920413c P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:52.753192 2881 raft_consensus.cc:2804] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d [term 3 FOLLOWER]: Leader election won for term 3
I20260202 02:52:52.753397 2881 raft_consensus.cc:697] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d [term 3 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:52.753684 2881 consensus_queue.cc:237] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:52.760493 1173 tablet_bootstrap.cc:492] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:52.760659 1173 ts_tablet_manager.cc:1403] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.006s
I20260202 02:52:52.761387 1173 raft_consensus.cc:359] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.761636 1173 raft_consensus.cc:385] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:52.761752 1173 raft_consensus.cc:740] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:52.761992 1173 consensus_queue.cc:260] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.762611 1173 ts_tablet_manager.cc:1434] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:52.763985 2964 raft_consensus.cc:493] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b1f05820b6034228bdad884cc348716d)
I20260202 02:52:52.764156 2964 raft_consensus.cc:515] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:52.764853 2964 leader_election.cc:290] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:52.764978 2964 raft_consensus.cc:493] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b1f05820b6034228bdad884cc348716d)
I20260202 02:52:52.765084 2964 raft_consensus.cc:515] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:52.765257 764 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "20ea39bc5d3d46f5a4ba427277d6c6a3" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:52.765465 764 raft_consensus.cc:2393] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1521a6d262b7451bbef8debe970dd6b5 in current term 2: Already voted for candidate 1f442ef85029461091de847118c34c0d in this term.
I20260202 02:52:52.765563 2964 leader_election.cc:290] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:52.765877 1174 tablet_bootstrap.cc:654] T 51ba0bcd498043c383c493f45ad1fe17 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:52.766505 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "20ea39bc5d3d46f5a4ba427277d6c6a3" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:52.766695 1046 raft_consensus.cc:2468] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 2.
I20260202 02:52:52.767045 859 leader_election.cc:304] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:52.767294 2929 raft_consensus.cc:2804] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260202 02:52:52.767364 2929 raft_consensus.cc:493] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b1f05820b6034228bdad884cc348716d)
I20260202 02:52:52.767453 2929 raft_consensus.cc:3060] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:52.767653 764 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e1284fde67bc4d65b067a54c416ccc33" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:52.767995 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e1284fde67bc4d65b067a54c416ccc33" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:52.768172 1060 raft_consensus.cc:2368] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1521a6d262b7451bbef8debe970dd6b5 for earlier term 2. Current term is 3.
I20260202 02:52:52.770059 2929 raft_consensus.cc:515] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:52.770720 2929 leader_election.cc:290] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:52.771253 764 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "20ea39bc5d3d46f5a4ba427277d6c6a3" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:52.771472 764 raft_consensus.cc:2393] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1521a6d262b7451bbef8debe970dd6b5 in current term 2: Already voted for candidate 1f442ef85029461091de847118c34c0d in this term.
I20260202 02:52:52.772289 859 leader_election.cc:400] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Vote denied by peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) with higher term. Message: Invalid argument: T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1521a6d262b7451bbef8debe970dd6b5 for earlier term 2. Current term is 3.
I20260202 02:52:52.772446 859 leader_election.cc:403] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260202 02:52:52.772655 2929 raft_consensus.cc:3060] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Advancing to term 3
I20260202 02:52:52.775346 2929 raft_consensus.cc:2749] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) with higher term. Message: Invalid argument: T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1521a6d262b7451bbef8debe970dd6b5 for earlier term 2. Current term is 3.
I20260202 02:52:52.775579 1174 tablet_bootstrap.cc:492] T 51ba0bcd498043c383c493f45ad1fe17 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:52.775797 1174 ts_tablet_manager.cc:1403] T 51ba0bcd498043c383c493f45ad1fe17 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.050s user 0.002s sys 0.002s
I20260202 02:52:52.777096 1174 raft_consensus.cc:359] T 51ba0bcd498043c383c493f45ad1fe17 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.777283 1174 raft_consensus.cc:385] T 51ba0bcd498043c383c493f45ad1fe17 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:52.777343 1174 raft_consensus.cc:740] T 51ba0bcd498043c383c493f45ad1fe17 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:52.777555 1174 consensus_queue.cc:260] T 51ba0bcd498043c383c493f45ad1fe17 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.777801 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "20ea39bc5d3d46f5a4ba427277d6c6a3" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:52.778028 1174 ts_tablet_manager.cc:1434] T 51ba0bcd498043c383c493f45ad1fe17 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260202 02:52:52.778286 1174 tablet_bootstrap.cc:492] T b427c7e897c64977b924e7d586420e69 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:52.779558 1174 tablet_bootstrap.cc:654] T b427c7e897c64977b924e7d586420e69 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:52.780213 1182 ts_tablet_manager.cc:1929] T 99e0b5e9a9874aec886155e0d614dec2 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:52.780817 1060 raft_consensus.cc:2468] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 2.
I20260202 02:52:52.781241 859 leader_election.cc:304] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:52.787767 745 tablet_service.cc:1552] Processing DeleteTablet for tablet c8a273ac8cd344719a48fd81945bed0a with delete_type TABLET_DATA_DELETED (Replaced by ae9a02dd8c3647f19abe28647ee2b3ce at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.787902 2929 raft_consensus.cc:2804] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Leader election won for term 2
I20260202 02:52:52.787971 755 tablet_service.cc:1552] Processing DeleteTablet for tablet 79badde1001b432b95f58b5161ace15c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.788012 2929 raft_consensus.cc:697] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [term 2 LEADER]: Becoming Leader. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:52.788141 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 1d8143f529b74e6ab779dd695920413c with delete_type TABLET_DATA_DELETED (Replaced by 84d68594516e4be4a3a1f4ebe9e217a5 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.788239 2929 consensus_queue.cc:237] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:52.788877 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 7709ea34429e4e91ad4922a3d73ebd14 with delete_type TABLET_DATA_DELETED (Replaced by 22c31519d7e642118ff1218e2a1b7621 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.780314 1182 log.cc:1199] T 99e0b5e9a9874aec886155e0d614dec2 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/99e0b5e9a9874aec886155e0d614dec2
I20260202 02:52:52.792708 1182 ts_tablet_manager.cc:1950] T 99e0b5e9a9874aec886155e0d614dec2 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:52.794126 1182 tablet_replica.cc:333] T f3eebefa74584c1791d52377f5ba1290 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:52.795389 1182 raft_consensus.cc:2243] T f3eebefa74584c1791d52377f5ba1290 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:52.795842 1182 raft_consensus.cc:2272] T f3eebefa74584c1791d52377f5ba1290 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:52.796623 1182 ts_tablet_manager.cc:1916] T f3eebefa74584c1791d52377f5ba1290 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:52.809041 1046 raft_consensus.cc:1275] T 78f122342dd144b48de89902faa0f94f P b1f05820b6034228bdad884cc348716d [term 3 FOLLOWER]: Refusing update from remote peer 1f442ef85029461091de847118c34c0d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260202 02:52:52.809669 2911 consensus_queue.cc:1048] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260202 02:52:52.822786 1182 ts_tablet_manager.cc:1929] T f3eebefa74584c1791d52377f5ba1290 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:52.825327 1182 log.cc:1199] T f3eebefa74584c1791d52377f5ba1290 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/f3eebefa74584c1791d52377f5ba1290
I20260202 02:52:52.826432 1182 ts_tablet_manager.cc:1950] T f3eebefa74584c1791d52377f5ba1290 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:52.830937 2977 raft_consensus.cc:493] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
I20260202 02:52:52.831126 2977 raft_consensus.cc:515] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:52.831888 2977 leader_election.cc:290] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:52.726482 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet ee04563d3b8e4cb2b989bb8d6ee9b20e (table default.Table_62b7624afec14fb6a860d51a1a09913d [id=78feba5e47064271ac41a928580d1f02]) successfully deleted
W20260202 02:52:52.837440 514 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 110990us
I20260202 02:52:52.837904 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet f5727f9b9bbb4e77abfef065c51606a1
W20260202 02:52:52.838193 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 27158aa96e2742d5b0b2c3fa9531178d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 27158aa96e2742d5b0b2c3fa9531178d already in progress: creating tablet
W20260202 02:52:52.726946 527 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 353bbad7f24f40e3b6316f619c749017 consensus state has no leader
I20260202 02:52:52.838454 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet eea6bb04ddf24276b46270d009ed78b4
I20260202 02:52:52.838938 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_1a44a09f9b4f49a9a5f1f9f2529b5a52" } new_table_name: "default.Table_1e77177eeb844b038f4b891eb5463e1b" modify_external_catalogs: true
I20260202 02:52:52.839027 890 tablet_service.cc:1552] Processing DeleteTablet for tablet c8a273ac8cd344719a48fd81945bed0a with delete_type TABLET_DATA_DELETED (Replaced by ae9a02dd8c3647f19abe28647ee2b3ce at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.839125 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet b427c7e897c64977b924e7d586420e69
I20260202 02:52:52.839639 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 8b0368e03f644f1db28dbb57c95c48c1
I20260202 02:52:52.839973 751 tablet_service.cc:1552] Processing DeleteTablet for tablet c8a273ac8cd344719a48fd81945bed0a with delete_type TABLET_DATA_DELETED (Replaced by ae9a02dd8c3647f19abe28647ee2b3ce at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.840193 755 tablet_service.cc:1552] Processing DeleteTablet for tablet f5727f9b9bbb4e77abfef065c51606a1 with delete_type TABLET_DATA_DELETED (Replaced by fdc72096ee4d440789899e6e71580746 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.840567 745 tablet_service.cc:1552] Processing DeleteTablet for tablet eea6bb04ddf24276b46270d009ed78b4 with delete_type TABLET_DATA_DELETED (Replaced by 54591d3d809648de930f7b4221c94e49 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.840514 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "80eb3013f5354022a2dd78bb294a8497" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:52.840713 766 raft_consensus.cc:2468] T 80eb3013f5354022a2dd78bb294a8497 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:52.840756 751 tablet_service.cc:1552] Processing DeleteTablet for tablet b427c7e897c64977b924e7d586420e69 with delete_type TABLET_DATA_DELETED (Replaced by 71dd6890e0e948d09d102296d1bd6464 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.841089 998 leader_election.cc:304] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:52.841303 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 8b0368e03f644f1db28dbb57c95c48c1 with delete_type TABLET_DATA_DELETED (Replaced by d0e00bed25e740f5b39e023f9ed61d5a at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.841406 2977 raft_consensus.cc:2804] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260202 02:52:52.841475 2977 raft_consensus.cc:493] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1521a6d262b7451bbef8debe970dd6b5)
I20260202 02:52:52.841535 2977 raft_consensus.cc:3060] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:52.739334 525 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet fa57a267b0484262ac31039862f40a76
I20260202 02:52:52.844153 1173 tablet_bootstrap.cc:492] T 7709ea34429e4e91ad4922a3d73ebd14 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:52.844220 525 catalog_manager.cc:6724] ReplaceTablet: tablet fa57a267b0484262ac31039862f40a76 deleted and replaced by tablet 98ee9e01f0354a2abdbc1c45f1e6acdf
I20260202 02:52:52.844188 2977 raft_consensus.cc:515] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:52.844642 525 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_cdcc140c2b8d4aabae614d58873dc22a" } modify_external_catalogs: true
I20260202 02:52:52.844805 525 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_cdcc140c2b8d4aabae614d58873dc22a" } modify_external_catalogs: true
I20260202 02:52:52.844941 751 tablet_service.cc:1552] Processing DeleteTablet for tablet fa57a267b0484262ac31039862f40a76 with delete_type TABLET_DATA_DELETED (replaced by tablet 98ee9e01f0354a2abdbc1c45f1e6acdf) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.845574 2977 leader_election.cc:290] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:52.845901 1173 tablet_bootstrap.cc:654] T 7709ea34429e4e91ad4922a3d73ebd14 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:52.846179 764 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "80eb3013f5354022a2dd78bb294a8497" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:52.846349 764 raft_consensus.cc:3060] T 80eb3013f5354022a2dd78bb294a8497 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:52:52.848253 2881 raft_consensus.cc:493] T 0138ba7326674741b0729324735f1de7 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:52.848405 2881 raft_consensus.cc:515] T 0138ba7326674741b0729324735f1de7 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.848980 2881 leader_election.cc:290] T 0138ba7326674741b0729324735f1de7 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:52.739928 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_71bd027cef9447b4895e5363c9f0d75c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:52.849656 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_71bd027cef9447b4895e5363c9f0d75c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:52.777711 519 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet efe78242e0654a1c9bdbaf9cfa88b034
I20260202 02:52:52.851389 519 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_ae21476eb08546c99d1f05bafcdd5eca" } new_table_name: "default.Table_5a0e43e914a94184ad2440e9f890ca4b" modify_external_catalogs: true
I20260202 02:52:52.851477 751 tablet_service.cc:1552] Processing DeleteTablet for tablet efe78242e0654a1c9bdbaf9cfa88b034 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.778630 521 master_service.cc:954] ReplaceTablet: received request to replace tablet 3857680ccbdc4dd4ad8247da96254434 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:52.853878 1054 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0138ba7326674741b0729324735f1de7" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:52.854076 1054 raft_consensus.cc:2468] T 0138ba7326674741b0729324735f1de7 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:52.782159 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_8161c74bbc974802bd0baade2f1fb103"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:52.854763 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8161c74bbc974802bd0baade2f1fb103 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:52.854799 715 leader_election.cc:304] T 0138ba7326674741b0729324735f1de7 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:52.855235 2633 raft_consensus.cc:2804] T 0138ba7326674741b0729324735f1de7 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:52.855310 2633 raft_consensus.cc:493] T 0138ba7326674741b0729324735f1de7 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:52.855366 2633 raft_consensus.cc:3060] T 0138ba7326674741b0729324735f1de7 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:52.784294 524 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_fe61f7e6b8c74ca1acb5bdf456cc6bf5" } modify_external_catalogs: true
I20260202 02:52:52.855868 524 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_fe61f7e6b8c74ca1acb5bdf456cc6bf5" } modify_external_catalogs: true
I20260202 02:52:52.787987 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_66a25197aee2457cb4d2ace107cb7f01"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:52.857402 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_66a25197aee2457cb4d2ace107cb7f01 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:52.858050 2633 raft_consensus.cc:515] T 0138ba7326674741b0729324735f1de7 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.796053 520 catalog_manager.cc:5654] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "b1f05820b6034228bdad884cc348716d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } }
I20260202 02:52:52.858673 2633 leader_election.cc:290] T 0138ba7326674741b0729324735f1de7 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:52.858829 1175 tablet_bootstrap.cc:654] T 9a1d7d173c41462c8e32a95b494bb708 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:52.858834 520 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 939fbcaf75c7439d9208bbbf83a72717 (table default.Table_12d68a317c8e4d809dbbe77c095a1f6e [id=7219522ffc3849faab94d754a42a8511]). Expected version 1 got 0
I20260202 02:52:52.799405 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_5d59e3cf9d5a41afa503a76353543218"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:52.859814 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5d59e3cf9d5a41afa503a76353543218 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:52.860239 2977 raft_consensus.cc:493] T 75a24a1bf54d4c799938a5723ac6fa99 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:52.860390 2977 raft_consensus.cc:515] T 75a24a1bf54d4c799938a5723ac6fa99 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.861008 2977 leader_election.cc:290] T 75a24a1bf54d4c799938a5723ac6fa99 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:52.861537 1054 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0138ba7326674741b0729324735f1de7" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:52.861697 1054 raft_consensus.cc:3060] T 0138ba7326674741b0729324735f1de7 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:52.861914 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "75a24a1bf54d4c799938a5723ac6fa99" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:52.862493 998 leader_election.cc:343] T 75a24a1bf54d4c799938a5723ac6fa99 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: Tablet not RUNNING: STOPPED
W20260202 02:52:52.838462 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 41a4143247ed4eed98defbdfe2895372 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 41a4143247ed4eed98defbdfe2895372 already in progress: creating tablet
W20260202 02:52:52.864914 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 64e2fcb966fc4a06afb65506be1711a2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 64e2fcb966fc4a06afb65506be1711a2 already in progress: creating tablet
I20260202 02:52:52.865067 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 99e0b5e9a9874aec886155e0d614dec2 (table default.Table_a8c4f67a58b74bf8bb3716174ed5654f [id=e7975854ed2f4810a2d438b590267584]) successfully deleted
I20260202 02:52:52.865250 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet f3eebefa74584c1791d52377f5ba1290 (table default.Table_37b8a9e1666a4ddbaf5a13d828e9702c [id=e0cea4989f66443faf074eacf434658c]) successfully deleted
I20260202 02:52:52.865831 890 tablet_service.cc:1552] Processing DeleteTablet for tablet f5727f9b9bbb4e77abfef065c51606a1 with delete_type TABLET_DATA_DELETED (Replaced by fdc72096ee4d440789899e6e71580746 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.865885 884 tablet_service.cc:1552] Processing DeleteTablet for tablet eea6bb04ddf24276b46270d009ed78b4 with delete_type TABLET_DATA_DELETED (Replaced by 54591d3d809648de930f7b4221c94e49 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.866098 884 tablet_service.cc:1552] Processing DeleteTablet for tablet fa57a267b0484262ac31039862f40a76 with delete_type TABLET_DATA_DELETED (replaced by tablet 98ee9e01f0354a2abdbc1c45f1e6acdf) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.866128 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 8b0368e03f644f1db28dbb57c95c48c1 with delete_type TABLET_DATA_DELETED (Replaced by d0e00bed25e740f5b39e023f9ed61d5a at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.866230 890 tablet_service.cc:1552] Processing DeleteTablet for tablet efe78242e0654a1c9bdbaf9cfa88b034 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.866271 898 tablet_service.cc:1552] Processing DeleteTablet for tablet b427c7e897c64977b924e7d586420e69 with delete_type TABLET_DATA_DELETED (Replaced by 71dd6890e0e948d09d102296d1bd6464 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.867060 764 raft_consensus.cc:2468] T 80eb3013f5354022a2dd78bb294a8497 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 2.
I20260202 02:52:52.867556 998 leader_election.cc:304] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:52.867882 2977 raft_consensus.cc:2804] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Leader election won for term 2
I20260202 02:52:52.867997 2977 raft_consensus.cc:697] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d [term 2 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:52.868254 2977 consensus_queue.cc:237] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:52.869670 1180 ts_tablet_manager.cc:1929] T 75a24a1bf54d4c799938a5723ac6fa99 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:52.875135 1054 raft_consensus.cc:2468] T 0138ba7326674741b0729324735f1de7 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:52.875919 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet c8a273ac8cd344719a48fd81945bed0a with delete_type TABLET_DATA_DELETED (Replaced by ae9a02dd8c3647f19abe28647ee2b3ce at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.876546 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet f5727f9b9bbb4e77abfef065c51606a1 with delete_type TABLET_DATA_DELETED (Replaced by fdc72096ee4d440789899e6e71580746 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.877050 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet eea6bb04ddf24276b46270d009ed78b4 with delete_type TABLET_DATA_DELETED (Replaced by 54591d3d809648de930f7b4221c94e49 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.878057 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet b427c7e897c64977b924e7d586420e69 with delete_type TABLET_DATA_DELETED (Replaced by 71dd6890e0e948d09d102296d1bd6464 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.879901 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 7709ea34429e4e91ad4922a3d73ebd14
I20260202 02:52:52.880079 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 8b0368e03f644f1db28dbb57c95c48c1 with delete_type TABLET_DATA_DELETED (Replaced by d0e00bed25e740f5b39e023f9ed61d5a at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.880414 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 528d4269f91a44679acfad16e9fbbe32
I20260202 02:52:52.880709 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet fa57a267b0484262ac31039862f40a76 with delete_type TABLET_DATA_DELETED (replaced by tablet 98ee9e01f0354a2abdbc1c45f1e6acdf) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.880811 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 1d8143f529b74e6ab779dd695920413c
I20260202 02:52:52.881223 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 51ba0bcd498043c383c493f45ad1fe17
I20260202 02:52:52.881429 898 tablet_service.cc:1552] Processing DeleteTablet for tablet 7709ea34429e4e91ad4922a3d73ebd14 with delete_type TABLET_DATA_DELETED (Replaced by 22c31519d7e642118ff1218e2a1b7621 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.881489 890 tablet_service.cc:1552] Processing DeleteTablet for tablet 528d4269f91a44679acfad16e9fbbe32 with delete_type TABLET_DATA_DELETED (Replaced by 6f2739c054ce4e6b87634eced940d548 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.881624 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 1d8143f529b74e6ab779dd695920413c with delete_type TABLET_DATA_DELETED (Replaced by 84d68594516e4be4a3a1f4ebe9e217a5 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.882423 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet efe78242e0654a1c9bdbaf9cfa88b034 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.883798 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 7709ea34429e4e91ad4922a3d73ebd14 with delete_type TABLET_DATA_DELETED (Replaced by 22c31519d7e642118ff1218e2a1b7621 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.884357 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 528d4269f91a44679acfad16e9fbbe32 with delete_type TABLET_DATA_DELETED (Replaced by 6f2739c054ce4e6b87634eced940d548 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.884862 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 1d8143f529b74e6ab779dd695920413c with delete_type TABLET_DATA_DELETED (Replaced by 84d68594516e4be4a3a1f4ebe9e217a5 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.887251 1882 ts_tablet_manager.cc:1929] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:52.889474 923 raft_consensus.cc:1275] T 78f122342dd144b48de89902faa0f94f P 1521a6d262b7451bbef8debe970dd6b5 [term 3 FOLLOWER]: Refusing update from remote peer 1f442ef85029461091de847118c34c0d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260202 02:52:52.899684 527 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c584f3163f1b45789492ccd27b7ad609 consensus state has no leader
W20260202 02:52:52.902922 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 898778f8f0b548ec89cb1417d6f07926 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 898778f8f0b548ec89cb1417d6f07926 already in progress: creating tablet
I20260202 02:52:52.906409 1182 tablet_replica.cc:333] T 8153510d5d154c7895b1ee794585612e P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:52.906626 1182 raft_consensus.cc:2243] T 8153510d5d154c7895b1ee794585612e P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:52.906778 1182 raft_consensus.cc:2272] T 8153510d5d154c7895b1ee794585612e P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:52.907281 1182 ts_tablet_manager.cc:1916] T 8153510d5d154c7895b1ee794585612e P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:52.911360 1175 tablet_bootstrap.cc:492] T 9a1d7d173c41462c8e32a95b494bb708 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:52.911531 1175 ts_tablet_manager.cc:1403] T 9a1d7d173c41462c8e32a95b494bb708 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.193s user 0.001s sys 0.001s
W20260202 02:52:52.912245 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet af29c7a2ad9d4797b08052606151a7d5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet af29c7a2ad9d4797b08052606151a7d5 already in progress: creating tablet
I20260202 02:52:52.912242 1175 raft_consensus.cc:359] T 9a1d7d173c41462c8e32a95b494bb708 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.912446 1175 raft_consensus.cc:385] T 9a1d7d173c41462c8e32a95b494bb708 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:52.912498 1175 raft_consensus.cc:740] T 9a1d7d173c41462c8e32a95b494bb708 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
W20260202 02:52:52.912563 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet b00b4a0b85374bfe83928162c4da6479 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b00b4a0b85374bfe83928162c4da6479 already in progress: creating tablet
I20260202 02:52:52.912708 1175 consensus_queue.cc:260] T 9a1d7d173c41462c8e32a95b494bb708 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.913010 525 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 6f2739c054ce4e6b87634eced940d548
I20260202 02:52:52.913283 1175 ts_tablet_manager.cc:1434] T 9a1d7d173c41462c8e32a95b494bb708 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:52.913570 1175 tablet_bootstrap.cc:492] T c17c85759e964f0ba3abcaf5fe938fba P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
W20260202 02:52:52.914903 519 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet eb410274d21e4fe097b0c312e4aa4446 consensus state has no leader
I20260202 02:52:52.915462 521 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 3857680ccbdc4dd4ad8247da96254434
I20260202 02:52:52.915915 521 catalog_manager.cc:6724] ReplaceTablet: tablet 3857680ccbdc4dd4ad8247da96254434 deleted and replaced by tablet 60feac4d4fb841c0834f030db0f599d1
I20260202 02:52:52.916635 1175 tablet_bootstrap.cc:654] T c17c85759e964f0ba3abcaf5fe938fba P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:52.921489 896 tablet_service.cc:1552] Processing DeleteTablet for tablet 6f2739c054ce4e6b87634eced940d548 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.921613 526 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_1676352d51154567a520a2e75b86277f" } new_table_name: "default.Table_4f55cbc4396f4d4abfe67362cad45232" modify_external_catalogs: true
I20260202 02:52:52.921937 528 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_4bfd0a9f04ae4defa2e130a44fe45b2f" } modify_external_catalogs: true
I20260202 02:52:52.922102 528 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_4bfd0a9f04ae4defa2e130a44fe45b2f" } modify_external_catalogs: true
I20260202 02:52:52.923082 524 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 9ee7687df15d494eaf4150db580de7e3
I20260202 02:52:52.923539 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 7a44585a217b48f39f247bae3ca2e255
I20260202 02:52:52.923890 524 rpcz_store.cc:275] Call kudu.master.MasterService.GetMasterRegistration from 127.0.0.1:47398 (request call id 3) took 2041 ms. Trace:
I20260202 02:52:52.923944 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 25a407001f964da6b51f42375136c79a
I20260202 02:52:52.924007 524 rpcz_store.cc:276] 0202 02:52:50.920001 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.923628 (+2003627us) service_pool.cc:225] Handling call
0202 02:52:52.923849 (+ 221us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:52.924376 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet e2edc6660b9a41e39e9529627fafcb03
I20260202 02:52:52.924841 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 3857680ccbdc4dd4ad8247da96254434 with delete_type TABLET_DATA_DELETED (replaced by tablet 60feac4d4fb841c0834f030db0f599d1) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.930750 1182 ts_tablet_manager.cc:1929] T 8153510d5d154c7895b1ee794585612e P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:52.930869 1182 log.cc:1199] T 8153510d5d154c7895b1ee794585612e P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/8153510d5d154c7895b1ee794585612e
I20260202 02:52:52.931123 1182 ts_tablet_manager.cc:1950] T 8153510d5d154c7895b1ee794585612e P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:52.931900 527 catalog_manager.cc:5654] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d reported cstate change: term changed from 1 to 3, leader changed from b1f05820b6034228bdad884cc348716d (127.0.104.131) to 1f442ef85029461091de847118c34c0d (127.0.104.129). New cstate: current_term: 3 leader_uuid: "1f442ef85029461091de847118c34c0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } }
I20260202 02:52:52.933121 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_c04e684df4804dfda34db628838a00c3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:52.933982 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c04e684df4804dfda34db628838a00c3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:52.936076 519 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_edaccffa45e248b587c41ba28dda9dbc" } modify_external_catalogs: true
I20260202 02:52:52.936363 519 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_edaccffa45e248b587c41ba28dda9dbc" } modify_external_catalogs: true
I20260202 02:52:52.938578 890 tablet_service.cc:1552] Processing DeleteTablet for tablet 7a44585a217b48f39f247bae3ca2e255 with delete_type TABLET_DATA_DELETED (Replaced by d8d706b1bf49494c871aee3ddc16e40c at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.938620 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 8153510d5d154c7895b1ee794585612e (table default.Table_0e7ceba4956f408c8a9fc0f61fa67c90 [id=c0c6499675894350ac008b534d030c93]) successfully deleted
I20260202 02:52:52.938632 898 tablet_service.cc:1552] Processing DeleteTablet for tablet 25a407001f964da6b51f42375136c79a with delete_type TABLET_DATA_DELETED (Replaced by b3e095157aec4096a2ea5db96872ae12 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.938817 884 tablet_service.cc:1552] Processing DeleteTablet for tablet e2edc6660b9a41e39e9529627fafcb03 with delete_type TABLET_DATA_DELETED (Replaced by 5692ca1e36894c3194f0fcff18d014ba at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.938877 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 9ee7687df15d494eaf4150db580de7e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.938971 896 tablet_service.cc:1552] Processing DeleteTablet for tablet 51ba0bcd498043c383c493f45ad1fe17 with delete_type TABLET_DATA_DELETED (Replaced by dac0f74c3d3f410f8222764ba9cd7f98 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.939074 902 tablet_service.cc:1505] Processing CreateTablet for tablet d1944de086c24058b00df38d2b3d7cfd (DEFAULT_TABLE table=default.Table_0bdc711d3bce45af9149f191f5d61ee2 [id=ddce2a099a1c40538900ce30b1921708]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:52.939463 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1944de086c24058b00df38d2b3d7cfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:52.943063 2925 raft_consensus.cc:493] T 5e20842de8cb4355b732685605c88834 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:52.943198 2925 raft_consensus.cc:515] T 5e20842de8cb4355b732685605c88834 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:52.943269 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_45815470ea1c49a4bb2c2c240a9d7b4e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:52.943843 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_45815470ea1c49a4bb2c2c240a9d7b4e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:52.943841 2925 leader_election.cc:290] T 5e20842de8cb4355b732685605c88834 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:52.924436 524 rpcz_store.cc:275] Call kudu.master.MasterService.GetMasterRegistration from 127.0.0.1:48936 (request call id 1) took 2001 ms. Trace:
I20260202 02:52:52.948992 524 rpcz_store.cc:276] 0202 02:52:50.922735 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.924208 (+2001473us) service_pool.cc:225] Handling call
0202 02:52:52.924394 (+ 186us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:52.950592 1182 tablet_replica.cc:333] T eea6bb04ddf24276b46270d009ed78b4 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:52.952543 528 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 08fbd094d4df46cf82a694b4e1f30c6d
W20260202 02:52:52.953585 526 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3ee6153db90b43c5866a8bb09bcebcb3 consensus state has no leader
I20260202 02:52:52.954005 1182 raft_consensus.cc:2243] T eea6bb04ddf24276b46270d009ed78b4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:52.954349 902 tablet_service.cc:1552] Processing DeleteTablet for tablet 08fbd094d4df46cf82a694b4e1f30c6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.954586 1182 raft_consensus.cc:2272] T eea6bb04ddf24276b46270d009ed78b4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:52.955271 1182 ts_tablet_manager.cc:1916] T eea6bb04ddf24276b46270d009ed78b4 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:52.955555 2925 raft_consensus.cc:493] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:52.955675 2925 raft_consensus.cc:515] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.956256 2925 leader_election.cc:290] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:52.958122 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_74a1a2c121f1430d87ced4156ee01e53"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:52.958768 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_74a1a2c121f1430d87ced4156ee01e53 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:52.963032 902 tablet_service.cc:1505] Processing CreateTablet for tablet 3f7da8bb0c2f46bb9c12ffc8c87a5967 (DEFAULT_TABLE table=default.Table_a1067e637b814119a16c9a4d8039b649 [id=e726212e556a441396ac44b3e563cac0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:52.963462 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f7da8bb0c2f46bb9c12ffc8c87a5967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:52.964960 896 tablet_service.cc:1505] Processing CreateTablet for tablet fc2bc0599d7144dba486f56d7ce52f2c (DEFAULT_TABLE table=default.Table_82031bd5aec3477bb4ccf0183961a528 [id=6561bda3e33745c1aa7bdd672ef9755a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:52.965353 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc2bc0599d7144dba486f56d7ce52f2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:52.964957 886 tablet_service.cc:1505] Processing CreateTablet for tablet 7792d72dd1054e56b11ed10b365167cf (DEFAULT_TABLE table=default.Table_d64d39313f1448de84c04387a8a6d33d [id=7c34cab2c3b04f678c7511063985eadb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:52.966079 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7792d72dd1054e56b11ed10b365167cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:52.968775 519 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 81abd5ef66cc4f0bb37844c6ff5c256d
W20260202 02:52:52.889050 999 leader_election.cc:343] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 80eb3013f5354022a2dd78bb294a8497
W20260202 02:52:52.969580 999 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 80536.6us
W20260202 02:52:52.970125 999 leader_election.cc:343] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 80eb3013f5354022a2dd78bb294a8497
W20260202 02:52:52.970422 999 leader_election.cc:343] T 75a24a1bf54d4c799938a5723ac6fa99 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 75a24a1bf54d4c799938a5723ac6fa99
I20260202 02:52:52.970508 999 leader_election.cc:304] T 75a24a1bf54d4c799938a5723ac6fa99 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:52.971187 1882 log.cc:1199] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/1116872cc25a4ab5b1a3a6bbba1f0da7
I20260202 02:52:52.971629 1882 ts_tablet_manager.cc:1950] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:52.973418 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 765) took 3855 ms. Trace:
I20260202 02:52:52.978354 527 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet 939fbcaf75c7439d9208bbbf83a72717 (table default.Table_12d68a317c8e4d809dbbe77c095a1f6e [id=7219522ffc3849faab94d754a42a8511]). Expected version 1 got 0
I20260202 02:52:52.899547 1173 tablet_bootstrap.cc:492] T 7709ea34429e4e91ad4922a3d73ebd14 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:52.917567 1043 tablet_service.cc:1552] Processing DeleteTablet for tablet 3857680ccbdc4dd4ad8247da96254434 with delete_type TABLET_DATA_DELETED (replaced by tablet 60feac4d4fb841c0834f030db0f599d1) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.918056 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 6f2739c054ce4e6b87634eced940d548 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.941823 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet 51ba0bcd498043c383c493f45ad1fe17 with delete_type TABLET_DATA_DELETED (Replaced by dac0f74c3d3f410f8222764ba9cd7f98 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.979799 1173 ts_tablet_manager.cc:1403] T 7709ea34429e4e91ad4922a3d73ebd14 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.136s user 0.008s sys 0.015s
I20260202 02:52:52.979665 1882 rpcz_store.cc:276] 0202 02:52:49.117931 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:49.118003 (+ 72us) service_pool.cc:225] Handling call
0202 02:52:52.675360 (+3557357us) ts_tablet_manager.cc:1151] Deleting tablet 1116872cc25a4ab5b1a3a6bbba1f0da7
0202 02:52:52.675372 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:52.873288 (+197916us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:52.887146 (+ 13858us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:52.973396 (+ 86250us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3557146}
I20260202 02:52:52.941990 1031 tablet_service.cc:1552] Processing DeleteTablet for tablet 9ee7687df15d494eaf4150db580de7e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.942075 1035 tablet_service.cc:1552] Processing DeleteTablet for tablet 7a44585a217b48f39f247bae3ca2e255 with delete_type TABLET_DATA_DELETED (Replaced by d8d706b1bf49494c871aee3ddc16e40c at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.980541 1173 raft_consensus.cc:359] T 7709ea34429e4e91ad4922a3d73ebd14 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.980698 1173 raft_consensus.cc:385] T 7709ea34429e4e91ad4922a3d73ebd14 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:52.980749 1173 raft_consensus.cc:740] T 7709ea34429e4e91ad4922a3d73ebd14 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:52.980909 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 1116872cc25a4ab5b1a3a6bbba1f0da7 (table default.Table_d6ba351d68ec4a10939a09519b7d4b67 [id=52a6d31f993d4c74a520d504390cf12b]) successfully deleted
I20260202 02:52:52.980937 1173 consensus_queue.cc:260] T 7709ea34429e4e91ad4922a3d73ebd14 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:52.981141 1882 tablet_replica.cc:333] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:52.981346 1173 ts_tablet_manager.cc:1434] T 7709ea34429e4e91ad4922a3d73ebd14 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260202 02:52:52.981611 1173 tablet_bootstrap.cc:492] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:52.895851 1060 raft_consensus.cc:1275] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Refusing update from remote peer 1521a6d262b7451bbef8debe970dd6b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260202 02:52:52.982625 1882 raft_consensus.cc:2243] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d [term 2 LEADER]: Raft consensus shutting down.
I20260202 02:52:52.982720 2925 consensus_queue.cc:1048] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260202 02:52:52.982870 1882 pending_rounds.cc:70] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d: Trying to abort 1 pending ops.
I20260202 02:52:52.982937 1882 pending_rounds.cc:77] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d: Aborting op as it isn't in flight: id { term: 2 index: 5 } timestamp: 7249923165671108608 op_type: NO_OP noop_request { }
I20260202 02:52:52.983038 1882 raft_consensus.cc:2889] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:52.983136 1882 raft_consensus.cc:2272] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:52.983242 1173 tablet_bootstrap.cc:654] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:52.943303 1032 tablet_service.cc:1552] Processing DeleteTablet for tablet 25a407001f964da6b51f42375136c79a with delete_type TABLET_DATA_DELETED (Replaced by b3e095157aec4096a2ea5db96872ae12 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.984426 902 tablet_service.cc:1552] Processing DeleteTablet for tablet 81abd5ef66cc4f0bb37844c6ff5c256d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:52.943656 1029 tablet_service.cc:1505] Processing CreateTablet for tablet d1944de086c24058b00df38d2b3d7cfd (DEFAULT_TABLE table=default.Table_0bdc711d3bce45af9149f191f5d61ee2 [id=ddce2a099a1c40538900ce30b1921708]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:52.984869 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1944de086c24058b00df38d2b3d7cfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:52.987294 1882 ts_tablet_manager.cc:1916] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:52.944569 1042 tablet_service.cc:1552] Processing DeleteTablet for tablet e2edc6660b9a41e39e9529627fafcb03 with delete_type TABLET_DATA_DELETED (Replaced by 5692ca1e36894c3194f0fcff18d014ba at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.945241 1054 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5e20842de8cb4355b732685605c88834" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:52.957433 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6a3dac5cd1ce47488a50b4c32eca67c7" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:52.944693 1040 tablet_service.cc:1552] Processing DeleteTablet for tablet 75a24a1bf54d4c799938a5723ac6fa99 with delete_type TABLET_DATA_DELETED (replaced by tablet 6750a1212ceb4e6e82e4ffa93caab590) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.957742 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 08fbd094d4df46cf82a694b4e1f30c6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:52.963922 1039 tablet_service.cc:1505] Processing CreateTablet for tablet 3f7da8bb0c2f46bb9c12ffc8c87a5967 (DEFAULT_TABLE table=default.Table_a1067e637b814119a16c9a4d8039b649 [id=e726212e556a441396ac44b3e563cac0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:52.989424 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f7da8bb0c2f46bb9c12ffc8c87a5967. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:52.989596 859 leader_election.cc:343] T 5e20842de8cb4355b732685605c88834 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:52.989817 859 leader_election.cc:343] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:52.977280 2977 raft_consensus.cc:2749] T 75a24a1bf54d4c799938a5723ac6fa99 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:52.978583 2740 consensus_queue.cc:1048] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:52.977993 1038 tablet_service.cc:1505] Processing CreateTablet for tablet fc2bc0599d7144dba486f56d7ce52f2c (DEFAULT_TABLE table=default.Table_82031bd5aec3477bb4ccf0183961a528 [id=6561bda3e33745c1aa7bdd672ef9755a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:52.993041 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc2bc0599d7144dba486f56d7ce52f2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.002259 1059 raft_consensus.cc:3060] T 7709ea34429e4e91ad4922a3d73ebd14 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:53.005054 999 consensus_peers.cc:597] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: da5435f969074364bc8f8a6ddc5e4f8a. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:53.007853 525 catalog_manager.cc:5654] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 reported cstate change: term changed from 1 to 2, leader changed from b1f05820b6034228bdad884cc348716d (127.0.104.131) to 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130). New cstate: current_term: 2 leader_uuid: "1521a6d262b7451bbef8debe970dd6b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: HEALTHY } } }
I20260202 02:52:53.008620 525 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 939fbcaf75c7439d9208bbbf83a72717 (table default.Table_12d68a317c8e4d809dbbe77c095a1f6e [id=7219522ffc3849faab94d754a42a8511]). Expected version 1 got 0
I20260202 02:52:53.008867 525 catalog_manager.cc:5654] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "1521a6d262b7451bbef8debe970dd6b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: HEALTHY } } }
I20260202 02:52:53.013280 526 master_service.cc:954] ReplaceTablet: received request to replace tablet 91a604fb877a48479914a5b60144a22d from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:53.018509 1882 ts_tablet_manager.cc:1929] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.5
I20260202 02:52:53.018782 1882 log.cc:1199] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/80eb3013f5354022a2dd78bb294a8497
I20260202 02:52:53.020666 526 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_78bb96bdb6a1489f94628205c7ccacfd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:53.021306 526 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_78bb96bdb6a1489f94628205c7ccacfd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:53.024016 1882 ts_tablet_manager.cc:1950] T 80eb3013f5354022a2dd78bb294a8497 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:53.024214 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_c5b9c9e8e8d043bdbb55326857791c2a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:53.024824 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c5b9c9e8e8d043bdbb55326857791c2a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:53.025158 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 766) took 3722 ms. Trace:
I20260202 02:52:53.025252 1882 rpcz_store.cc:276] 0202 02:52:49.302637 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:49.302698 (+ 61us) service_pool.cc:225] Handling call
0202 02:52:52.981091 (+3678393us) ts_tablet_manager.cc:1151] Deleting tablet 80eb3013f5354022a2dd78bb294a8497
0202 02:52:52.981102 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:53.003619 (+ 22517us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:53.018352 (+ 14733us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:53.025117 (+ 6765us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3678179}
I20260202 02:52:53.019513 3032 raft_consensus.cc:493] T 99e0b5e9a9874aec886155e0d614dec2 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:53.025753 3032 raft_consensus.cc:515] T 99e0b5e9a9874aec886155e0d614dec2 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.026505 522 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_988dd0b38bd141bf8cba853c4d6685b0" } new_table_name: "default.Table_75db64440693483083cf6cb51164e115" modify_external_catalogs: true
W20260202 02:52:53.028396 999 leader_election.cc:343] T 99e0b5e9a9874aec886155e0d614dec2 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 99e0b5e9a9874aec886155e0d614dec2
I20260202 02:52:53.028712 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_bd2d37a87601403eb2a45621e900472b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:53.028963 3032 leader_election.cc:290] T 99e0b5e9a9874aec886155e0d614dec2 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:53.029299 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bd2d37a87601403eb2a45621e900472b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:53.030699 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 80eb3013f5354022a2dd78bb294a8497 (table default.Table_4e4d461d49574c68a7d93ac891db67c6 [id=316b8a276d9b415fafe8511aa7c32fc5]) successfully deleted
I20260202 02:52:53.033526 1882 tablet_replica.cc:333] T 73f8db8f30f14d369d2227c9be7c4647 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:53.033805 1882 raft_consensus.cc:2243] T 73f8db8f30f14d369d2227c9be7c4647 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:53.033995 1882 raft_consensus.cc:2272] T 73f8db8f30f14d369d2227c9be7c4647 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:53.034565 1882 ts_tablet_manager.cc:1916] T 73f8db8f30f14d369d2227c9be7c4647 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:53.035604 902 tablet_service.cc:1552] Processing DeleteTablet for tablet 5e20842de8cb4355b732685605c88834 with delete_type TABLET_DATA_DELETED (replaced by tablet 34a01a4ce1be4ad08d911763de158073) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:53.040557 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 7792d72dd1054e56b11ed10b365167cf (DEFAULT_TABLE table=default.Table_d64d39313f1448de84c04387a8a6d33d [id=7c34cab2c3b04f678c7511063985eadb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.040838 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 81abd5ef66cc4f0bb37844c6ff5c256d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:53.041015 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7792d72dd1054e56b11ed10b365167cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.041597 526 catalog_manager.cc:5654] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 reported cstate change: term changed from 1 to 2, leader changed from b1f05820b6034228bdad884cc348716d (127.0.104.131) to 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130). New cstate: current_term: 2 leader_uuid: "1521a6d262b7451bbef8debe970dd6b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } }
I20260202 02:52:53.042757 3035 tablet.cc:1722] T 939fbcaf75c7439d9208bbbf83a72717 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:53.044634 2929 consensus_queue.cc:1048] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260202 02:52:53.046449 522 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8cc5e23a86a3467c89794ae8c403230e consensus state has no leader
I20260202 02:52:53.052613 3036 tablet.cc:1722] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260202 02:52:53.064112 999 consensus_peers.cc:597] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2f5aae089fcc492f9ce55f84674a452c. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:53.066646 902 tablet_service.cc:1505] Processing CreateTablet for tablet f2460dbd2a27487db94899b54d519099 (DEFAULT_TABLE table=default.Table_e0bea82391b342c1a81564e3b2789f94 [id=f155562b32254df8aa3c0b0c840b1e17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.067098 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2460dbd2a27487db94899b54d519099. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.076627 2964 consensus_queue.cc:1048] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:52.875703 715 leader_election.cc:304] T 0138ba7326674741b0729324735f1de7 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
W20260202 02:52:53.077690 715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 202008us
I20260202 02:52:52.880903 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 7709ea34429e4e91ad4922a3d73ebd14 with delete_type TABLET_DATA_DELETED (Replaced by 22c31519d7e642118ff1218e2a1b7621 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:53.078430 2911 raft_consensus.cc:2804] T 0138ba7326674741b0729324735f1de7 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:53.078680 2881 raft_consensus.cc:493] T 79c73f4a92044bafb5b8384229cb3a4d P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:53.078784 2881 raft_consensus.cc:515] T 79c73f4a92044bafb5b8384229cb3a4d P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:53.079397 2881 leader_election.cc:290] T 79c73f4a92044bafb5b8384229cb3a4d P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:52.881331 745 tablet_service.cc:1552] Processing DeleteTablet for tablet 528d4269f91a44679acfad16e9fbbe32 with delete_type TABLET_DATA_DELETED (Replaced by 6f2739c054ce4e6b87634eced940d548 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:52.881734 755 tablet_service.cc:1552] Processing DeleteTablet for tablet 1d8143f529b74e6ab779dd695920413c with delete_type TABLET_DATA_DELETED (Replaced by 84d68594516e4be4a3a1f4ebe9e217a5 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:42950
W20260202 02:52:52.893728 714 leader_election.cc:343] T 0138ba7326674741b0729324735f1de7 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 0138ba7326674741b0729324735f1de7
W20260202 02:52:53.080601 714 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 186874us
W20260202 02:52:53.080834 714 leader_election.cc:343] T 0138ba7326674741b0729324735f1de7 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 0138ba7326674741b0729324735f1de7
I20260202 02:52:52.894060 2633 consensus_queue.cc:1048] T 78f122342dd144b48de89902faa0f94f P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260202 02:52:52.894443 764 raft_consensus.cc:1275] T f6746ccbf37e477196a9706396d1e3fd P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Refusing update from remote peer 1521a6d262b7451bbef8debe970dd6b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260202 02:52:52.906157 1174 tablet_bootstrap.cc:492] T b427c7e897c64977b924e7d586420e69 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:52.945017 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5e20842de8cb4355b732685605c88834" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:53.082751 1174 ts_tablet_manager.cc:1403] T b427c7e897c64977b924e7d586420e69 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.305s user 0.018s sys 0.036s
I20260202 02:52:52.958081 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6a3dac5cd1ce47488a50b4c32eca67c7" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:52.987274 3030 tablet.cc:1722] T 939fbcaf75c7439d9208bbbf83a72717 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:53.083438 1174 raft_consensus.cc:359] T b427c7e897c64977b924e7d586420e69 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.083595 1174 raft_consensus.cc:385] T b427c7e897c64977b924e7d586420e69 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.083640 1174 raft_consensus.cc:740] T b427c7e897c64977b924e7d586420e69 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:53.083814 1174 consensus_queue.cc:260] T b427c7e897c64977b924e7d586420e69 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.084255 1174 ts_tablet_manager.cc:1434] T b427c7e897c64977b924e7d586420e69 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260202 02:52:53.084544 1174 tablet_bootstrap.cc:492] T 5e20842de8cb4355b732685605c88834 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:53.086167 1174 tablet_bootstrap.cc:654] T 5e20842de8cb4355b732685605c88834 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:53.087325 3030 tablet.cc:1722] T c050a2882f764dda8df2a7df6547f149 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260202 02:52:53.029022 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "99e0b5e9a9874aec886155e0d614dec2" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:53.049475 2878 raft_consensus.cc:493] T 2b352d9a815e4bccafb7e2251500c5bc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:53.088603 2878 raft_consensus.cc:515] T 2b352d9a815e4bccafb7e2251500c5bc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.089210 2878 leader_election.cc:290] T 2b352d9a815e4bccafb7e2251500c5bc P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:53.070765 775 raft_consensus.cc:1275] T 16b47a2831f54d809ed0b16deaec7e3d P 1f442ef85029461091de847118c34c0d [term 3 FOLLOWER]: Refusing update from remote peer b1f05820b6034228bdad884cc348716d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260202 02:52:53.089279 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_399d6088949d4b3488bdde3ee5dcf41f" } new_table_name: "default.Table_1b41581f01c94cc497e8af87b594b565" modify_external_catalogs: true
I20260202 02:52:53.092164 521 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_f7636b54f7f34f2a9487c03e57a621a9" } modify_external_catalogs: true
I20260202 02:52:53.092419 521 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_f7636b54f7f34f2a9487c03e57a621a9" } modify_external_catalogs: true
I20260202 02:52:53.092839 2911 raft_consensus.cc:697] T 0138ba7326674741b0729324735f1de7 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:53.093181 2911 consensus_queue.cc:237] T 0138ba7326674741b0729324735f1de7 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.096917 526 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_4f72bc6302d84055ba28584ec5e3ace9 [id=0c8385b71ceb44199472e1ce640b54e0] alter complete (version 1)
I20260202 02:52:53.100829 1174 tablet_bootstrap.cc:492] T 5e20842de8cb4355b732685605c88834 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:53.101008 1174 ts_tablet_manager.cc:1403] T 5e20842de8cb4355b732685605c88834 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.002s
I20260202 02:52:53.101794 1174 raft_consensus.cc:359] T 5e20842de8cb4355b732685605c88834 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:53.102074 1174 raft_consensus.cc:385] T 5e20842de8cb4355b732685605c88834 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.102133 1174 raft_consensus.cc:740] T 5e20842de8cb4355b732685605c88834 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:53.102358 1174 consensus_queue.cc:260] T 5e20842de8cb4355b732685605c88834 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:53.102849 1174 ts_tablet_manager.cc:1434] T 5e20842de8cb4355b732685605c88834 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:53.103179 1174 tablet_bootstrap.cc:492] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:53.105846 1174 tablet_bootstrap.cc:654] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:53.106592 521 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 477c599f8c9b4eb0b435ea50abd5d36b
W20260202 02:52:53.107946 527 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0883a26fce2540a5bc1c38e67b2f0dda has no consensus state
I20260202 02:52:53.108585 902 tablet_service.cc:1552] Processing DeleteTablet for tablet 477c599f8c9b4eb0b435ea50abd5d36b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:53 UTC) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:53.108716 857 leader_election.cc:343] T 5e20842de8cb4355b732685605c88834 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:53.108850 857 leader_election.cc:304] T 5e20842de8cb4355b732685605c88834 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:53.109076 2925 raft_consensus.cc:493] T dc90eeb79ded4739883ad98928476680 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:53.109210 2925 raft_consensus.cc:515] T dc90eeb79ded4739883ad98928476680 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.109884 2925 leader_election.cc:290] T dc90eeb79ded4739883ad98928476680 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:53.110038 2925 consensus_queue.cc:1048] T f6746ccbf37e477196a9706396d1e3fd P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260202 02:52:53.110463 2925 raft_consensus.cc:2749] T 5e20842de8cb4355b732685605c88834 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:53.111199 857 leader_election.cc:343] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:53.111413 857 leader_election.cc:304] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:53.112421 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dc90eeb79ded4739883ad98928476680" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:53.113883 526 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_c57473bfb15e41da8627401aafd82bce" } new_table_name: "default.Table_7cdb7249dcfe4b5aa11091f41f7bf50e" modify_external_catalogs: true
I20260202 02:52:53.114882 1174 tablet_bootstrap.cc:492] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:53.114879 2925 raft_consensus.cc:2749] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:53.115031 1174 ts_tablet_manager.cc:1403] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.003s
I20260202 02:52:53.115785 1174 raft_consensus.cc:359] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.115963 1174 raft_consensus.cc:385] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.116026 1174 raft_consensus.cc:740] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:53.116216 1174 consensus_queue.cc:260] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.118922 1175 tablet_bootstrap.cc:492] T c17c85759e964f0ba3abcaf5fe938fba P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:53.119081 1175 ts_tablet_manager.cc:1403] T c17c85759e964f0ba3abcaf5fe938fba P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.206s user 0.030s sys 0.045s
I20260202 02:52:53.120024 1175 raft_consensus.cc:359] T c17c85759e964f0ba3abcaf5fe938fba P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.120296 1175 raft_consensus.cc:385] T c17c85759e964f0ba3abcaf5fe938fba P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.120429 1175 raft_consensus.cc:740] T c17c85759e964f0ba3abcaf5fe938fba P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
W20260202 02:52:53.120610 526 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet de76ca9a98c840d0967a97154b940b60 consensus state has no leader
I20260202 02:52:53.120873 1175 consensus_queue.cc:260] T c17c85759e964f0ba3abcaf5fe938fba P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.121483 1175 ts_tablet_manager.cc:1434] T c17c85759e964f0ba3abcaf5fe938fba P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260202 02:52:53.121542 527 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_cb2845f3c47d4f96bb58a4ed5fe5b92b" } modify_external_catalogs: true
I20260202 02:52:53.121924 1175 tablet_bootstrap.cc:492] T 7f323175ee254df789bcb04e65047dff P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:53.123922 1175 tablet_bootstrap.cc:654] T 7f323175ee254df789bcb04e65047dff P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:53.123651 527 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_cb2845f3c47d4f96bb58a4ed5fe5b92b" } modify_external_catalogs: true
I20260202 02:52:53.124289 1174 ts_tablet_manager.cc:1434] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260202 02:52:53.124805 1174 tablet_bootstrap.cc:492] T 59ad016e15ec4a488921eb564029b689 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:53.126556 1174 tablet_bootstrap.cc:654] T 59ad016e15ec4a488921eb564029b689 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:53.126710 526 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_82031bd5aec3477bb4ccf0183961a528" } new_table_name: "default.Table_56daf62a7ab548b39dc1c8b765532597" modify_external_catalogs: true
I20260202 02:52:53.128652 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_20f2fd4d31db4d66be77799ede02027e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:53.129328 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_20f2fd4d31db4d66be77799ede02027e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:52:53.134289 857 leader_election.cc:343] T dc90eeb79ded4739883ad98928476680 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:53.138068 1174 tablet_bootstrap.cc:492] T 59ad016e15ec4a488921eb564029b689 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:53.138315 1174 ts_tablet_manager.cc:1403] T 59ad016e15ec4a488921eb564029b689 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.006s
I20260202 02:52:53.139125 1174 raft_consensus.cc:359] T 59ad016e15ec4a488921eb564029b689 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.140676 1175 tablet_bootstrap.cc:492] T 7f323175ee254df789bcb04e65047dff P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:53.140889 1175 ts_tablet_manager.cc:1403] T 7f323175ee254df789bcb04e65047dff P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.003s
I20260202 02:52:53.141651 1175 raft_consensus.cc:359] T 7f323175ee254df789bcb04e65047dff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.141886 1175 raft_consensus.cc:385] T 7f323175ee254df789bcb04e65047dff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.141988 1175 raft_consensus.cc:740] T 7f323175ee254df789bcb04e65047dff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:53.142279 1175 consensus_queue.cc:260] T 7f323175ee254df789bcb04e65047dff P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.145460 1175 ts_tablet_manager.cc:1434] T 7f323175ee254df789bcb04e65047dff P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260202 02:52:53.139876 1174 raft_consensus.cc:385] T 59ad016e15ec4a488921eb564029b689 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.145759 1174 raft_consensus.cc:740] T 59ad016e15ec4a488921eb564029b689 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:53.146126 1175 tablet_bootstrap.cc:492] T 4a518e8ad2e34ec98a0425c7ed3a470a P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:53.147857 1175 tablet_bootstrap.cc:654] T 4a518e8ad2e34ec98a0425c7ed3a470a P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:53.149902 1174 consensus_queue.cc:260] T 59ad016e15ec4a488921eb564029b689 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.150695 1174 ts_tablet_manager.cc:1434] T 59ad016e15ec4a488921eb564029b689 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.012s user 0.001s sys 0.001s
I20260202 02:52:53.151046 1174 tablet_bootstrap.cc:492] T f3eebefa74584c1791d52377f5ba1290 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:53.151149 528 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 7f323175ee254df789bcb04e65047dff (table default.Table_c08aceb74c164c06a3a149c454c9c59e [id=0d6674bc4f264a36aad4ea7962a9e0e7]). Expected version 1 got 0
I20260202 02:52:53.154122 1174 tablet_bootstrap.cc:654] T f3eebefa74584c1791d52377f5ba1290 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:53.154331 527 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet d3f6a30e08244a07a1f81b5e98bc6965
I20260202 02:52:53.155515 902 tablet_service.cc:1552] Processing DeleteTablet for tablet d3f6a30e08244a07a1f81b5e98bc6965 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:53 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:53.158202 527 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_42bf0269c9e64c62adfa02c34d69459e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:53.158910 527 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_42bf0269c9e64c62adfa02c34d69459e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:53.163237 521 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_320a9a8a16b949a8a4dfe8fec6a9e5a4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:53.163933 521 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_320a9a8a16b949a8a4dfe8fec6a9e5a4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:53.164685 1175 tablet_bootstrap.cc:492] T 4a518e8ad2e34ec98a0425c7ed3a470a P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:53.164850 1175 ts_tablet_manager.cc:1403] T 4a518e8ad2e34ec98a0425c7ed3a470a P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.000s
I20260202 02:52:53.164897 1180 log.cc:1199] T 75a24a1bf54d4c799938a5723ac6fa99 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/75a24a1bf54d4c799938a5723ac6fa99
I20260202 02:52:53.165237 1180 ts_tablet_manager.cc:1950] T 75a24a1bf54d4c799938a5723ac6fa99 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:53.165606 1175 raft_consensus.cc:359] T 4a518e8ad2e34ec98a0425c7ed3a470a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.165783 1175 raft_consensus.cc:385] T 4a518e8ad2e34ec98a0425c7ed3a470a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.166028 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 726) took 5224 ms. Trace:
I20260202 02:52:53.165841 1175 raft_consensus.cc:740] T 4a518e8ad2e34ec98a0425c7ed3a470a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:53.166121 1180 rpcz_store.cc:276] 0202 02:52:47.941099 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:47.952455 (+ 11356us) service_pool.cc:225] Handling call
0202 02:52:52.681777 (+4729322us) ts_tablet_manager.cc:1151] Deleting tablet 75a24a1bf54d4c799938a5723ac6fa99
0202 02:52:52.681795 (+ 18us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:52.866926 (+185131us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:52.869554 (+ 2628us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:53.166004 (+296450us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4625338}
I20260202 02:52:53.166368 528 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_71bd027cef9447b4895e5363c9f0d75c" } modify_external_catalogs: true
I20260202 02:52:53.166435 1175 consensus_queue.cc:260] T 4a518e8ad2e34ec98a0425c7ed3a470a P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.166551 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 727) took 5225 ms. Trace:
I20260202 02:52:53.166584 528 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_71bd027cef9447b4895e5363c9f0d75c" } modify_external_catalogs: true
I20260202 02:52:53.166615 1180 rpcz_store.cc:276] 0202 02:52:47.941170 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:47.944741 (+ 3571us) service_pool.cc:225] Handling call
0202 02:52:53.166423 (+5221682us) ts_tablet_manager.cc:1151] Deleting tablet b8f2b34b4f1e4ff2b2be4ff40aeb0c66
0202 02:52:53.166432 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:53.166536 (+ 104us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5109730}
I20260202 02:52:53.167169 1175 ts_tablet_manager.cc:1434] T 4a518e8ad2e34ec98a0425c7ed3a470a P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:53.167810 1175 tablet_bootstrap.cc:492] T b9fc33e8efc04734a951f7fcf9048142 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:53.170097 1175 tablet_bootstrap.cc:654] T b9fc33e8efc04734a951f7fcf9048142 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:53.172974 1174 tablet_bootstrap.cc:492] T f3eebefa74584c1791d52377f5ba1290 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:53.172977 522 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_8c3c9c0b564041f99e2006f890fc7559"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:53.173198 2878 raft_consensus.cc:493] T 3284213e4b0447b49a7718ba896d3fa8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:53.173336 1174 ts_tablet_manager.cc:1403] T f3eebefa74584c1791d52377f5ba1290 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.012s
I20260202 02:52:53.173341 2878 raft_consensus.cc:515] T 3284213e4b0447b49a7718ba896d3fa8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
W20260202 02:52:53.173635 522 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8c3c9c0b564041f99e2006f890fc7559 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:53.173979 2878 leader_election.cc:290] T 3284213e4b0447b49a7718ba896d3fa8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:53.174893 1174 raft_consensus.cc:359] T f3eebefa74584c1791d52377f5ba1290 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.175122 1174 raft_consensus.cc:385] T f3eebefa74584c1791d52377f5ba1290 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.175220 1174 raft_consensus.cc:740] T f3eebefa74584c1791d52377f5ba1290 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:53.175510 1174 consensus_queue.cc:260] T f3eebefa74584c1791d52377f5ba1290 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.176266 1174 ts_tablet_manager.cc:1434] T f3eebefa74584c1791d52377f5ba1290 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260202 02:52:53.176540 519 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_672611622d8d492f97aa14af71206575" } new_table_name: "default.Table_fefb919a83c14757b698b97adee1adb9" modify_external_catalogs: true
I20260202 02:52:53.176892 1174 tablet_bootstrap.cc:492] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
W20260202 02:52:53.089021 998 leader_election.cc:343] T 99e0b5e9a9874aec886155e0d614dec2 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:53.179090 1175 tablet_bootstrap.cc:492] T b9fc33e8efc04734a951f7fcf9048142 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:53.179248 1175 ts_tablet_manager.cc:1403] T b9fc33e8efc04734a951f7fcf9048142 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.003s
I20260202 02:52:53.179798 525 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_0c357b5eb1e14dbba252f4edfc630705" } modify_external_catalogs: true
I20260202 02:52:53.180012 525 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_0c357b5eb1e14dbba252f4edfc630705" } modify_external_catalogs: true
I20260202 02:52:53.180092 1175 raft_consensus.cc:359] T b9fc33e8efc04734a951f7fcf9048142 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.180246 1175 raft_consensus.cc:385] T b9fc33e8efc04734a951f7fcf9048142 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.180289 1175 raft_consensus.cc:740] T b9fc33e8efc04734a951f7fcf9048142 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:53.186328 1174 tablet_bootstrap.cc:654] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:53.180578 1175 consensus_queue.cc:260] T b9fc33e8efc04734a951f7fcf9048142 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.187733 1175 ts_tablet_manager.cc:1434] T b9fc33e8efc04734a951f7fcf9048142 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.008s user 0.003s sys 0.000s
I20260202 02:52:53.188082 1175 tablet_bootstrap.cc:492] T 7f9dfac19c04475aadb45e95a6474eb4 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:53.189460 524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_8aca64ada6af469b9946c447af43c260"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:53.190104 1175 tablet_bootstrap.cc:654] T 7f9dfac19c04475aadb45e95a6474eb4 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:53.190222 524 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8aca64ada6af469b9946c447af43c260 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:53.194005 1175 tablet_bootstrap.cc:492] T 7f9dfac19c04475aadb45e95a6474eb4 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:53.194162 1175 ts_tablet_manager.cc:1403] T 7f9dfac19c04475aadb45e95a6474eb4 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260202 02:52:53.194094 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_c59a4ef49a364dcd8acdee41798c5793"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:53.194684 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c59a4ef49a364dcd8acdee41798c5793 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:53.194969 1175 raft_consensus.cc:359] T 7f9dfac19c04475aadb45e95a6474eb4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.195122 1175 raft_consensus.cc:385] T 7f9dfac19c04475aadb45e95a6474eb4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.195170 1175 raft_consensus.cc:740] T 7f9dfac19c04475aadb45e95a6474eb4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:53.197705 1174 tablet_bootstrap.cc:492] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:53.197882 1174 ts_tablet_manager.cc:1403] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.005s
I20260202 02:52:53.202095 523 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 7f323175ee254df789bcb04e65047dff (table default.Table_c08aceb74c164c06a3a149c454c9c59e [id=0d6674bc4f264a36aad4ea7962a9e0e7]). Expected version 1 got 0
W20260202 02:52:53.204602 999 consensus_peers.cc:597] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8153510d5d154c7895b1ee794585612e. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:53.205272 998 leader_election.cc:304] T 99e0b5e9a9874aec886155e0d614dec2 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:53.205858 1882 ts_tablet_manager.cc:1929] T 73f8db8f30f14d369d2227c9be7c4647 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:53.206780 1173 tablet_bootstrap.cc:492] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:53.207195 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet d3f6a30e08244a07a1f81b5e98bc6965 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:53 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:53.207860 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dc90eeb79ded4739883ad98928476680" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:53.207969 1038 tablet_service.cc:1505] Processing CreateTablet for tablet f2460dbd2a27487db94899b54d519099 (DEFAULT_TABLE table=default.Table_e0bea82391b342c1a81564e3b2789f94 [id=f155562b32254df8aa3c0b0c840b1e17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.207938 1174 raft_consensus.cc:359] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.208123 1174 raft_consensus.cc:385] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.208173 1174 raft_consensus.cc:740] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:53.208374 1174 consensus_queue.cc:260] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
W20260202 02:52:53.208679 999 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 70068.1us
I20260202 02:52:53.209000 1174 ts_tablet_manager.cc:1434] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20260202 02:52:53.209008 3032 raft_consensus.cc:493] T c8a273ac8cd344719a48fd81945bed0a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260202 02:52:53.209151 998 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 116615us
I20260202 02:52:53.209290 1174 tablet_bootstrap.cc:492] T dc90eeb79ded4739883ad98928476680 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:53.209565 2740 raft_consensus.cc:493] T 07580ad1d049475a9645acfad0b1d219 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:53.210175 1882 log.cc:1199] T 73f8db8f30f14d369d2227c9be7c4647 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/73f8db8f30f14d369d2227c9be7c4647
I20260202 02:52:53.210381 766 raft_consensus.cc:3060] T 8153510d5d154c7895b1ee794585612e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:53.210584 1173 ts_tablet_manager.cc:1403] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.225s user 0.003s sys 0.005s
I20260202 02:52:53.210104 2977 consensus_queue.cc:1048] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260202 02:52:53.211166 2794 raft_consensus.cc:2749] T 99e0b5e9a9874aec886155e0d614dec2 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:53.211494 859 leader_election.cc:343] T dc90eeb79ded4739883ad98928476680 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:53.213392 1174 tablet_bootstrap.cc:654] T dc90eeb79ded4739883ad98928476680 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:53.214267 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_06e158b452e14d3eba4e5e9290d48c7c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:53.214468 1175 consensus_queue.cc:260] T 7f9dfac19c04475aadb45e95a6474eb4 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
W20260202 02:52:53.214932 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_06e158b452e14d3eba4e5e9290d48c7c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:53.216732 1175 ts_tablet_manager.cc:1434] T 7f9dfac19c04475aadb45e95a6474eb4 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.023s user 0.001s sys 0.001s
I20260202 02:52:53.216770 859 leader_election.cc:304] T dc90eeb79ded4739883ad98928476680 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:53.217048 1175 tablet_bootstrap.cc:492] T c977c146bac34638874b7a990604a0d5 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:53.217485 2964 raft_consensus.cc:2749] T dc90eeb79ded4739883ad98928476680 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:53.218209 2977 consensus_queue.cc:1048] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260202 02:52:53.215648 2740 raft_consensus.cc:515] T 07580ad1d049475a9645acfad0b1d219 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.220919 2964 raft_consensus.cc:493] T b9fc33e8efc04734a951f7fcf9048142 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:53.221079 2964 raft_consensus.cc:515] T b9fc33e8efc04734a951f7fcf9048142 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.221359 1175 tablet_bootstrap.cc:654] T c977c146bac34638874b7a990604a0d5 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:53.221917 2964 leader_election.cc:290] T b9fc33e8efc04734a951f7fcf9048142 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:53.222786 519 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 943835f6f43e41c99480295fbbb65adb consensus state has no leader
I20260202 02:52:53.222841 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b9fc33e8efc04734a951f7fcf9048142" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:53.223166 525 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 137dae39f8f54be5af6354e2b6226adc
W20260202 02:52:53.223428 857 leader_election.cc:343] T b9fc33e8efc04734a951f7fcf9048142 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:53.215655 1039 tablet_service.cc:1552] Processing DeleteTablet for tablet 477c599f8c9b4eb0b435ea50abd5d36b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:53 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:53.225440 528 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet c1958ed658fa44488e416e3312abcc33
I20260202 02:52:53.225960 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2460dbd2a27487db94899b54d519099. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.226226 3038 tablet.cc:1722] T 939fbcaf75c7439d9208bbbf83a72717 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:53.226488 2740 leader_election.cc:290] T 07580ad1d049475a9645acfad0b1d219 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:53.226732 3032 raft_consensus.cc:515] T c8a273ac8cd344719a48fd81945bed0a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.227382 3032 leader_election.cc:290] T c8a273ac8cd344719a48fd81945bed0a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:53.227759 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "07580ad1d049475a9645acfad0b1d219" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:53.228569 998 leader_election.cc:343] T 07580ad1d049475a9645acfad0b1d219 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:53.229166 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c8a273ac8cd344719a48fd81945bed0a" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:53.229395 773 raft_consensus.cc:2393] T c8a273ac8cd344719a48fd81945bed0a P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1f05820b6034228bdad884cc348716d in current term 1: Already voted for candidate 1521a6d262b7451bbef8debe970dd6b5 in this term.
I20260202 02:52:53.230044 1174 tablet_bootstrap.cc:492] T dc90eeb79ded4739883ad98928476680 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:53.230178 1174 ts_tablet_manager.cc:1403] T dc90eeb79ded4739883ad98928476680 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.000s
I20260202 02:52:53.231010 1174 raft_consensus.cc:359] T dc90eeb79ded4739883ad98928476680 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.231184 1174 raft_consensus.cc:385] T dc90eeb79ded4739883ad98928476680 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.231241 1174 raft_consensus.cc:740] T dc90eeb79ded4739883ad98928476680 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
W20260202 02:52:53.231245 526 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fc2bc0599d7144dba486f56d7ce52f2c consensus state has no leader
I20260202 02:52:53.231419 1174 consensus_queue.cc:260] T dc90eeb79ded4739883ad98928476680 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.232302 1882 ts_tablet_manager.cc:1950] T 73f8db8f30f14d369d2227c9be7c4647 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:53.232465 1174 ts_tablet_manager.cc:1434] T dc90eeb79ded4739883ad98928476680 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:53.232761 1174 tablet_bootstrap.cc:492] T abfe7f76b2424c4d9ab07b29b66ec103 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:53.233253 915 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c8a273ac8cd344719a48fd81945bed0a" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:53.233897 902 tablet_service.cc:1552] Processing DeleteTablet for tablet 137dae39f8f54be5af6354e2b6226adc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:53 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:53.234222 1173 raft_consensus.cc:359] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.235131 1174 tablet_bootstrap.cc:654] T abfe7f76b2424c4d9ab07b29b66ec103 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:53.238149 2794 consensus_queue.cc:1048] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260202 02:52:53.239730 1174 tablet_bootstrap.cc:492] T abfe7f76b2424c4d9ab07b29b66ec103 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:53.240424 1174 ts_tablet_manager.cc:1403] T abfe7f76b2424c4d9ab07b29b66ec103 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260202 02:52:53.241540 1174 raft_consensus.cc:359] T abfe7f76b2424c4d9ab07b29b66ec103 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
W20260202 02:52:53.242573 999 leader_election.cc:343] T 07580ad1d049475a9645acfad0b1d219 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 07580ad1d049475a9645acfad0b1d219
I20260202 02:52:53.242873 1173 raft_consensus.cc:385] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.241166 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b9fc33e8efc04734a951f7fcf9048142" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:53.243453 1174 raft_consensus.cc:385] T abfe7f76b2424c4d9ab07b29b66ec103 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.243536 1174 raft_consensus.cc:740] T abfe7f76b2424c4d9ab07b29b66ec103 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:53.243508 1039 tablet_service.cc:1552] Processing DeleteTablet for tablet 137dae39f8f54be5af6354e2b6226adc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:53 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:53.243656 999 leader_election.cc:304] T 07580ad1d049475a9645acfad0b1d219 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:53.244180 999 leader_election.cc:304] T c8a273ac8cd344719a48fd81945bed0a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:53.244742 2740 raft_consensus.cc:3060] T c8a273ac8cd344719a48fd81945bed0a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:53.244844 3032 raft_consensus.cc:2749] T 07580ad1d049475a9645acfad0b1d219 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:53.245021 1173 raft_consensus.cc:740] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:53.245262 1173 consensus_queue.cc:260] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.245865 1173 ts_tablet_manager.cc:1434] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.035s user 0.000s sys 0.002s
I20260202 02:52:53.248287 2740 raft_consensus.cc:2749] T c8a273ac8cd344719a48fd81945bed0a P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:53.248575 1174 consensus_queue.cc:260] T abfe7f76b2424c4d9ab07b29b66ec103 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.248723 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 767) took 3928 ms. Trace:
I20260202 02:52:53.248847 1882 rpcz_store.cc:276] 0202 02:52:49.314776 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:49.314839 (+ 63us) service_pool.cc:225] Handling call
0202 02:52:53.033159 (+3718320us) ts_tablet_manager.cc:1151] Deleting tablet 73f8db8f30f14d369d2227c9be7c4647
0202 02:52:53.033172 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:53.064576 (+ 31404us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:53.177843 (+113267us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:53.243208 (+ 65365us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3718113}
I20260202 02:52:53.249802 1882 tablet_replica.cc:333] T ccd6bc8bd0504bf18d108c73685ef9ff P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:53.250159 1174 ts_tablet_manager.cc:1434] T abfe7f76b2424c4d9ab07b29b66ec103 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260202 02:52:53.250497 1174 tablet_bootstrap.cc:492] T 99e0b5e9a9874aec886155e0d614dec2 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:53.250029 1882 raft_consensus.cc:2243] T ccd6bc8bd0504bf18d108c73685ef9ff P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:53.251595 1882 raft_consensus.cc:2272] T ccd6bc8bd0504bf18d108c73685ef9ff P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:53.251781 1173 tablet_bootstrap.cc:492] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:53.252247 1174 tablet_bootstrap.cc:654] T 99e0b5e9a9874aec886155e0d614dec2 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:53.253502 1173 tablet_bootstrap.cc:654] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:53.254503 859 leader_election.cc:343] T b9fc33e8efc04734a951f7fcf9048142 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:53.254626 859 leader_election.cc:304] T b9fc33e8efc04734a951f7fcf9048142 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:53.254848 2929 raft_consensus.cc:493] T f5727f9b9bbb4e77abfef065c51606a1 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:53.254973 2929 raft_consensus.cc:515] T f5727f9b9bbb4e77abfef065c51606a1 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:53.255524 2929 leader_election.cc:290] T f5727f9b9bbb4e77abfef065c51606a1 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:53.255657 2929 raft_consensus.cc:2749] T b9fc33e8efc04734a951f7fcf9048142 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:53.256520 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f5727f9b9bbb4e77abfef065c51606a1" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:53.256997 2878 raft_consensus.cc:493] T 59ad016e15ec4a488921eb564029b689 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:53.257148 2878 raft_consensus.cc:515] T 59ad016e15ec4a488921eb564029b689 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.257608 2633 raft_consensus.cc:493] T 7ea7a5d8a92b4ac0903483627258ae66 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:53.257690 2878 leader_election.cc:290] T 59ad016e15ec4a488921eb564029b689 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:53.257735 2633 raft_consensus.cc:515] T 7ea7a5d8a92b4ac0903483627258ae66 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.257872 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 73f8db8f30f14d369d2227c9be7c4647 (table default.Table_0276ce9d8aaa44fa832bb88a8b1f9446 [id=a8eac3717bad4d1496acd1c857a510a6]) successfully deleted
I20260202 02:52:53.258246 2633 leader_election.cc:290] T 7ea7a5d8a92b4ac0903483627258ae66 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:53.257467 1882 ts_tablet_manager.cc:1916] T ccd6bc8bd0504bf18d108c73685ef9ff P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:53.259272 859 leader_election.cc:343] T f5727f9b9bbb4e77abfef065c51606a1 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:53.259467 3038 tablet.cc:1722] T f6746ccbf37e477196a9706396d1e3fd P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:53.263231 2964 raft_consensus.cc:493] T 9a1d7d173c41462c8e32a95b494bb708 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:53.263389 2964 raft_consensus.cc:515] T 9a1d7d173c41462c8e32a95b494bb708 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.263808 418 master-stress-test.cc:577] Killing test thread 1/25
I20260202 02:52:53.265482 2964 leader_election.cc:290] T 9a1d7d173c41462c8e32a95b494bb708 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:53.266013 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9a1d7d173c41462c8e32a95b494bb708" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:53.266510 859 leader_election.cc:343] T 9a1d7d173c41462c8e32a95b494bb708 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:53.268435 418 master-stress-test.cc:579] Killed test thread 1/25
I20260202 02:52:53.268540 418 master-stress-test.cc:577] Killing test thread 2/25
I20260202 02:52:53.268622 418 master-stress-test.cc:579] Killed test thread 2/25
I20260202 02:52:53.268663 418 master-stress-test.cc:577] Killing test thread 3/25
I20260202 02:52:53.268733 418 master-stress-test.cc:579] Killed test thread 3/25
I20260202 02:52:53.268788 418 master-stress-test.cc:577] Killing test thread 4/25
I20260202 02:52:53.268863 418 master-stress-test.cc:579] Killed test thread 4/25
I20260202 02:52:53.268908 418 master-stress-test.cc:577] Killing test thread 5/25
I20260202 02:52:53.269366 1174 tablet_bootstrap.cc:492] T 99e0b5e9a9874aec886155e0d614dec2 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:53.269877 1174 ts_tablet_manager.cc:1403] T 99e0b5e9a9874aec886155e0d614dec2 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.000s
I20260202 02:52:53.270695 1173 tablet_bootstrap.cc:492] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:53.270937 1173 ts_tablet_manager.cc:1403] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.025s user 0.003s sys 0.001s
I20260202 02:52:53.271608 1173 raft_consensus.cc:359] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.271811 1173 raft_consensus.cc:385] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.271889 1173 raft_consensus.cc:740] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:53.271618 1174 raft_consensus.cc:359] T 99e0b5e9a9874aec886155e0d614dec2 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.272383 1174 raft_consensus.cc:385] T 99e0b5e9a9874aec886155e0d614dec2 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.272579 1174 raft_consensus.cc:740] T 99e0b5e9a9874aec886155e0d614dec2 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:53.272966 1173 consensus_queue.cc:260] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.273010 1174 consensus_queue.cc:260] T 99e0b5e9a9874aec886155e0d614dec2 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.273814 1174 ts_tablet_manager.cc:1434] T 99e0b5e9a9874aec886155e0d614dec2 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260202 02:52:53.277920 523 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_070200300d5d4886b185f9a99aa77100"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:52:53.277779 764 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9a1d7d173c41462c8e32a95b494bb708" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:53.278578 523 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_070200300d5d4886b185f9a99aa77100 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:53.279572 1174 tablet_bootstrap.cc:492] T 528d4269f91a44679acfad16e9fbbe32 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:53.279735 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f5727f9b9bbb4e77abfef065c51606a1" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:53.280670 902 tablet_service.cc:1552] Processing DeleteTablet for tablet b9fc33e8efc04734a951f7fcf9048142 with delete_type TABLET_DATA_DELETED (replaced by tablet 8a3ebe597da64da48db881cef46cfe4e) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:53.280879 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 7f9dfac19c04475aadb45e95a6474eb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:53.281009 915 raft_consensus.cc:1275] T 16b47a2831f54d809ed0b16deaec7e3d P 1521a6d262b7451bbef8debe970dd6b5 [term 3 FOLLOWER]: Refusing update from remote peer b1f05820b6034228bdad884cc348716d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260202 02:52:53.281576 526 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_dfa22e5a08544023b64919ccf651c07a" } modify_external_catalogs: true
I20260202 02:52:53.281801 2964 raft_consensus.cc:493] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:53.281948 526 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_dfa22e5a08544023b64919ccf651c07a" } modify_external_catalogs: true
I20260202 02:52:53.281930 2964 raft_consensus.cc:515] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.282471 2964 leader_election.cc:290] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:53.282884 1174 tablet_bootstrap.cc:654] T 528d4269f91a44679acfad16e9fbbe32 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:53.283152 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8b0368e03f644f1db28dbb57c95c48c1" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:53.283420 1060 raft_consensus.cc:2468] T 8b0368e03f644f1db28dbb57c95c48c1 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
I20260202 02:52:53.283893 859 leader_election.cc:304] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:53.284224 2964 raft_consensus.cc:2804] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:53.284353 2964 raft_consensus.cc:493] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:53.284494 2964 raft_consensus.cc:3060] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:53.285902 527 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47430:
table { table_name: "default.Table_e7dff4eb18124743848a9f61ba603f6f" } new_table_name: "default.Table_c70528658e2f49bcb97d3c9c815379ec" modify_external_catalogs: true
W20260202 02:52:53.287317 512 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 943835f6f43e41c99480295fbbb65adb (table default.Table_fefb919a83c14757b698b97adee1adb9 [id=3c66c314193a4a6996d890303a273bd3], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 943835f6f43e41c99480295fbbb65adb consensus state has no leader
I20260202 02:52:53.287240 2964 raft_consensus.cc:515] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.287926 2964 leader_election.cc:290] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:53.288784 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8b0368e03f644f1db28dbb57c95c48c1" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:53.289089 1060 raft_consensus.cc:3060] T 8b0368e03f644f1db28dbb57c95c48c1 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:53.290819 528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_1403dbd979384147afa3d43063fdc558"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:53.291396 528 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1403dbd979384147afa3d43063fdc558 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:53.291945 1060 raft_consensus.cc:2468] T 8b0368e03f644f1db28dbb57c95c48c1 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
I20260202 02:52:53.293227 859 leader_election.cc:304] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:53.293557 2964 raft_consensus.cc:2804] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:53.293735 2964 raft_consensus.cc:697] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:53.293974 2964 consensus_queue.cc:237] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.295432 520 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47430:
name: "default.Table_76012bec9c634dbb848a193ad3c6d4e0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:52:53.296029 520 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_76012bec9c634dbb848a193ad3c6d4e0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:52:53.299659 1173 ts_tablet_manager.cc:1434] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20260202 02:52:53.300199 1173 tablet_bootstrap.cc:492] T 25a407001f964da6b51f42375136c79a P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:53.301998 1174 tablet_bootstrap.cc:492] T 528d4269f91a44679acfad16e9fbbe32 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:53.302098 1173 tablet_bootstrap.cc:654] T 25a407001f964da6b51f42375136c79a P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:53.302493 857 leader_election.cc:343] T 9a1d7d173c41462c8e32a95b494bb708 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:53.302760 857 leader_election.cc:304] T 9a1d7d173c41462c8e32a95b494bb708 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:53.303027 2929 raft_consensus.cc:2749] T 9a1d7d173c41462c8e32a95b494bb708 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:53.302152 1174 ts_tablet_manager.cc:1403] T 528d4269f91a44679acfad16e9fbbe32 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.023s user 0.003s sys 0.000s
W20260202 02:52:53.303434 857 leader_election.cc:343] T f5727f9b9bbb4e77abfef065c51606a1 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:53.303555 857 leader_election.cc:304] T f5727f9b9bbb4e77abfef065c51606a1 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:53.303763 2929 raft_consensus.cc:2749] T f5727f9b9bbb4e77abfef065c51606a1 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:53.303980 1175 tablet_bootstrap.cc:492] T c977c146bac34638874b7a990604a0d5 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:53.304116 1175 ts_tablet_manager.cc:1403] T c977c146bac34638874b7a990604a0d5 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.087s user 0.000s sys 0.016s
I20260202 02:52:53.304411 2794 consensus_queue.cc:1048] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260202 02:52:53.304930 1175 raft_consensus.cc:359] T c977c146bac34638874b7a990604a0d5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.304976 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8b0368e03f644f1db28dbb57c95c48c1" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:53.305101 1175 raft_consensus.cc:385] T c977c146bac34638874b7a990604a0d5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.305157 1175 raft_consensus.cc:740] T c977c146bac34638874b7a990604a0d5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:53.305176 773 raft_consensus.cc:2468] T 8b0368e03f644f1db28dbb57c95c48c1 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
I20260202 02:52:53.305371 1175 consensus_queue.cc:260] T c977c146bac34638874b7a990604a0d5 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.305985 1175 ts_tablet_manager.cc:1434] T c977c146bac34638874b7a990604a0d5 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260202 02:52:53.306288 1175 tablet_bootstrap.cc:492] T cd1b6662da254889ae4bbe8d0b1ef1bb P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:53.307992 1175 tablet_bootstrap.cc:654] T cd1b6662da254889ae4bbe8d0b1ef1bb P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:53.309485 1174 raft_consensus.cc:359] T 528d4269f91a44679acfad16e9fbbe32 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.309669 1174 raft_consensus.cc:385] T 528d4269f91a44679acfad16e9fbbe32 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.309711 1174 raft_consensus.cc:740] T 528d4269f91a44679acfad16e9fbbe32 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:53.311203 1174 consensus_queue.cc:260] T 528d4269f91a44679acfad16e9fbbe32 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.311968 1174 ts_tablet_manager.cc:1434] T 528d4269f91a44679acfad16e9fbbe32 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260202 02:52:53.312264 1174 tablet_bootstrap.cc:492] T eea6bb04ddf24276b46270d009ed78b4 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:53.314044 1174 tablet_bootstrap.cc:654] T eea6bb04ddf24276b46270d009ed78b4 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:53.315629 2929 raft_consensus.cc:493] T 4a518e8ad2e34ec98a0425c7ed3a470a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:53.315788 2929 raft_consensus.cc:515] T 4a518e8ad2e34ec98a0425c7ed3a470a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.316462 2929 leader_election.cc:290] T 4a518e8ad2e34ec98a0425c7ed3a470a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:53.317582 2633 raft_consensus.cc:493] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:53.317740 2633 raft_consensus.cc:515] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:53.318348 2633 leader_election.cc:290] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:53.322021 527 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 916f52df8e0a4838b46772ecab4f852c consensus state has no leader
I20260202 02:52:53.323841 526 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet bb1005103c25423783736ace1866c328
I20260202 02:52:53.323879 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4a518e8ad2e34ec98a0425c7ed3a470a" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:53.324892 859 leader_election.cc:343] T 4a518e8ad2e34ec98a0425c7ed3a470a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:53.326730 418 master-stress-test.cc:579] Killed test thread 5/25
I20260202 02:52:53.327042 418 master-stress-test.cc:577] Killing test thread 6/25
I20260202 02:52:53.327160 418 master-stress-test.cc:579] Killed test thread 6/25
I20260202 02:52:53.327219 418 master-stress-test.cc:577] Killing test thread 7/25
I20260202 02:52:53.327433 418 master-stress-test.cc:579] Killed test thread 7/25
I20260202 02:52:53.327491 418 master-stress-test.cc:577] Killing test thread 8/25
I20260202 02:52:53.327579 418 master-stress-test.cc:579] Killed test thread 8/25
I20260202 02:52:53.327627 418 master-stress-test.cc:577] Killing test thread 9/25
I20260202 02:52:53.327582 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet bb1005103c25423783736ace1866c328 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:53 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:53.327697 418 master-stress-test.cc:579] Killed test thread 9/25
I20260202 02:52:53.327741 418 master-stress-test.cc:577] Killing test thread 10/25
I20260202 02:52:53.327816 418 master-stress-test.cc:579] Killed test thread 10/25
I20260202 02:52:53.327863 418 master-stress-test.cc:577] Killing test thread 11/25
I20260202 02:52:53.327930 418 master-stress-test.cc:579] Killed test thread 11/25
I20260202 02:52:53.327975 418 master-stress-test.cc:577] Killing test thread 12/25
I20260202 02:52:53.328039 418 master-stress-test.cc:579] Killed test thread 12/25
I20260202 02:52:53.328083 418 master-stress-test.cc:577] Killing test thread 13/25
I20260202 02:52:53.328150 418 master-stress-test.cc:579] Killed test thread 13/25
I20260202 02:52:53.328193 418 master-stress-test.cc:577] Killing test thread 14/25
I20260202 02:52:53.328258 418 master-stress-test.cc:579] Killed test thread 14/25
I20260202 02:52:53.328301 418 master-stress-test.cc:577] Killing test thread 15/25
I20260202 02:52:53.328403 418 master-stress-test.cc:579] Killed test thread 15/25
I20260202 02:52:53.328447 418 master-stress-test.cc:577] Killing test thread 16/25
I20260202 02:52:53.328516 418 master-stress-test.cc:579] Killed test thread 16/25
I20260202 02:52:53.328572 418 master-stress-test.cc:577] Killing test thread 17/25
I20260202 02:52:53.328640 418 master-stress-test.cc:579] Killed test thread 17/25
I20260202 02:52:53.328684 418 master-stress-test.cc:577] Killing test thread 18/25
I20260202 02:52:53.328756 418 master-stress-test.cc:579] Killed test thread 18/25
I20260202 02:52:53.328804 418 master-stress-test.cc:577] Killing test thread 19/25
I20260202 02:52:53.328876 418 master-stress-test.cc:579] Killed test thread 19/25
I20260202 02:52:53.328922 418 master-stress-test.cc:577] Killing test thread 20/25
I20260202 02:52:53.328902 2794 raft_consensus.cc:493] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:53.328989 418 master-stress-test.cc:579] Killed test thread 20/25
I20260202 02:52:53.329031 418 master-stress-test.cc:577] Killing test thread 21/25
I20260202 02:52:53.329092 2794 raft_consensus.cc:515] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.329869 2794 leader_election.cc:290] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:53.330681 1175 tablet_bootstrap.cc:492] T cd1b6662da254889ae4bbe8d0b1ef1bb P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:53.330889 1175 ts_tablet_manager.cc:1403] T cd1b6662da254889ae4bbe8d0b1ef1bb P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.025s user 0.000s sys 0.003s
I20260202 02:52:53.330857 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f1c8f7e1dc9644a6b6dd2bc264cce8ef" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:53.331516 998 leader_election.cc:343] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:53.331693 1175 raft_consensus.cc:359] T cd1b6662da254889ae4bbe8d0b1ef1bb P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:53.331848 1175 raft_consensus.cc:385] T cd1b6662da254889ae4bbe8d0b1ef1bb P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.331897 1175 raft_consensus.cc:740] T cd1b6662da254889ae4bbe8d0b1ef1bb P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:53.332104 1175 consensus_queue.cc:260] T cd1b6662da254889ae4bbe8d0b1ef1bb P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:53.332708 1175 ts_tablet_manager.cc:1434] T cd1b6662da254889ae4bbe8d0b1ef1bb P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260202 02:52:53.333007 1175 tablet_bootstrap.cc:492] T bbcc723a8079443db455ae20fb0d585d P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:53.334625 1175 tablet_bootstrap.cc:654] T bbcc723a8079443db455ae20fb0d585d P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:53.340482 1174 tablet_bootstrap.cc:492] T eea6bb04ddf24276b46270d009ed78b4 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:53.340664 1174 ts_tablet_manager.cc:1403] T eea6bb04ddf24276b46270d009ed78b4 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.028s user 0.004s sys 0.000s
I20260202 02:52:53.341485 1174 raft_consensus.cc:359] T eea6bb04ddf24276b46270d009ed78b4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.341679 1174 raft_consensus.cc:385] T eea6bb04ddf24276b46270d009ed78b4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.341740 1174 raft_consensus.cc:740] T eea6bb04ddf24276b46270d009ed78b4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:53.343921 755 tablet_service.cc:1552] Processing DeleteTablet for tablet 6f2739c054ce4e6b87634eced940d548 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:53.344367 755 tablet_service.cc:1552] Processing DeleteTablet for tablet 3857680ccbdc4dd4ad8247da96254434 with delete_type TABLET_DATA_DELETED (replaced by tablet 60feac4d4fb841c0834f030db0f599d1) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:53.344756 755 tablet_service.cc:1552] Processing DeleteTablet for tablet 51ba0bcd498043c383c493f45ad1fe17 with delete_type TABLET_DATA_DELETED (Replaced by dac0f74c3d3f410f8222764ba9cd7f98 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:53.345162 755 tablet_service.cc:1552] Processing DeleteTablet for tablet 9ee7687df15d494eaf4150db580de7e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:53.345547 755 tablet_service.cc:1552] Processing DeleteTablet for tablet 7a44585a217b48f39f247bae3ca2e255 with delete_type TABLET_DATA_DELETED (Replaced by d8d706b1bf49494c871aee3ddc16e40c at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:53.347532 755 tablet_service.cc:1552] Processing DeleteTablet for tablet 25a407001f964da6b51f42375136c79a with delete_type TABLET_DATA_DELETED (Replaced by b3e095157aec4096a2ea5db96872ae12 at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:53.348016 755 tablet_service.cc:1552] Processing DeleteTablet for tablet e2edc6660b9a41e39e9529627fafcb03 with delete_type TABLET_DATA_DELETED (Replaced by 5692ca1e36894c3194f0fcff18d014ba at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:53.348125 1174 consensus_queue.cc:260] T eea6bb04ddf24276b46270d009ed78b4 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.348742 755 tablet_service.cc:1505] Processing CreateTablet for tablet d1944de086c24058b00df38d2b3d7cfd (DEFAULT_TABLE table=default.Table_0bdc711d3bce45af9149f191f5d61ee2 [id=ddce2a099a1c40538900ce30b1921708]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.349138 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1944de086c24058b00df38d2b3d7cfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.349470 1175 tablet_bootstrap.cc:492] T bbcc723a8079443db455ae20fb0d585d P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:53.349610 1175 ts_tablet_manager.cc:1403] T bbcc723a8079443db455ae20fb0d585d P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.003s
I20260202 02:52:53.350412 1175 raft_consensus.cc:359] T bbcc723a8079443db455ae20fb0d585d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.350607 1175 raft_consensus.cc:385] T bbcc723a8079443db455ae20fb0d585d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.350678 1175 raft_consensus.cc:740] T bbcc723a8079443db455ae20fb0d585d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:53.350878 1175 consensus_queue.cc:260] T bbcc723a8079443db455ae20fb0d585d P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.351431 1174 ts_tablet_manager.cc:1434] T eea6bb04ddf24276b46270d009ed78b4 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20260202 02:52:53.351752 1174 tablet_bootstrap.cc:492] T 2616e1bbce4049dbb584550ccc150288 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:53.353506 1174 tablet_bootstrap.cc:654] T 2616e1bbce4049dbb584550ccc150288 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:53.355934 1175 ts_tablet_manager.cc:1434] T bbcc723a8079443db455ae20fb0d585d P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260202 02:52:53.356292 1175 tablet_bootstrap.cc:492] T 09504070c9ec4bcfbb59e3fb4c189775 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
W20260202 02:52:53.358126 715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 279988us
I20260202 02:52:53.359589 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2b352d9a815e4bccafb7e2251500c5bc" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:53.359849 1062 raft_consensus.cc:2468] T 2b352d9a815e4bccafb7e2251500c5bc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:53.361442 3042 tablet.cc:1722] T f6746ccbf37e477196a9706396d1e3fd P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260202 02:52:53.365523 1038 tablet_service.cc:1505] Processing CreateTablet for tablet 9ee7687df15d494eaf4150db580de7e3 (DEFAULT_TABLE table=default.Table_fe61f7e6b8c74ca1acb5bdf456cc6bf5 [id=4d58a6308fa74606ba10c68bdcfea1ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.366005 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ee7687df15d494eaf4150db580de7e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.367393 1175 tablet_bootstrap.cc:654] T 09504070c9ec4bcfbb59e3fb4c189775 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:53.368641 1039 tablet_service.cc:1505] Processing CreateTablet for tablet d677b534cba94354b6d0379a4efc4432 (DEFAULT_TABLE table=default.Table_a8c4f67a58b74bf8bb3716174ed5654f [id=e7975854ed2f4810a2d438b590267584]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.369061 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d677b534cba94354b6d0379a4efc4432. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.370054 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 1f171e07223b4895873a513ee05c8f21 (DEFAULT_TABLE table=default.Table_37b8a9e1666a4ddbaf5a13d828e9702c [id=e0cea4989f66443faf074eacf434658c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.370412 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f171e07223b4895873a513ee05c8f21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.376377 1174 tablet_bootstrap.cc:492] T 2616e1bbce4049dbb584550ccc150288 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:53.376425 526 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 939fbcaf75c7439d9208bbbf83a72717 (table default.Table_12d68a317c8e4d809dbbe77c095a1f6e [id=7219522ffc3849faab94d754a42a8511]). Expected version 1 got 0
I20260202 02:52:53.376564 1174 ts_tablet_manager.cc:1403] T 2616e1bbce4049dbb584550ccc150288 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.025s user 0.003s sys 0.000s
I20260202 02:52:53.376807 526 catalog_manager.cc:5746] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257) does not have the latest schema for tablet 44c8d5b74c8e4c499f7f68e8185efc17 (table default.Table_f48ed1c7e02349adad7ce22c0e51ed81 [id=46b5d47377474ff9bac2acdb26b08c65]). Expected version 3 got 0
I20260202 02:52:53.377370 1174 raft_consensus.cc:359] T 2616e1bbce4049dbb584550ccc150288 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.377558 1174 raft_consensus.cc:385] T 2616e1bbce4049dbb584550ccc150288 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.377610 1174 raft_consensus.cc:740] T 2616e1bbce4049dbb584550ccc150288 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:53.377698 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3284213e4b0447b49a7718ba896d3fa8" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:53.377787 1174 consensus_queue.cc:260] T 2616e1bbce4049dbb584550ccc150288 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.378021 1060 raft_consensus.cc:2468] T 3284213e4b0447b49a7718ba896d3fa8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:53.378283 1174 ts_tablet_manager.cc:1434] T 2616e1bbce4049dbb584550ccc150288 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:53.383571 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet f1c8f7e1dc9644a6b6dd2bc264cce8ef with delete_type TABLET_DATA_DELETED (replaced by tablet 21fe651f45bf43609cf972bef5eca149) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:53.383874 1039 tablet_service.cc:1552] Processing DeleteTablet for tablet 80eb3013f5354022a2dd78bb294a8497 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:53.383983 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 07580ad1d049475a9645acfad0b1d219 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:53.384104 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet 0138ba7326674741b0729324735f1de7 with delete_type TABLET_DATA_DELETED (replaced by tablet 6671d3b545eb4371bfa71fe703c214f2) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:53.378628 1174 tablet_bootstrap.cc:492] T e2edc6660b9a41e39e9529627fafcb03 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:53.380147 773 raft_consensus.cc:1275] T b76d1837813c46e8a0925b49939d8884 P 1f442ef85029461091de847118c34c0d [term 3 FOLLOWER]: Refusing update from remote peer 1521a6d262b7451bbef8debe970dd6b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260202 02:52:53.387820 1174 tablet_bootstrap.cc:654] T e2edc6660b9a41e39e9529627fafcb03 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:53.380267 3043 raft_consensus.cc:493] T 5e20842de8cb4355b732685605c88834 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:53.388312 3043 raft_consensus.cc:515] T 5e20842de8cb4355b732685605c88834 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:53.388808 765 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4a518e8ad2e34ec98a0425c7ed3a470a" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:53.389722 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8b0368e03f644f1db28dbb57c95c48c1" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:53.390296 775 raft_consensus.cc:3060] T 8b0368e03f644f1db28dbb57c95c48c1 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:53.393646 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "59ad016e15ec4a488921eb564029b689" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:53.400830 1174 tablet_bootstrap.cc:492] T e2edc6660b9a41e39e9529627fafcb03 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:53.402273 1182 ts_tablet_manager.cc:1929] T eea6bb04ddf24276b46270d009ed78b4 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:53.403162 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7ea7a5d8a92b4ac0903483627258ae66" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:53.403386 1062 raft_consensus.cc:2468] T 7ea7a5d8a92b4ac0903483627258ae66 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:53.404160 1174 ts_tablet_manager.cc:1403] T e2edc6660b9a41e39e9529627fafcb03 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.026s user 0.004s sys 0.000s
I20260202 02:52:53.405042 1174 raft_consensus.cc:359] T e2edc6660b9a41e39e9529627fafcb03 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.405208 1174 raft_consensus.cc:385] T e2edc6660b9a41e39e9529627fafcb03 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.405261 1174 raft_consensus.cc:740] T e2edc6660b9a41e39e9529627fafcb03 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:53.408242 1174 consensus_queue.cc:260] T e2edc6660b9a41e39e9529627fafcb03 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.409008 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 725) took 5467 ms. Trace:
I20260202 02:52:53.409152 1180 rpcz_store.cc:276] 0202 02:52:47.941031 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:47.945397 (+ 4366us) service_pool.cc:225] Handling call
0202 02:52:53.408854 (+5463457us) ts_tablet_manager.cc:1151] Deleting tablet 09a8aa4dd70b4fe2806293d2620e521c
0202 02:52:53.408867 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:53.408987 (+ 120us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5348424}
I20260202 02:52:53.409404 1174 ts_tablet_manager.cc:1434] T e2edc6660b9a41e39e9529627fafcb03 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260202 02:52:53.409618 1180 tablet_replica.cc:333] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:53.409718 1174 tablet_bootstrap.cc:492] T 7a44585a217b48f39f247bae3ca2e255 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:53.409885 1180 raft_consensus.cc:2243] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:53.410193 1180 pending_rounds.cc:70] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d: Trying to abort 1 pending ops.
I20260202 02:52:53.410370 1180 pending_rounds.cc:77] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923152168869888 op_type: NO_OP noop_request { }
I20260202 02:52:53.410540 1180 raft_consensus.cc:2889] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:53.410674 1180 raft_consensus.cc:2272] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:53.411268 1180 ts_tablet_manager.cc:1916] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:53.411689 1882 ts_tablet_manager.cc:1929] T ccd6bc8bd0504bf18d108c73685ef9ff P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:53.411885 1882 log.cc:1199] T ccd6bc8bd0504bf18d108c73685ef9ff P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/ccd6bc8bd0504bf18d108c73685ef9ff
I20260202 02:52:53.412318 1882 ts_tablet_manager.cc:1950] T ccd6bc8bd0504bf18d108c73685ef9ff P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:53.415307 1054 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cb634d4bae554ef3b39eeaa9958adc6c" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:53.415544 1054 raft_consensus.cc:2468] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 2.
I20260202 02:52:53.418476 1054 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5e20842de8cb4355b732685605c88834" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:53.421653 526 master_service.cc:954] ReplaceTablet: received request to replace tablet 2becb2f65a124c3990ea3a8318ff1d8c from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:53.421144 3025 tablet.cc:1722] T c050a2882f764dda8df2a7df6547f149 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260202 02:52:53.431692 526 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 2becb2f65a124c3990ea3a8318ff1d8c
I20260202 02:52:53.432818 526 catalog_manager.cc:6724] ReplaceTablet: tablet 2becb2f65a124c3990ea3a8318ff1d8c deleted and replaced by tablet 3590bfe57ac346b48e059a8c2deaafd9
I20260202 02:52:53.433888 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet 2becb2f65a124c3990ea3a8318ff1d8c with delete_type TABLET_DATA_DELETED (replaced by tablet 3590bfe57ac346b48e059a8c2deaafd9) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:53.438371 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 768) took 4022 ms. Trace:
I20260202 02:52:53.439153 1882 rpcz_store.cc:276] 0202 02:52:49.416272 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:49.416379 (+ 107us) service_pool.cc:225] Handling call
0202 02:52:53.249201 (+3832822us) ts_tablet_manager.cc:1151] Deleting tablet ccd6bc8bd0504bf18d108c73685ef9ff
0202 02:52:53.249213 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:53.268745 (+ 19532us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:53.411541 (+142796us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:53.438316 (+ 26775us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3832553}
I20260202 02:52:53.439950 528 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_12d68a317c8e4d809dbbe77c095a1f6e [id=7219522ffc3849faab94d754a42a8511] alter complete (version 1)
I20260202 02:52:53.440065 1882 tablet_replica.cc:333] T 2c31b45dc8004eae993e972e9faa3ff8 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:53.440225 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet ccd6bc8bd0504bf18d108c73685ef9ff (table default.Table_5ecb81038c9f48438e74fe874fc83e74 [id=c28e30416db3407b8c20c68091f392b3]) successfully deleted
I20260202 02:52:53.440565 1882 raft_consensus.cc:2243] T 2c31b45dc8004eae993e972e9faa3ff8 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:53.441056 1882 raft_consensus.cc:2272] T 2c31b45dc8004eae993e972e9faa3ff8 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:53.442997 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet 2616e1bbce4049dbb584550ccc150288 with delete_type TABLET_DATA_DELETED (replaced by tablet 8dd62de8e8234d6c9ce380b1aadc097d) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:53.411371 1174 tablet_bootstrap.cc:654] T 7a44585a217b48f39f247bae3ca2e255 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:53.451139 1173 tablet_bootstrap.cc:492] T 25a407001f964da6b51f42375136c79a P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:53.451414 1173 ts_tablet_manager.cc:1403] T 25a407001f964da6b51f42375136c79a P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.151s user 0.004s sys 0.000s
I20260202 02:52:53.452495 1173 raft_consensus.cc:359] T 25a407001f964da6b51f42375136c79a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.452807 1173 raft_consensus.cc:385] T 25a407001f964da6b51f42375136c79a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.452965 1173 raft_consensus.cc:740] T 25a407001f964da6b51f42375136c79a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:53.453502 1173 consensus_queue.cc:260] T 25a407001f964da6b51f42375136c79a P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.466876 1882 ts_tablet_manager.cc:1916] T 2c31b45dc8004eae993e972e9faa3ff8 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:53.467374 526 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_e699c7601f354d13907893985186dc4d [id=5f8923fc0fb84598a1fb7c3a8abbb8cb] alter complete (version 3)
I20260202 02:52:53.474004 1173 ts_tablet_manager.cc:1434] T 25a407001f964da6b51f42375136c79a P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.022s user 0.001s sys 0.001s
I20260202 02:52:53.413738 715 leader_election.cc:304] T 2b352d9a815e4bccafb7e2251500c5bc P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:53.414444 743 tablet_service.cc:1552] Processing DeleteTablet for tablet 81abd5ef66cc4f0bb37844c6ff5c256d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:53.414536 745 tablet_service.cc:1505] Processing CreateTablet for tablet 3f7da8bb0c2f46bb9c12ffc8c87a5967 (DEFAULT_TABLE table=default.Table_a1067e637b814119a16c9a4d8039b649 [id=e726212e556a441396ac44b3e563cac0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.476553 1173 tablet_bootstrap.cc:492] T f3eebefa74584c1791d52377f5ba1290 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:53.476799 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f7da8bb0c2f46bb9c12ffc8c87a5967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.479725 1173 tablet_bootstrap.cc:654] T f3eebefa74584c1791d52377f5ba1290 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:53.414785 747 tablet_service.cc:1552] Processing DeleteTablet for tablet 7ea7a5d8a92b4ac0903483627258ae66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:53.414889 751 tablet_service.cc:1505] Processing CreateTablet for tablet fc2bc0599d7144dba486f56d7ce52f2c (DEFAULT_TABLE table=default.Table_82031bd5aec3477bb4ccf0183961a528 [id=6561bda3e33745c1aa7bdd672ef9755a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.480221 3040 raft_consensus.cc:2804] T 2b352d9a815e4bccafb7e2251500c5bc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:53.480293 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc2bc0599d7144dba486f56d7ce52f2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.480348 3040 raft_consensus.cc:493] T 2b352d9a815e4bccafb7e2251500c5bc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:53.480436 3040 raft_consensus.cc:3060] T 2b352d9a815e4bccafb7e2251500c5bc P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:53.480720 715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 67006.4us
W20260202 02:52:53.480953 715 leader_election.cc:343] T 79c73f4a92044bafb5b8384229cb3a4d P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Not found: Tablet not found: 79c73f4a92044bafb5b8384229cb3a4d
W20260202 02:52:53.481160 715 consensus_peers.cc:597] T ccd6bc8bd0504bf18d108c73685ef9ff P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:53.481305 715 leader_election.cc:304] T 3284213e4b0447b49a7718ba896d3fa8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:53.483057 2881 raft_consensus.cc:2804] T 3284213e4b0447b49a7718ba896d3fa8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:53.483175 2881 raft_consensus.cc:493] T 3284213e4b0447b49a7718ba896d3fa8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:53.483270 2881 raft_consensus.cc:3060] T 3284213e4b0447b49a7718ba896d3fa8 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:53.415234 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 2f5aae089fcc492f9ce55f84674a452c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:20 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:53.415442 750 tablet_service.cc:1552] Processing DeleteTablet for tablet 0138ba7326674741b0729324735f1de7 with delete_type TABLET_DATA_DELETED (replaced by tablet 6671d3b545eb4371bfa71fe703c214f2) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:53.415688 753 tablet_service.cc:1552] Processing DeleteTablet for tablet 2b352d9a815e4bccafb7e2251500c5bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:22 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:53.415745 740 tablet_service.cc:1552] Processing DeleteTablet for tablet 587cfd0ae62f4cfd937faf6890e78f4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:53.416059 748 tablet_service.cc:1552] Processing DeleteTablet for tablet ac2ab12f5040488aa306cb69b107b485 with delete_type TABLET_DATA_DELETED (replaced by tablet 898778f8f0b548ec89cb1417d6f07926) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:53.416244 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 477c599f8c9b4eb0b435ea50abd5d36b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:53 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:53.416404 744 tablet_service.cc:1505] Processing CreateTablet for tablet 7792d72dd1054e56b11ed10b365167cf (DEFAULT_TABLE table=default.Table_d64d39313f1448de84c04387a8a6d33d [id=7c34cab2c3b04f678c7511063985eadb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.416566 759 tablet_service.cc:1552] Processing DeleteTablet for tablet d3f6a30e08244a07a1f81b5e98bc6965 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:53 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:53.416771 742 tablet_service.cc:1552] Processing DeleteTablet for tablet bb1005103c25423783736ace1866c328 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:53 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:53.416857 746 tablet_service.cc:1505] Processing CreateTablet for tablet f2460dbd2a27487db94899b54d519099 (DEFAULT_TABLE table=default.Table_e0bea82391b342c1a81564e3b2789f94 [id=f155562b32254df8aa3c0b0c840b1e17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.417039 758 tablet_service.cc:1505] Processing CreateTablet for tablet 9ee7687df15d494eaf4150db580de7e3 (DEFAULT_TABLE table=default.Table_fe61f7e6b8c74ca1acb5bdf456cc6bf5 [id=4d58a6308fa74606ba10c68bdcfea1ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.483956 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7792d72dd1054e56b11ed10b365167cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.417088 752 tablet_service.cc:1505] Processing CreateTablet for tablet d677b534cba94354b6d0379a4efc4432 (DEFAULT_TABLE table=default.Table_a8c4f67a58b74bf8bb3716174ed5654f [id=e7975854ed2f4810a2d438b590267584]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.417035 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 137dae39f8f54be5af6354e2b6226adc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:53 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:53.417574 741 tablet_service.cc:1505] Processing CreateTablet for tablet 1f171e07223b4895873a513ee05c8f21 (DEFAULT_TABLE table=default.Table_37b8a9e1666a4ddbaf5a13d828e9702c [id=e0cea4989f66443faf074eacf434658c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.420708 755 tablet_service.cc:1552] Processing DeleteTablet for tablet 08fbd094d4df46cf82a694b4e1f30c6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:52 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:53.389066 3043 leader_election.cc:290] T 5e20842de8cb4355b732685605c88834 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:53.443456 2944 raft_consensus.cc:493] T eea6bb04ddf24276b46270d009ed78b4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:53.484797 2944 raft_consensus.cc:515] T eea6bb04ddf24276b46270d009ed78b4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.485421 2944 leader_election.cc:290] T eea6bb04ddf24276b46270d009ed78b4 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:53.484211 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ee7687df15d494eaf4150db580de7e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.485713 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f171e07223b4895873a513ee05c8f21. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:53.486796 715 leader_election.cc:343] T 59ad016e15ec4a488921eb564029b689 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:53.487015 715 leader_election.cc:304] T 7ea7a5d8a92b4ac0903483627258ae66 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:53.487270 715 leader_election.cc:304] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:53.488446 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d677b534cba94354b6d0379a4efc4432. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.488969 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2460dbd2a27487db94899b54d519099. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.492231 3041 raft_consensus.cc:493] T 51ba0bcd498043c383c493f45ad1fe17 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:53.492424 3041 raft_consensus.cc:515] T 51ba0bcd498043c383c493f45ad1fe17 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.493095 3041 leader_election.cc:290] T 51ba0bcd498043c383c493f45ad1fe17 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:53.493253 3041 raft_consensus.cc:2804] T 7ea7a5d8a92b4ac0903483627258ae66 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:53.493340 3041 raft_consensus.cc:493] T 7ea7a5d8a92b4ac0903483627258ae66 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:53.493410 3041 raft_consensus.cc:3060] T 7ea7a5d8a92b4ac0903483627258ae66 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:53.496506 2944 raft_consensus.cc:493] T abfe7f76b2424c4d9ab07b29b66ec103 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:53.496676 2944 raft_consensus.cc:515] T abfe7f76b2424c4d9ab07b29b66ec103 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.497368 2944 leader_election.cc:290] T abfe7f76b2424c4d9ab07b29b66ec103 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:53.497505 3043 raft_consensus.cc:2804] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260202 02:52:53.497609 3043 raft_consensus.cc:493] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:53.497689 3043 raft_consensus.cc:3060] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Advancing to term 3
W20260202 02:52:53.498268 715 leader_election.cc:343] T 5e20842de8cb4355b732685605c88834 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:53.499532 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "eea6bb04ddf24276b46270d009ed78b4" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:53.500303 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "51ba0bcd498043c383c493f45ad1fe17" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:53.500707 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "abfe7f76b2424c4d9ab07b29b66ec103" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:53.508237 775 raft_consensus.cc:2468] T 8b0368e03f644f1db28dbb57c95c48c1 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 1.
I20260202 02:52:53.510076 1182 log.cc:1199] T eea6bb04ddf24276b46270d009ed78b4 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/eea6bb04ddf24276b46270d009ed78b4
I20260202 02:52:53.433429 1175 tablet_bootstrap.cc:492] T 09504070c9ec4bcfbb59e3fb4c189775 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:53.513006 1175 ts_tablet_manager.cc:1403] T 09504070c9ec4bcfbb59e3fb4c189775 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.157s user 0.010s sys 0.002s
I20260202 02:52:53.513849 1175 raft_consensus.cc:359] T 09504070c9ec4bcfbb59e3fb4c189775 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:53.514027 1175 raft_consensus.cc:385] T 09504070c9ec4bcfbb59e3fb4c189775 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.514091 1175 raft_consensus.cc:740] T 09504070c9ec4bcfbb59e3fb4c189775 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:53.514293 1175 consensus_queue.cc:260] T 09504070c9ec4bcfbb59e3fb4c189775 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:53.515477 3081 raft_consensus.cc:493] T 7f323175ee254df789bcb04e65047dff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:53.516263 3081 raft_consensus.cc:515] T 7f323175ee254df789bcb04e65047dff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.516891 1175 ts_tablet_manager.cc:1434] T 09504070c9ec4bcfbb59e3fb4c189775 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260202 02:52:53.518021 2881 raft_consensus.cc:515] T 3284213e4b0447b49a7718ba896d3fa8 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.518646 2881 leader_election.cc:290] T 3284213e4b0447b49a7718ba896d3fa8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:53.518878 715 leader_election.cc:343] T eea6bb04ddf24276b46270d009ed78b4 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:53.520634 3040 raft_consensus.cc:515] T 2b352d9a815e4bccafb7e2251500c5bc P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.520943 3081 leader_election.cc:290] T 7f323175ee254df789bcb04e65047dff P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:53.521591 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7f323175ee254df789bcb04e65047dff" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:53.522374 859 leader_election.cc:343] T 7f323175ee254df789bcb04e65047dff P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:53.444460 857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 63121.7us
I20260202 02:52:53.441622 3076 raft_consensus.cc:493] T cd1b6662da254889ae4bbe8d0b1ef1bb P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:53.528124 3041 raft_consensus.cc:515] T 7ea7a5d8a92b4ac0903483627258ae66 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.528182 3076 raft_consensus.cc:515] T cd1b6662da254889ae4bbe8d0b1ef1bb P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:53.528951 3041 leader_election.cc:290] T 7ea7a5d8a92b4ac0903483627258ae66 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:53.528978 3076 leader_election.cc:290] T cd1b6662da254889ae4bbe8d0b1ef1bb P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:53.534009 3040 leader_election.cc:290] T 2b352d9a815e4bccafb7e2251500c5bc P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:53.538774 1174 tablet_bootstrap.cc:492] T 7a44585a217b48f39f247bae3ca2e255 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:53.539081 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cd1b6662da254889ae4bbe8d0b1ef1bb" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:53.539429 715 leader_election.cc:343] T 51ba0bcd498043c383c493f45ad1fe17 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:53.539829 859 leader_election.cc:343] T cd1b6662da254889ae4bbe8d0b1ef1bb P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:53.540591 1175 tablet_bootstrap.cc:492] T 3dcf4f794ec14b2f9d181d015fc437fb P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:53.542088 1038 tablet_service.cc:1505] Processing CreateTablet for tablet 4fe1960a80d24cf5a63a49997cb60220 (DEFAULT_TABLE table=default.Table_1f4c697ec6ea43519f5330b2274557f3 [id=9489fb84a43448b59dbbcf4e820036b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.542246 1175 tablet_bootstrap.cc:654] T 3dcf4f794ec14b2f9d181d015fc437fb P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:53.542474 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fe1960a80d24cf5a63a49997cb60220. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.542496 3043 raft_consensus.cc:515] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:53.549309 3081 consensus_queue.cc:1048] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260202 02:52:53.549412 526 catalog_manager.cc:5746] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539) does not have the latest schema for tablet 09504070c9ec4bcfbb59e3fb4c189775 (table default.Table_abd89cee70a04e2aa3a0d59b7eb943c0 [id=4c17da59a0554cee93ce4e626838a2c6]). Expected version 1 got 0
I20260202 02:52:53.550365 3040 raft_consensus.cc:493] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260202 02:52:53.550235 857 leader_election.cc:343] T 4a518e8ad2e34ec98a0425c7ed3a470a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:53.550415 766 raft_consensus.cc:3060] T e2edc6660b9a41e39e9529627fafcb03 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:53.550644 857 leader_election.cc:304] T 4a518e8ad2e34ec98a0425c7ed3a470a P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:53.550886 3081 raft_consensus.cc:2749] T 4a518e8ad2e34ec98a0425c7ed3a470a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:53.551627 886 tablet_service.cc:1552] Processing DeleteTablet for tablet bb1005103c25423783736ace1866c328 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:53 UTC) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:53.550369 715 leader_election.cc:343] T abfe7f76b2424c4d9ab07b29b66ec103 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:53.552239 886 tablet_service.cc:1505] Processing CreateTablet for tablet 9ee7687df15d494eaf4150db580de7e3 (DEFAULT_TABLE table=default.Table_fe61f7e6b8c74ca1acb5bdf456cc6bf5 [id=4d58a6308fa74606ba10c68bdcfea1ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.552708 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ee7687df15d494eaf4150db580de7e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.558717 1060 raft_consensus.cc:1275] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Refusing update from remote peer 1521a6d262b7451bbef8debe970dd6b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260202 02:52:53.559984 3081 raft_consensus.cc:493] T c977c146bac34638874b7a990604a0d5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:53.560154 3081 raft_consensus.cc:515] T c977c146bac34638874b7a990604a0d5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.561003 3081 leader_election.cc:290] T c977c146bac34638874b7a990604a0d5 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:53.561231 3043 leader_election.cc:290] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 3 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:53.561231 3040 raft_consensus.cc:515] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:53.561865 3040 leader_election.cc:290] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:53.559475 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3284213e4b0447b49a7718ba896d3fa8" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:53.565706 1062 raft_consensus.cc:3060] T 3284213e4b0447b49a7718ba896d3fa8 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:53.560829 1054 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7ea7a5d8a92b4ac0903483627258ae66" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:53.561105 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c977c146bac34638874b7a990604a0d5" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:53.568428 1054 raft_consensus.cc:3060] T 7ea7a5d8a92b4ac0903483627258ae66 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:53.564635 3032 consensus_queue.cc:1048] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:53.568938 886 tablet_service.cc:1505] Processing CreateTablet for tablet d677b534cba94354b6d0379a4efc4432 (DEFAULT_TABLE table=default.Table_a8c4f67a58b74bf8bb3716174ed5654f [id=e7975854ed2f4810a2d438b590267584]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.569332 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d677b534cba94354b6d0379a4efc4432. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:53.569767 859 leader_election.cc:343] T c977c146bac34638874b7a990604a0d5 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:53.570928 1054 raft_consensus.cc:2468] T 7ea7a5d8a92b4ac0903483627258ae66 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:53.565387 1061 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cb634d4bae554ef3b39eeaa9958adc6c" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:53.568252 1062 raft_consensus.cc:2468] T 3284213e4b0447b49a7718ba896d3fa8 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:53.574260 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "20ea39bc5d3d46f5a4ba427277d6c6a3" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:53.574478 1062 raft_consensus.cc:2468] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 2.
I20260202 02:52:53.576972 902 tablet_service.cc:1505] Processing CreateTablet for tablet 1f171e07223b4895873a513ee05c8f21 (DEFAULT_TABLE table=default.Table_37b8a9e1666a4ddbaf5a13d828e9702c [id=e0cea4989f66443faf074eacf434658c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.577435 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f171e07223b4895873a513ee05c8f21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.579021 715 leader_election.cc:304] T 7ea7a5d8a92b4ac0903483627258ae66 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:53.579296 3040 raft_consensus.cc:2804] T 7ea7a5d8a92b4ac0903483627258ae66 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:53.579530 3040 raft_consensus.cc:697] T 7ea7a5d8a92b4ac0903483627258ae66 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:53.579813 3040 consensus_queue.cc:237] T 7ea7a5d8a92b4ac0903483627258ae66 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.581329 715 leader_election.cc:304] T 3284213e4b0447b49a7718ba896d3fa8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:53.581604 2881 raft_consensus.cc:2804] T 3284213e4b0447b49a7718ba896d3fa8 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:53.581720 2881 raft_consensus.cc:697] T 3284213e4b0447b49a7718ba896d3fa8 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:53.581977 2881 consensus_queue.cc:237] T 3284213e4b0447b49a7718ba896d3fa8 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.582957 3080 consensus_queue.cc:1048] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260202 02:52:53.583433 3080 raft_consensus.cc:493] T c17c85759e964f0ba3abcaf5fe938fba P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:53.583581 3080 raft_consensus.cc:515] T c17c85759e964f0ba3abcaf5fe938fba P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.584255 3080 leader_election.cc:290] T c17c85759e964f0ba3abcaf5fe938fba P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:53.511879 1182 ts_tablet_manager.cc:1950] T eea6bb04ddf24276b46270d009ed78b4 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:53.586010 1182 tablet_replica.cc:333] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:53.586272 1182 raft_consensus.cc:2243] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:53.586581 1182 raft_consensus.cc:2272] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:53.587148 896 tablet_service.cc:1552] Processing DeleteTablet for tablet 2becb2f65a124c3990ea3a8318ff1d8c with delete_type TABLET_DATA_DELETED (replaced by tablet 3590bfe57ac346b48e059a8c2deaafd9) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:53.587937 896 tablet_service.cc:1505] Processing CreateTablet for tablet 4fe1960a80d24cf5a63a49997cb60220 (DEFAULT_TABLE table=default.Table_1f4c697ec6ea43519f5330b2274557f3 [id=9489fb84a43448b59dbbcf4e820036b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.588616 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fe1960a80d24cf5a63a49997cb60220. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.589985 715 leader_election.cc:304] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:53.591194 3043 raft_consensus.cc:2804] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260202 02:52:53.591289 3043 raft_consensus.cc:493] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:53.591385 3043 raft_consensus.cc:3060] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Advancing to term 3
I20260202 02:52:53.592208 1061 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c17c85759e964f0ba3abcaf5fe938fba" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:53.593412 859 leader_election.cc:343] T c17c85759e964f0ba3abcaf5fe938fba P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:53.594089 3043 raft_consensus.cc:515] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:53.594853 3043 leader_election.cc:290] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 3 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:53.600780 1054 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "20ea39bc5d3d46f5a4ba427277d6c6a3" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:53.600911 1038 tablet_service.cc:1505] Processing CreateTablet for tablet 9f805fa060d54d7e9067b5120f3d28ad (DEFAULT_TABLE table=default.Table_0e7ceba4956f408c8a9fc0f61fa67c90 [id=c0c6499675894350ac008b534d030c93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.601006 1054 raft_consensus.cc:3060] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Advancing to term 3
I20260202 02:52:53.601305 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f805fa060d54d7e9067b5120f3d28ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.603914 1054 raft_consensus.cc:2468] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 3.
W20260202 02:52:53.605369 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet b8f2b34b4f1e4ff2b2be4ff40aeb0c66 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b8f2b34b4f1e4ff2b2be4ff40aeb0c66 already in progress: creating tablet
W20260202 02:52:53.605934 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet 09a8aa4dd70b4fe2806293d2620e521c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 09a8aa4dd70b4fe2806293d2620e521c already in progress: creating tablet
I20260202 02:52:53.612092 902 tablet_service.cc:1552] Processing DeleteTablet for tablet c977c146bac34638874b7a990604a0d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:53.616874 2979 raft_consensus.cc:493] T b427c7e897c64977b924e7d586420e69 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:53.617058 2979 raft_consensus.cc:515] T b427c7e897c64977b924e7d586420e69 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.617733 2979 leader_election.cc:290] T b427c7e897c64977b924e7d586420e69 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:53.619195 765 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b427c7e897c64977b924e7d586420e69" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:53.619448 765 raft_consensus.cc:2468] T b427c7e897c64977b924e7d586420e69 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:53.619947 998 leader_election.cc:304] T b427c7e897c64977b924e7d586420e69 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:53.620429 2979 raft_consensus.cc:2804] T b427c7e897c64977b924e7d586420e69 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:53.620539 2979 raft_consensus.cc:493] T b427c7e897c64977b924e7d586420e69 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:53.620610 2979 raft_consensus.cc:3060] T b427c7e897c64977b924e7d586420e69 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:53.623262 2979 raft_consensus.cc:515] T b427c7e897c64977b924e7d586420e69 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.624549 902 tablet_service.cc:1552] Processing DeleteTablet for tablet cd1b6662da254889ae4bbe8d0b1ef1bb with delete_type TABLET_DATA_DELETED (replaced by tablet 83f30c7f03504cdbb379fc0469324306) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:53.625667 918 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b427c7e897c64977b924e7d586420e69" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:53.625849 918 raft_consensus.cc:2468] T b427c7e897c64977b924e7d586420e69 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:53.626199 763 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cd1b6662da254889ae4bbe8d0b1ef1bb" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:53.626855 763 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c977c146bac34638874b7a990604a0d5" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:53.626971 764 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c17c85759e964f0ba3abcaf5fe938fba" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:53.628178 923 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b427c7e897c64977b924e7d586420e69" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:53.628386 923 raft_consensus.cc:3060] T b427c7e897c64977b924e7d586420e69 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:53.628818 999 leader_election.cc:343] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: f1c8f7e1dc9644a6b6dd2bc264cce8ef
I20260202 02:52:53.628976 999 leader_election.cc:304] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:53.629776 3032 raft_consensus.cc:2749] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:53.631229 923 raft_consensus.cc:2468] T b427c7e897c64977b924e7d586420e69 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:53.631700 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7f323175ee254df789bcb04e65047dff" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:53.632437 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet eea6bb04ddf24276b46270d009ed78b4 (table default.Table_1366b68811c14d28b1785266d6f65a3a [id=b7139f5062824d66af2dbd62469a9c03]) successfully deleted
I20260202 02:52:53.633169 3035 tablet.cc:1722] T c050a2882f764dda8df2a7df6547f149 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260202 02:52:53.634500 765 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b427c7e897c64977b924e7d586420e69" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:53.634709 765 raft_consensus.cc:3060] T b427c7e897c64977b924e7d586420e69 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:53.639078 3080 consensus_queue.cc:1048] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:53.640192 773 raft_consensus.cc:1275] T 25765fb687294e9db1289f76530dd877 P 1f442ef85029461091de847118c34c0d [term 3 FOLLOWER]: Refusing update from remote peer b1f05820b6034228bdad884cc348716d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260202 02:52:53.641037 1060 raft_consensus.cc:1275] T b76d1837813c46e8a0925b49939d8884 P b1f05820b6034228bdad884cc348716d [term 3 FOLLOWER]: Refusing update from remote peer 1521a6d262b7451bbef8debe970dd6b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260202 02:52:53.640985 2794 consensus_queue.cc:1048] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260202 02:52:53.641594 2925 consensus_queue.cc:1048] T b76d1837813c46e8a0925b49939d8884 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260202 02:52:53.642915 765 raft_consensus.cc:2468] T b427c7e897c64977b924e7d586420e69 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:53.644129 998 leader_election.cc:304] T b427c7e897c64977b924e7d586420e69 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:53.644447 2794 raft_consensus.cc:2804] T b427c7e897c64977b924e7d586420e69 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:53.644604 2794 raft_consensus.cc:697] T b427c7e897c64977b924e7d586420e69 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:53.644855 2794 consensus_queue.cc:237] T b427c7e897c64977b924e7d586420e69 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.645246 2979 leader_election.cc:290] T b427c7e897c64977b924e7d586420e69 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:53.647166 3043 raft_consensus.cc:493] T ac2ab12f5040488aa306cb69b107b485 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:53.647439 3043 raft_consensus.cc:515] T ac2ab12f5040488aa306cb69b107b485 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.648146 3043 leader_election.cc:290] T ac2ab12f5040488aa306cb69b107b485 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:53.655076 3041 raft_consensus.cc:493] T 02b434fa058b45529ff3938b053263c6 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:53.655287 3041 raft_consensus.cc:515] T 02b434fa058b45529ff3938b053263c6 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.655889 3041 leader_election.cc:290] T 02b434fa058b45529ff3938b053263c6 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:53.656121 1174 ts_tablet_manager.cc:1403] T 7a44585a217b48f39f247bae3ca2e255 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.130s user 0.011s sys 0.005s
W20260202 02:52:53.661064 714 leader_election.cc:343] T 79c73f4a92044bafb5b8384229cb3a4d P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 79c73f4a92044bafb5b8384229cb3a4d
I20260202 02:52:53.668445 714 leader_election.cc:304] T 79c73f4a92044bafb5b8384229cb3a4d P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:53.661823 1174 raft_consensus.cc:359] T 7a44585a217b48f39f247bae3ca2e255 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.669893 1174 raft_consensus.cc:385] T 7a44585a217b48f39f247bae3ca2e255 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.670356 1174 raft_consensus.cc:740] T 7a44585a217b48f39f247bae3ca2e255 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:53.670763 1174 consensus_queue.cc:260] T 7a44585a217b48f39f247bae3ca2e255 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.668282 1182 ts_tablet_manager.cc:1916] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:53.674536 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2b352d9a815e4bccafb7e2251500c5bc" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:53.674818 1059 raft_consensus.cc:3060] T 2b352d9a815e4bccafb7e2251500c5bc P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:53.677575 1029 tablet_service.cc:1505] Processing CreateTablet for tablet ae9a02dd8c3647f19abe28647ee2b3ce (DEFAULT_TABLE table=default.Table_5a703dc84eb34674bdc5793ac7295098 [id=62c3c39fb73249349cee49b86d535612]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.677997 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae9a02dd8c3647f19abe28647ee2b3ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.679315 526 catalog_manager.cc:5746] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403) does not have the latest schema for tablet f6746ccbf37e477196a9706396d1e3fd (table default.Table_4f72bc6302d84055ba28584ec5e3ace9 [id=0c8385b71ceb44199472e1ce640b54e0]). Expected version 1 got 0
I20260202 02:52:53.679984 1174 ts_tablet_manager.cc:1434] T 7a44585a217b48f39f247bae3ca2e255 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.024s user 0.002s sys 0.000s
I20260202 02:52:53.680285 1174 tablet_bootstrap.cc:492] T b4a4c75a906f40beb896bf6fff705ae6 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:53.685281 2979 raft_consensus.cc:493] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:53.685516 2979 raft_consensus.cc:515] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.686414 2979 leader_election.cc:290] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:53.686978 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ac2ab12f5040488aa306cb69b107b485" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:53.687074 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2616e1bbce4049dbb584550ccc150288" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:53.687156 1046 raft_consensus.cc:2468] T ac2ab12f5040488aa306cb69b107b485 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:53.687291 766 raft_consensus.cc:2468] T 2616e1bbce4049dbb584550ccc150288 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:53.687703 998 leader_election.cc:304] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:53.688115 2979 raft_consensus.cc:2804] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:53.688217 2979 raft_consensus.cc:493] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:53.688288 2979 raft_consensus.cc:3060] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:53.689742 1038 tablet_service.cc:1505] Processing CreateTablet for tablet d23d2b52f058471bb0da6e2608016fa5 (DEFAULT_TABLE table=default.Table_502b1ec344f64ad78b8613e843e3f73b [id=baeb590c153e4a79928e5e33161f4f19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.690142 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d23d2b52f058471bb0da6e2608016fa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.691399 2979 raft_consensus.cc:515] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.692943 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2616e1bbce4049dbb584550ccc150288" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:53.693186 766 raft_consensus.cc:3060] T 2616e1bbce4049dbb584550ccc150288 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:53.693935 2979 leader_election.cc:290] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:53.694262 715 leader_election.cc:304] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:53.694583 3041 raft_consensus.cc:2804] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [term 3 FOLLOWER]: Leader election won for term 3
I20260202 02:52:53.694685 3041 raft_consensus.cc:697] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [term 3 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:53.695183 2881 consensus_queue.cc:1048] T 587cfd0ae62f4cfd937faf6890e78f4a P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:53.700888 2979 raft_consensus.cc:493] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:53.701051 2979 raft_consensus.cc:515] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.702150 2979 leader_election.cc:290] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:53.702276 765 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7a44585a217b48f39f247bae3ca2e255" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:53.702514 765 raft_consensus.cc:2468] T 7a44585a217b48f39f247bae3ca2e255 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:53.703001 998 leader_election.cc:304] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:53.703275 715 leader_election.cc:304] T ac2ab12f5040488aa306cb69b107b485 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:53.703570 2911 raft_consensus.cc:2804] T ac2ab12f5040488aa306cb69b107b485 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:53.703646 2911 raft_consensus.cc:493] T ac2ab12f5040488aa306cb69b107b485 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:53.703706 2911 raft_consensus.cc:3060] T ac2ab12f5040488aa306cb69b107b485 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:53.704713 715 leader_election.cc:343] T 02b434fa058b45529ff3938b053263c6 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Not found: Tablet not found: 02b434fa058b45529ff3938b053263c6
W20260202 02:52:53.705008 715 consensus_peers.cc:597] T 2c31b45dc8004eae993e972e9faa3ff8 P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:53.705495 2979 raft_consensus.cc:2804] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:53.705791 1046 raft_consensus.cc:1275] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Refusing update from remote peer 1f442ef85029461091de847118c34c0d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260202 02:52:53.706269 2979 raft_consensus.cc:493] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:53.706775 2979 raft_consensus.cc:3060] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:53.710021 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cb634d4bae554ef3b39eeaa9958adc6c" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:53.712852 2979 raft_consensus.cc:515] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.717170 1029 tablet_service.cc:1505] Processing CreateTablet for tablet fdc72096ee4d440789899e6e71580746 (DEFAULT_TABLE table=default.Table_4e6c3886f58b49698a00d45b957f4c9a [id=b5cc03b77f414304b29c5fb4bb7dc3ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.719740 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7a44585a217b48f39f247bae3ca2e255" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:53.720028 2979 leader_election.cc:290] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:53.720819 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5e20842de8cb4355b732685605c88834" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:53.721045 914 raft_consensus.cc:2468] T 5e20842de8cb4355b732685605c88834 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:53.721233 775 raft_consensus.cc:3060] T 7a44585a217b48f39f247bae3ca2e255 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:53.725634 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "51ba0bcd498043c383c493f45ad1fe17" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:53.726001 914 raft_consensus.cc:2468] T 51ba0bcd498043c383c493f45ad1fe17 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:53.728170 1039 tablet_service.cc:1505] Processing CreateTablet for tablet 54591d3d809648de930f7b4221c94e49 (DEFAULT_TABLE table=default.Table_1366b68811c14d28b1785266d6f65a3a [id=b7139f5062824d66af2dbd62469a9c03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.729971 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 271b7d97a03b4f389e4c3ba991ba522a (DEFAULT_TABLE table=default.Table_dbb40510d128424fa8b9b668c4f372c8 [id=1393a89f57284d85b02fb576dc011916]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.732749 1882 ts_tablet_manager.cc:1929] T 2c31b45dc8004eae993e972e9faa3ff8 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:53.732983 1882 log.cc:1199] T 2c31b45dc8004eae993e972e9faa3ff8 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/2c31b45dc8004eae993e972e9faa3ff8
I20260202 02:52:53.733438 1882 ts_tablet_manager.cc:1950] T 2c31b45dc8004eae993e972e9faa3ff8 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:53.735740 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cb634d4bae554ef3b39eeaa9958adc6c" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:53.736111 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "20ea39bc5d3d46f5a4ba427277d6c6a3" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:53.753142 1174 tablet_bootstrap.cc:654] T b4a4c75a906f40beb896bf6fff705ae6 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:53.754113 3041 consensus_queue.cc:237] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:53.756383 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 2becb2f65a124c3990ea3a8318ff1d8c with delete_type TABLET_DATA_DELETED (replaced by tablet 3590bfe57ac346b48e059a8c2deaafd9) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:53.756995 751 tablet_service.cc:1505] Processing CreateTablet for tablet 4fe1960a80d24cf5a63a49997cb60220 (DEFAULT_TABLE table=default.Table_1f4c697ec6ea43519f5330b2274557f3 [id=9489fb84a43448b59dbbcf4e820036b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.757414 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fe1960a80d24cf5a63a49997cb60220. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.758859 758 tablet_service.cc:1505] Processing CreateTablet for tablet 9f805fa060d54d7e9067b5120f3d28ad (DEFAULT_TABLE table=default.Table_0e7ceba4956f408c8a9fc0f61fa67c90 [id=c0c6499675894350ac008b534d030c93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.758952 746 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a3dac5cd1ce47488a50b4c32eca67c7 with delete_type TABLET_DATA_DELETED (replaced by tablet 365417e050304073b5de887082fc699b) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:53.759228 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f805fa060d54d7e9067b5120f3d28ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.759275 746 tablet_service.cc:1552] Processing DeleteTablet for tablet 59ad016e15ec4a488921eb564029b689 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:21 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:53.759543 746 tablet_service.cc:1552] Processing DeleteTablet for tablet 5e20842de8cb4355b732685605c88834 with delete_type TABLET_DATA_DELETED (replaced by tablet 34a01a4ce1be4ad08d911763de158073) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:53.759989 746 tablet_service.cc:1505] Processing CreateTablet for tablet d23d2b52f058471bb0da6e2608016fa5 (DEFAULT_TABLE table=default.Table_502b1ec344f64ad78b8613e843e3f73b [id=baeb590c153e4a79928e5e33161f4f19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.760030 745 tablet_service.cc:1552] Processing DeleteTablet for tablet dc90eeb79ded4739883ad98928476680 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:53.760124 741 tablet_service.cc:1505] Processing CreateTablet for tablet ae9a02dd8c3647f19abe28647ee2b3ce (DEFAULT_TABLE table=default.Table_5a703dc84eb34674bdc5793ac7295098 [id=62c3c39fb73249349cee49b86d535612]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.760294 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d23d2b52f058471bb0da6e2608016fa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.760520 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae9a02dd8c3647f19abe28647ee2b3ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.761539 752 tablet_service.cc:1505] Processing CreateTablet for tablet 54591d3d809648de930f7b4221c94e49 (DEFAULT_TABLE table=default.Table_1366b68811c14d28b1785266d6f65a3a [id=b7139f5062824d66af2dbd62469a9c03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.762006 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54591d3d809648de930f7b4221c94e49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.763793 1175 tablet_bootstrap.cc:492] T 3dcf4f794ec14b2f9d181d015fc437fb P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:53.768761 526 master_service.cc:954] ReplaceTablet: received request to replace tablet 8cc5e23a86a3467c89794ae8c403230e from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:53.770468 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54591d3d809648de930f7b4221c94e49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.770779 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 271b7d97a03b4f389e4c3ba991ba522a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.778208 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdc72096ee4d440789899e6e71580746. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.779738 2881 consensus_queue.cc:1048] T 8bb14bf551f441dcbaaff7057b51b2be P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260202 02:52:53.786201 526 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 8cc5e23a86a3467c89794ae8c403230e
I20260202 02:52:53.787292 2881 raft_consensus.cc:493] T f3eebefa74584c1791d52377f5ba1290 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:53.787614 2881 raft_consensus.cc:515] T f3eebefa74584c1791d52377f5ba1290 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.788218 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 8cc5e23a86a3467c89794ae8c403230e with delete_type TABLET_DATA_DELETED (replaced by tablet 9c1e408a75fe4d948a355f0dede46316) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:53.788904 1173 tablet_bootstrap.cc:492] T f3eebefa74584c1791d52377f5ba1290 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:53.788879 886 tablet_service.cc:1505] Processing CreateTablet for tablet 9f805fa060d54d7e9067b5120f3d28ad (DEFAULT_TABLE table=default.Table_0e7ceba4956f408c8a9fc0f61fa67c90 [id=c0c6499675894350ac008b534d030c93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.789222 526 catalog_manager.cc:6724] ReplaceTablet: tablet 8cc5e23a86a3467c89794ae8c403230e deleted and replaced by tablet 9c1e408a75fe4d948a355f0dede46316
I20260202 02:52:53.789273 1173 ts_tablet_manager.cc:1403] T f3eebefa74584c1791d52377f5ba1290 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.313s user 0.021s sys 0.047s
I20260202 02:52:53.789605 902 tablet_service.cc:1505] Processing CreateTablet for tablet ae9a02dd8c3647f19abe28647ee2b3ce (DEFAULT_TABLE table=default.Table_5a703dc84eb34674bdc5793ac7295098 [id=62c3c39fb73249349cee49b86d535612]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.789963 1173 raft_consensus.cc:359] T f3eebefa74584c1791d52377f5ba1290 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.790055 896 tablet_service.cc:1505] Processing CreateTablet for tablet d23d2b52f058471bb0da6e2608016fa5 (DEFAULT_TABLE table=default.Table_502b1ec344f64ad78b8613e843e3f73b [id=baeb590c153e4a79928e5e33161f4f19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.790153 1173 raft_consensus.cc:385] T f3eebefa74584c1791d52377f5ba1290 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.790205 1173 raft_consensus.cc:740] T f3eebefa74584c1791d52377f5ba1290 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:53.790416 1173 consensus_queue.cc:260] T f3eebefa74584c1791d52377f5ba1290 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.790910 1173 ts_tablet_manager.cc:1434] T f3eebefa74584c1791d52377f5ba1290 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260202 02:52:53.791320 1173 tablet_bootstrap.cc:492] T abfe7f76b2424c4d9ab07b29b66ec103 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:53.793401 1173 tablet_bootstrap.cc:654] T abfe7f76b2424c4d9ab07b29b66ec103 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:53.793556 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 769) took 4377 ms. Trace:
I20260202 02:52:53.794678 1882 rpcz_store.cc:276] 0202 02:52:49.416429 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:49.416691 (+ 262us) service_pool.cc:225] Handling call
0202 02:52:53.439993 (+4023302us) ts_tablet_manager.cc:1151] Deleting tablet 2c31b45dc8004eae993e972e9faa3ff8
0202 02:52:53.440005 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:53.639304 (+199299us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:53.732618 (+ 93314us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:53.793520 (+ 60902us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4023120}
I20260202 02:52:53.795681 1174 tablet_bootstrap.cc:492] T b4a4c75a906f40beb896bf6fff705ae6 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:53.795841 1174 ts_tablet_manager.cc:1403] T b4a4c75a906f40beb896bf6fff705ae6 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.116s user 0.003s sys 0.000s
I20260202 02:52:53.794695 1061 raft_consensus.cc:1275] T e1284fde67bc4d65b067a54c416ccc33 P b1f05820b6034228bdad884cc348716d [term 3 FOLLOWER]: Refusing update from remote peer 1f442ef85029461091de847118c34c0d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260202 02:52:53.797762 2911 raft_consensus.cc:515] T ac2ab12f5040488aa306cb69b107b485 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.797930 766 raft_consensus.cc:2468] T 2616e1bbce4049dbb584550ccc150288 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:53.798379 2911 leader_election.cc:290] T ac2ab12f5040488aa306cb69b107b485 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:53.798847 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 2c31b45dc8004eae993e972e9faa3ff8 (table default.Table_5db99844c1bb484a863a9e289a19da3e [id=f1f6c97fc88340da8bab0347ad6430ab]) successfully deleted
I20260202 02:52:53.664065 1180 ts_tablet_manager.cc:1929] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:53.669772 3043 raft_consensus.cc:2749] T 79c73f4a92044bafb5b8384229cb3a4d P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260202 02:52:53.799612 423 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 45149.7us
I20260202 02:52:53.799856 1180 log.cc:1199] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/f94f25f4059a43b8babedb6b45a08a19
I20260202 02:52:53.803051 1180 ts_tablet_manager.cc:1950] T f94f25f4059a43b8babedb6b45a08a19 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:53.803987 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 742) took 5861 ms. Trace:
I20260202 02:52:53.763952 1175 ts_tablet_manager.cc:1403] T 3dcf4f794ec14b2f9d181d015fc437fb P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.223s user 0.009s sys 0.001s
I20260202 02:52:53.805074 1175 raft_consensus.cc:359] T 3dcf4f794ec14b2f9d181d015fc437fb P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:53.805982 1175 raft_consensus.cc:385] T 3dcf4f794ec14b2f9d181d015fc437fb P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.796304 1059 raft_consensus.cc:2468] T 2b352d9a815e4bccafb7e2251500c5bc P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:53.798559 998 leader_election.cc:304] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:53.807031 2977 raft_consensus.cc:2804] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:53.807137 2977 raft_consensus.cc:697] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:53.807354 2977 consensus_queue.cc:237] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.812176 2740 consensus_queue.cc:1048] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260202 02:52:53.813021 999 consensus_peers.cc:597] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 25765fb687294e9db1289f76530dd877. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:53.819839 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 71dd6890e0e948d09d102296d1bd6464 (DEFAULT_TABLE table=default.Table_616a8ade4b5e481b8d4906b7a2a026d0 [id=98ee1a7044cd46d8b39dfda755ed8a7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.820259 1039 tablet_service.cc:1505] Processing CreateTablet for tablet 154d8a83adb94adab40615da10a9ddb9 (DEFAULT_TABLE table=default.Table_c783f2691903413c90a5e8502f5ddffa [id=5a36f14b8e134e5bafbfd7bd4d04b974]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.820294 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71dd6890e0e948d09d102296d1bd6464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.826785 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 154d8a83adb94adab40615da10a9ddb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.797582 1882 tablet_replica.cc:333] T 474df41e4fe24a818fa45a4ce84d5474 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:53.829017 1882 raft_consensus.cc:2243] T 474df41e4fe24a818fa45a4ce84d5474 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:53.829207 1882 raft_consensus.cc:2272] T 474df41e4fe24a818fa45a4ce84d5474 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:53.829840 1882 ts_tablet_manager.cc:1916] T 474df41e4fe24a818fa45a4ce84d5474 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:53.836022 1029 tablet_service.cc:1505] Processing CreateTablet for tablet f7bf82f274234991b0bab4c3fa3cf4bd (DEFAULT_TABLE table=default.Table_f21cdfd093884683ab4ec1af5345e9d5 [id=ddab3068794747a89a4950dc1ba79b5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.836539 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7bf82f274234991b0bab4c3fa3cf4bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.853546 1173 tablet_bootstrap.cc:492] T abfe7f76b2424c4d9ab07b29b66ec103 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:53.853735 1173 ts_tablet_manager.cc:1403] T abfe7f76b2424c4d9ab07b29b66ec103 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.063s user 0.001s sys 0.002s
I20260202 02:52:53.854507 1173 raft_consensus.cc:359] T abfe7f76b2424c4d9ab07b29b66ec103 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.854688 1173 raft_consensus.cc:385] T abfe7f76b2424c4d9ab07b29b66ec103 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.854733 1173 raft_consensus.cc:740] T abfe7f76b2424c4d9ab07b29b66ec103 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:53.855005 1173 consensus_queue.cc:260] T abfe7f76b2424c4d9ab07b29b66ec103 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.855535 1173 ts_tablet_manager.cc:1434] T abfe7f76b2424c4d9ab07b29b66ec103 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:53.855844 1173 tablet_bootstrap.cc:492] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:53.857316 1173 tablet_bootstrap.cc:654] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:53.862552 1038 tablet_service.cc:1505] Processing CreateTablet for tablet d0e00bed25e740f5b39e023f9ed61d5a (DEFAULT_TABLE table=default.Table_f8d8615b34774b738faeb83039f9601a [id=8882e1fbac2949838c99adb1290f273e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.863474 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0e00bed25e740f5b39e023f9ed61d5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.868088 1173 tablet_bootstrap.cc:492] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:53.868263 1173 ts_tablet_manager.cc:1403] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.000s
I20260202 02:52:53.869244 1173 raft_consensus.cc:359] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.869469 1173 raft_consensus.cc:385] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.869520 1173 raft_consensus.cc:740] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:53.869740 1173 consensus_queue.cc:260] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.870357 1173 ts_tablet_manager.cc:1434] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:53.870779 1173 tablet_bootstrap.cc:492] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:53.872242 1173 tablet_bootstrap.cc:654] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:53.875463 1061 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f3eebefa74584c1791d52377f5ba1290" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:53.875710 1061 raft_consensus.cc:2468] T f3eebefa74584c1791d52377f5ba1290 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:53.876995 1039 tablet_service.cc:1505] Processing CreateTablet for tablet 22c31519d7e642118ff1218e2a1b7621 (DEFAULT_TABLE table=default.Table_a9d14eea240d4bca96de4ea77b5b31f2 [id=c6e5418bf48e43e2adb6d1b60445f0b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.877389 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22c31519d7e642118ff1218e2a1b7621. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.876895 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ac2ab12f5040488aa306cb69b107b485" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:53.879577 1060 raft_consensus.cc:3060] T ac2ab12f5040488aa306cb69b107b485 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:53.882186 1060 raft_consensus.cc:2468] T ac2ab12f5040488aa306cb69b107b485 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:53.886519 1173 tablet_bootstrap.cc:492] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:53.886935 1173 ts_tablet_manager.cc:1403] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.000s
I20260202 02:52:53.887705 1173 raft_consensus.cc:359] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.888127 1173 raft_consensus.cc:385] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.888180 1173 raft_consensus.cc:740] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:53.888414 1173 consensus_queue.cc:260] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.888990 1173 ts_tablet_manager.cc:1434] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:53.889267 1173 tablet_bootstrap.cc:492] T 51ba0bcd498043c383c493f45ad1fe17 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:53.811772 890 tablet_service.cc:1505] Processing CreateTablet for tablet 271b7d97a03b4f389e4c3ba991ba522a (DEFAULT_TABLE table=default.Table_dbb40510d128424fa8b9b668c4f372c8 [id=1393a89f57284d85b02fb576dc011916]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.811990 901 tablet_service.cc:1552] Processing DeleteTablet for tablet 8cc5e23a86a3467c89794ae8c403230e with delete_type TABLET_DATA_DELETED (replaced by tablet 9c1e408a75fe4d948a355f0dede46316) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:53.812276 899 tablet_service.cc:1505] Processing CreateTablet for tablet 71dd6890e0e948d09d102296d1bd6464 (DEFAULT_TABLE table=default.Table_616a8ade4b5e481b8d4906b7a2a026d0 [id=98ee1a7044cd46d8b39dfda755ed8a7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.812595 900 tablet_service.cc:1505] Processing CreateTablet for tablet d0e00bed25e740f5b39e023f9ed61d5a (DEFAULT_TABLE table=default.Table_f8d8615b34774b738faeb83039f9601a [id=8882e1fbac2949838c99adb1290f273e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.812888 894 tablet_service.cc:1505] Processing CreateTablet for tablet 154d8a83adb94adab40615da10a9ddb9 (DEFAULT_TABLE table=default.Table_c783f2691903413c90a5e8502f5ddffa [id=5a36f14b8e134e5bafbfd7bd4d04b974]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.812978 884 tablet_service.cc:1505] Processing CreateTablet for tablet fdc72096ee4d440789899e6e71580746 (DEFAULT_TABLE table=default.Table_4e6c3886f58b49698a00d45b957f4c9a [id=b5cc03b77f414304b29c5fb4bb7dc3ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.816586 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7a44585a217b48f39f247bae3ca2e255" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:53.816627 898 tablet_service.cc:1505] Processing CreateTablet for tablet 54591d3d809648de930f7b4221c94e49 (DEFAULT_TABLE table=default.Table_1366b68811c14d28b1785266d6f65a3a [id=b7139f5062824d66af2dbd62469a9c03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.890542 911 raft_consensus.cc:3060] T 7a44585a217b48f39f247bae3ca2e255 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:53.892269 1173 tablet_bootstrap.cc:654] T 51ba0bcd498043c383c493f45ad1fe17 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:53.893383 911 raft_consensus.cc:2468] T 7a44585a217b48f39f247bae3ca2e255 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:53.816586 915 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7a44585a217b48f39f247bae3ca2e255" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:53.817039 895 tablet_service.cc:1505] Processing CreateTablet for tablet f7bf82f274234991b0bab4c3fa3cf4bd (DEFAULT_TABLE table=default.Table_f21cdfd093884683ab4ec1af5345e9d5 [id=ddab3068794747a89a4950dc1ba79b5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.893764 915 raft_consensus.cc:2376] T 7a44585a217b48f39f247bae3ca2e255 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1. Re-sending same reply.
I20260202 02:52:53.842628 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d23d2b52f058471bb0da6e2608016fa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.894167 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdc72096ee4d440789899e6e71580746. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.843480 1182 ts_tablet_manager.cc:1929] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:53.894335 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0e00bed25e740f5b39e023f9ed61d5a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:53.873564 857 leader_election.cc:343] T cd1b6662da254889ae4bbe8d0b1ef1bb P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:53.903998 3032 raft_consensus.cc:493] T 25a407001f964da6b51f42375136c79a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:53.904403 3025 tablet.cc:1722] T 44c8d5b74c8e4c499f7f68e8185efc17 P b1f05820b6034228bdad884cc348716d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260202 02:52:53.904464 3032 raft_consensus.cc:515] T 25a407001f964da6b51f42375136c79a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.904873 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 6f2739c054ce4e6b87634eced940d548 (DEFAULT_TABLE table=default.Table_cdcc140c2b8d4aabae614d58873dc22a [id=c2c454eb645044839fa81b33b6460558]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.905290 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f2739c054ce4e6b87634eced940d548. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.906884 3032 leader_election.cc:290] T 25a407001f964da6b51f42375136c79a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:53.895555 857 leader_election.cc:304] T cd1b6662da254889ae4bbe8d0b1ef1bb P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:53.908887 3081 raft_consensus.cc:2749] T cd1b6662da254889ae4bbe8d0b1ef1bb P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:53.909780 857 leader_election.cc:343] T c977c146bac34638874b7a990604a0d5 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:53.910187 857 leader_election.cc:304] T c977c146bac34638874b7a990604a0d5 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:53.910737 3081 raft_consensus.cc:2749] T c977c146bac34638874b7a990604a0d5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:53.911548 857 leader_election.cc:343] T c17c85759e964f0ba3abcaf5fe938fba P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:53.911882 857 leader_election.cc:304] T c17c85759e964f0ba3abcaf5fe938fba P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:53.912448 3081 raft_consensus.cc:2749] T c17c85759e964f0ba3abcaf5fe938fba P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:53.913551 857 leader_election.cc:343] T 7f323175ee254df789bcb04e65047dff P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:53.894532 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f805fa060d54d7e9067b5120f3d28ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.914273 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae9a02dd8c3647f19abe28647ee2b3ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.914481 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54591d3d809648de930f7b4221c94e49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.916460 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7bf82f274234991b0bab4c3fa3cf4bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.919639 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 271b7d97a03b4f389e4c3ba991ba522a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.922005 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 84d68594516e4be4a3a1f4ebe9e217a5 (DEFAULT_TABLE table=default.Table_33fc32ec03094e3c8fe78de4f6a26777 [id=85340c1716bd4235bdad7042b53ce58c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.922410 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84d68594516e4be4a3a1f4ebe9e217a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.924477 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 154d8a83adb94adab40615da10a9ddb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.928453 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71dd6890e0e948d09d102296d1bd6464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.930029 1040 tablet_service.cc:1505] Processing CreateTablet for tablet dac0f74c3d3f410f8222764ba9cd7f98 (DEFAULT_TABLE table=default.Table_619d2fcf726a46bfb5e939bbe6f55612 [id=038692c44a0043bab69ab29f77aa13be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.931097 1042 tablet_service.cc:1505] Processing CreateTablet for tablet d8d706b1bf49494c871aee3ddc16e40c (DEFAULT_TABLE table=default.Table_2ee0b536f96e44e4a3a4a9439c7bccc9 [id=d95486b11c494863be7380146ebcf0c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.931510 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8d706b1bf49494c871aee3ddc16e40c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.894423 1182 log.cc:1199] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/7709ea34429e4e91ad4922a3d73ebd14
I20260202 02:52:53.932716 1182 ts_tablet_manager.cc:1950] T 7709ea34429e4e91ad4922a3d73ebd14 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:53.934264 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 900) took 1224 ms. Trace:
I20260202 02:52:53.934352 1182 rpcz_store.cc:276] 0202 02:52:52.710268 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.710404 (+ 136us) service_pool.cc:225] Handling call
0202 02:52:53.585934 (+875530us) ts_tablet_manager.cc:1151] Deleting tablet 7709ea34429e4e91ad4922a3d73ebd14
0202 02:52:53.585945 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:53.676078 (+ 90133us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:53.843245 (+167167us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:53.934249 (+ 91004us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":875385}
I20260202 02:52:53.934724 1182 tablet_replica.cc:333] T dc90eeb79ded4739883ad98928476680 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:53.934891 1182 raft_consensus.cc:2243] T dc90eeb79ded4739883ad98928476680 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:53.935019 1182 raft_consensus.cc:2272] T dc90eeb79ded4739883ad98928476680 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:53.935509 1182 ts_tablet_manager.cc:1916] T dc90eeb79ded4739883ad98928476680 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:53.936195 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dac0f74c3d3f410f8222764ba9cd7f98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.945444 1029 tablet_service.cc:1505] Processing CreateTablet for tablet b3e095157aec4096a2ea5db96872ae12 (DEFAULT_TABLE table=default.Table_48226e7c77da41cdba56e04221eb9689 [id=05e52ce8e94e4975b51849059b904272]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.945881 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3e095157aec4096a2ea5db96872ae12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.957224 1173 tablet_bootstrap.cc:492] T 51ba0bcd498043c383c493f45ad1fe17 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:53.957398 1173 ts_tablet_manager.cc:1403] T 51ba0bcd498043c383c493f45ad1fe17 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.068s user 0.003s sys 0.000s
I20260202 02:52:53.958127 1173 raft_consensus.cc:359] T 51ba0bcd498043c383c493f45ad1fe17 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.958305 1173 raft_consensus.cc:385] T 51ba0bcd498043c383c493f45ad1fe17 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.958359 1173 raft_consensus.cc:740] T 51ba0bcd498043c383c493f45ad1fe17 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:53.958575 1173 consensus_queue.cc:260] T 51ba0bcd498043c383c493f45ad1fe17 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.959249 1173 ts_tablet_manager.cc:1434] T 51ba0bcd498043c383c493f45ad1fe17 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:53.959571 1173 tablet_bootstrap.cc:492] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:53.961050 1040 tablet_service.cc:1505] Processing CreateTablet for tablet f5c0e9f5b3d643659cdde15cbde856ab (DEFAULT_TABLE table=default.Table_0af7d8f82d2b4503aca40ada95dde3fc [id=90956c2303384b6cb0a7f4771e19a9b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.961478 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5c0e9f5b3d643659cdde15cbde856ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.961704 1182 ts_tablet_manager.cc:1929] T dc90eeb79ded4739883ad98928476680 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:53.961829 1182 log.cc:1199] T dc90eeb79ded4739883ad98928476680 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/dc90eeb79ded4739883ad98928476680
I20260202 02:52:53.962137 1182 ts_tablet_manager.cc:1950] T dc90eeb79ded4739883ad98928476680 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:53.963217 1173 tablet_bootstrap.cc:654] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:53.967487 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 901) took 1257 ms. Trace:
I20260202 02:52:53.967617 1182 rpcz_store.cc:276] 0202 02:52:52.710354 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.710417 (+ 63us) service_pool.cc:225] Handling call
0202 02:52:53.934675 (+1224258us) ts_tablet_manager.cc:1151] Deleting tablet dc90eeb79ded4739883ad98928476680
0202 02:52:53.934686 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:53.947845 (+ 13159us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:53.961602 (+ 13757us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:53.967460 (+ 5858us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1224083}
I20260202 02:52:53.967933 1182 tablet_replica.cc:333] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:53.968103 1182 raft_consensus.cc:2243] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:53.969238 1182 raft_consensus.cc:2272] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:53.969831 1182 ts_tablet_manager.cc:1916] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:53.970315 1042 tablet_service.cc:1505] Processing CreateTablet for tablet aa7ebfc320944c818cf7398729b3ae96 (DEFAULT_TABLE table=default.Table_d866f96ecc014b50a4c39f4f93653c9e [id=df985a2551e1405a8b9b02adf4043e7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.970702 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa7ebfc320944c818cf7398729b3ae96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.973407 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 00caa7d9a3ba445c97209bfb68de10d4 (DEFAULT_TABLE table=default.Table_86f302ed9e76457888b61469aa6b2e7f [id=a16d364dde354c58a0cdc51be094b09f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.973814 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00caa7d9a3ba445c97209bfb68de10d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.970261 1039 tablet_service.cc:1505] Processing CreateTablet for tablet 5692ca1e36894c3194f0fcff18d014ba (DEFAULT_TABLE table=default.Table_9d4fbf619567432a88aa9a0c1b19cb3e [id=fff85ebd1bd1415c932f7c9c5a0da42a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.976615 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5692ca1e36894c3194f0fcff18d014ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.981170 1173 tablet_bootstrap.cc:492] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:53.981329 1173 ts_tablet_manager.cc:1403] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.000s
I20260202 02:52:53.982018 1173 raft_consensus.cc:359] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.982174 1173 raft_consensus.cc:385] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.982218 1173 raft_consensus.cc:740] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:53.982410 1173 consensus_queue.cc:260] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.982868 1173 ts_tablet_manager.cc:1434] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260202 02:52:53.984088 1173 tablet_bootstrap.cc:492] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:53.913884 857 leader_election.cc:304] T 7f323175ee254df789bcb04e65047dff P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:53.986390 3081 raft_consensus.cc:2749] T 7f323175ee254df789bcb04e65047dff P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:53.986804 857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 73296.9us
I20260202 02:52:53.987699 2964 consensus_queue.cc:1048] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:53.815986 766 raft_consensus.cc:1240] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [term 3 FOLLOWER]: Rejecting Update request from peer 1521a6d262b7451bbef8debe970dd6b5 for earlier term 2. Current term is 3. Ops: []
I20260202 02:52:53.989178 1180 rpcz_store.cc:276] 0202 02:52:47.942427 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:47.947118 (+ 4691us) service_pool.cc:225] Handling call
0202 02:52:53.409570 (+5462452us) ts_tablet_manager.cc:1151] Deleting tablet f94f25f4059a43b8babedb6b45a08a19
0202 02:52:53.409580 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:53.490271 (+ 80691us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:53.663943 (+173672us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:53.803962 (+140019us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5337559}
I20260202 02:52:53.989647 1180 tablet_replica.cc:333] T 79c73f4a92044bafb5b8384229cb3a4d P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:53.989894 1180 raft_consensus.cc:2243] T 79c73f4a92044bafb5b8384229cb3a4d P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:53.990084 1180 raft_consensus.cc:2272] T 79c73f4a92044bafb5b8384229cb3a4d P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:53.990711 1180 ts_tablet_manager.cc:1916] T 79c73f4a92044bafb5b8384229cb3a4d P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:53.817350 745 tablet_service.cc:1505] Processing CreateTablet for tablet fdc72096ee4d440789899e6e71580746 (DEFAULT_TABLE table=default.Table_4e6c3886f58b49698a00d45b957f4c9a [id=b5cc03b77f414304b29c5fb4bb7dc3ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:53.991658 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdc72096ee4d440789899e6e71580746. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:53.894608 1175 raft_consensus.cc:740] T 3dcf4f794ec14b2f9d181d015fc437fb P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:53.993340 1175 consensus_queue.cc:260] T 3dcf4f794ec14b2f9d181d015fc437fb P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:53.800781 775 raft_consensus.cc:2468] T 7a44585a217b48f39f247bae3ca2e255 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:53.826110 3041 raft_consensus.cc:493] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b1f05820b6034228bdad884cc348716d)
I20260202 02:52:53.995797 3041 raft_consensus.cc:515] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.826582 2881 leader_election.cc:290] T f3eebefa74584c1791d52377f5ba1290 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:53.996755 998 leader_election.cc:304] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:53.832998 1174 raft_consensus.cc:359] T b4a4c75a906f40beb896bf6fff705ae6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.997040 1174 raft_consensus.cc:385] T b4a4c75a906f40beb896bf6fff705ae6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:53.997107 1174 raft_consensus.cc:740] T b4a4c75a906f40beb896bf6fff705ae6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:53.997293 1174 consensus_queue.cc:260] T b4a4c75a906f40beb896bf6fff705ae6 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:53.997771 1174 ts_tablet_manager.cc:1434] T b4a4c75a906f40beb896bf6fff705ae6 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.202s user 0.002s sys 0.000s
I20260202 02:52:53.998064 1174 tablet_bootstrap.cc:492] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:53.999663 1174 tablet_bootstrap.cc:654] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:53.837658 3043 raft_consensus.cc:493] T 99e0b5e9a9874aec886155e0d614dec2 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:54.000458 3043 raft_consensus.cc:515] T 99e0b5e9a9874aec886155e0d614dec2 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.001013 3043 leader_election.cc:290] T 99e0b5e9a9874aec886155e0d614dec2 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:54.001374 3032 raft_consensus.cc:2804] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:54.001493 3032 raft_consensus.cc:697] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:54.001778 3032 consensus_queue.cc:237] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.003366 3041 leader_election.cc:290] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:54.009966 1175 ts_tablet_manager.cc:1434] T 3dcf4f794ec14b2f9d181d015fc437fb P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.206s user 0.002s sys 0.000s
I20260202 02:52:54.010464 1175 tablet_bootstrap.cc:492] T 2f664f6119ba4d28b5ef1c212e611866 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:54.012753 1175 tablet_bootstrap.cc:654] T 2f664f6119ba4d28b5ef1c212e611866 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:54.013531 1174 tablet_bootstrap.cc:492] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:54.013784 1174 ts_tablet_manager.cc:1403] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.000s
I20260202 02:52:54.014591 1174 raft_consensus.cc:359] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.014863 1174 raft_consensus.cc:385] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:54.014956 1174 raft_consensus.cc:740] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:54.015187 1174 consensus_queue.cc:260] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.015851 1174 ts_tablet_manager.cc:1434] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
W20260202 02:52:54.019198 999 leader_election.cc:343] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 2616e1bbce4049dbb584550ccc150288
W20260202 02:52:54.019573 999 leader_election.cc:343] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 2616e1bbce4049dbb584550ccc150288
I20260202 02:52:54.019850 1174 tablet_bootstrap.cc:492] T 25a407001f964da6b51f42375136c79a P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:54.021600 1174 tablet_bootstrap.cc:654] T 25a407001f964da6b51f42375136c79a P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:53.874846 715 consensus_peers.cc:597] T 1116872cc25a4ab5b1a3a6bbba1f0da7 P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1116872cc25a4ab5b1a3a6bbba1f0da7. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:53.907632 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "25a407001f964da6b51f42375136c79a" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:53.942914 3042 tablet.cc:1722] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1f442ef85029461091de847118c34c0d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
W20260202 02:52:53.821302 714 leader_election.cc:343] T 2b352d9a815e4bccafb7e2251500c5bc P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 2b352d9a815e4bccafb7e2251500c5bc
W20260202 02:52:54.023413 714 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 202108us
W20260202 02:52:54.023954 714 leader_election.cc:343] T 3284213e4b0447b49a7718ba896d3fa8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 3284213e4b0447b49a7718ba896d3fa8
W20260202 02:52:54.024206 714 leader_election.cc:343] T 59ad016e15ec4a488921eb564029b689 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 59ad016e15ec4a488921eb564029b689
W20260202 02:52:54.025193 998 leader_election.cc:343] T 25a407001f964da6b51f42375136c79a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:54.024269 714 leader_election.cc:304] T 59ad016e15ec4a488921eb564029b689 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
W20260202 02:52:54.025785 714 leader_election.cc:343] T 7ea7a5d8a92b4ac0903483627258ae66 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 7ea7a5d8a92b4ac0903483627258ae66
W20260202 02:52:54.026242 714 consensus_peers.cc:597] T 587cfd0ae62f4cfd937faf6890e78f4a P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 587cfd0ae62f4cfd937faf6890e78f4a. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:54.026422 714 leader_election.cc:304] T 5e20842de8cb4355b732685605c88834 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
W20260202 02:52:54.026795 714 leader_election.cc:343] T eea6bb04ddf24276b46270d009ed78b4 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: eea6bb04ddf24276b46270d009ed78b4
I20260202 02:52:54.026872 714 leader_election.cc:304] T eea6bb04ddf24276b46270d009ed78b4 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:54.026924 2881 raft_consensus.cc:2749] T 59ad016e15ec4a488921eb564029b689 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:54.027112 2881 raft_consensus.cc:2804] T 5e20842de8cb4355b732685605c88834 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:54.027173 2881 raft_consensus.cc:493] T 5e20842de8cb4355b732685605c88834 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:54.027148 714 leader_election.cc:304] T 51ba0bcd498043c383c493f45ad1fe17 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:54.027232 2881 raft_consensus.cc:3060] T 5e20842de8cb4355b732685605c88834 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:52:54.027442 714 leader_election.cc:343] T 7ea7a5d8a92b4ac0903483627258ae66 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 7ea7a5d8a92b4ac0903483627258ae66
I20260202 02:52:54.027530 3040 raft_consensus.cc:2749] T eea6bb04ddf24276b46270d009ed78b4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:54.027665 3040 raft_consensus.cc:2804] T 51ba0bcd498043c383c493f45ad1fe17 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260202 02:52:54.027688 714 leader_election.cc:343] T 2b352d9a815e4bccafb7e2251500c5bc P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 2b352d9a815e4bccafb7e2251500c5bc
I20260202 02:52:54.027722 3040 raft_consensus.cc:493] T 51ba0bcd498043c383c493f45ad1fe17 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:54.027791 3040 raft_consensus.cc:3060] T 51ba0bcd498043c383c493f45ad1fe17 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:54.027827 714 leader_election.cc:304] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:54.028558 3041 raft_consensus.cc:2749] T cb634d4bae554ef3b39eeaa9958adc6c P 1f442ef85029461091de847118c34c0d [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260202 02:52:54.029207 714 leader_election.cc:343] T abfe7f76b2424c4d9ab07b29b66ec103 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: abfe7f76b2424c4d9ab07b29b66ec103
I20260202 02:52:54.029357 714 leader_election.cc:304] T abfe7f76b2424c4d9ab07b29b66ec103 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:54.029630 3043 raft_consensus.cc:2749] T abfe7f76b2424c4d9ab07b29b66ec103 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:54.030016 2881 raft_consensus.cc:515] T 5e20842de8cb4355b732685605c88834 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:54.030164 3040 raft_consensus.cc:515] T 51ba0bcd498043c383c493f45ad1fe17 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.030619 2881 leader_election.cc:290] T 5e20842de8cb4355b732685605c88834 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:54.030663 3040 leader_election.cc:290] T 51ba0bcd498043c383c493f45ad1fe17 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:54.030165 714 leader_election.cc:343] T 3284213e4b0447b49a7718ba896d3fa8 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 3284213e4b0447b49a7718ba896d3fa8
I20260202 02:52:54.031805 752 tablet_service.cc:1505] Processing CreateTablet for tablet 271b7d97a03b4f389e4c3ba991ba522a (DEFAULT_TABLE table=default.Table_dbb40510d128424fa8b9b668c4f372c8 [id=1393a89f57284d85b02fb576dc011916]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.031855 751 tablet_service.cc:1505] Processing CreateTablet for tablet 71dd6890e0e948d09d102296d1bd6464 (DEFAULT_TABLE table=default.Table_616a8ade4b5e481b8d4906b7a2a026d0 [id=98ee1a7044cd46d8b39dfda755ed8a7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.032204 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 271b7d97a03b4f389e4c3ba991ba522a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.032472 741 tablet_service.cc:1505] Processing CreateTablet for tablet f7bf82f274234991b0bab4c3fa3cf4bd (DEFAULT_TABLE table=default.Table_f21cdfd093884683ab4ec1af5345e9d5 [id=ddab3068794747a89a4950dc1ba79b5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.033072 746 tablet_service.cc:1505] Processing CreateTablet for tablet d0e00bed25e740f5b39e023f9ed61d5a (DEFAULT_TABLE table=default.Table_f8d8615b34774b738faeb83039f9601a [id=8882e1fbac2949838c99adb1290f273e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.033092 758 tablet_service.cc:1505] Processing CreateTablet for tablet 154d8a83adb94adab40615da10a9ddb9 (DEFAULT_TABLE table=default.Table_c783f2691903413c90a5e8502f5ddffa [id=5a36f14b8e134e5bafbfd7bd4d04b974]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.033455 744 tablet_service.cc:1505] Processing CreateTablet for tablet 22c31519d7e642118ff1218e2a1b7621 (DEFAULT_TABLE table=default.Table_a9d14eea240d4bca96de4ea77b5b31f2 [id=c6e5418bf48e43e2adb6d1b60445f0b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.033947 755 tablet_service.cc:1505] Processing CreateTablet for tablet 6f2739c054ce4e6b87634eced940d548 (DEFAULT_TABLE table=default.Table_cdcc140c2b8d4aabae614d58873dc22a [id=c2c454eb645044839fa81b33b6460558]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.034089 745 tablet_service.cc:1552] Processing DeleteTablet for tablet 8cc5e23a86a3467c89794ae8c403230e with delete_type TABLET_DATA_DELETED (replaced by tablet 9c1e408a75fe4d948a355f0dede46316) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:54.034274 757 tablet_service.cc:1505] Processing CreateTablet for tablet 84d68594516e4be4a3a1f4ebe9e217a5 (DEFAULT_TABLE table=default.Table_33fc32ec03094e3c8fe78de4f6a26777 [id=85340c1716bd4235bdad7042b53ce58c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.034590 742 tablet_service.cc:1505] Processing CreateTablet for tablet d8d706b1bf49494c871aee3ddc16e40c (DEFAULT_TABLE table=default.Table_2ee0b536f96e44e4a3a4a9439c7bccc9 [id=d95486b11c494863be7380146ebcf0c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.035105 745 tablet_service.cc:1505] Processing CreateTablet for tablet dac0f74c3d3f410f8222764ba9cd7f98 (DEFAULT_TABLE table=default.Table_619d2fcf726a46bfb5e939bbe6f55612 [id=038692c44a0043bab69ab29f77aa13be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.035472 759 tablet_service.cc:1505] Processing CreateTablet for tablet b3e095157aec4096a2ea5db96872ae12 (DEFAULT_TABLE table=default.Table_48226e7c77da41cdba56e04221eb9689 [id=05e52ce8e94e4975b51849059b904272]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.035591 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7bf82f274234991b0bab4c3fa3cf4bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.035787 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3e095157aec4096a2ea5db96872ae12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.036044 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0e00bed25e740f5b39e023f9ed61d5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.036182 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22c31519d7e642118ff1218e2a1b7621. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.036445 749 tablet_service.cc:1505] Processing CreateTablet for tablet f5c0e9f5b3d643659cdde15cbde856ab (DEFAULT_TABLE table=default.Table_0af7d8f82d2b4503aca40ada95dde3fc [id=90956c2303384b6cb0a7f4771e19a9b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.040488 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 154d8a83adb94adab40615da10a9ddb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.042580 748 tablet_service.cc:1505] Processing CreateTablet for tablet aa7ebfc320944c818cf7398729b3ae96 (DEFAULT_TABLE table=default.Table_d866f96ecc014b50a4c39f4f93653c9e [id=df985a2551e1405a8b9b02adf4043e7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.043264 740 tablet_service.cc:1505] Processing CreateTablet for tablet 5692ca1e36894c3194f0fcff18d014ba (DEFAULT_TABLE table=default.Table_9d4fbf619567432a88aa9a0c1b19cb3e [id=fff85ebd1bd1415c932f7c9c5a0da42a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.043779 753 tablet_service.cc:1505] Processing CreateTablet for tablet 00caa7d9a3ba445c97209bfb68de10d4 (DEFAULT_TABLE table=default.Table_86f302ed9e76457888b61469aa6b2e7f [id=a16d364dde354c58a0cdc51be094b09f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.045364 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet f94f25f4059a43b8babedb6b45a08a19 (table default.Table_22445ccfd7b64298b92a0f70733f3625 [id=300ea019d2724bee8211d7462710dc26]) successfully deleted
I20260202 02:52:54.050362 526 catalog_manager.cc:5654] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d reported cstate change: term changed from 2 to 3, leader changed from 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130) to 1f442ef85029461091de847118c34c0d (127.0.104.129). New cstate: current_term: 3 leader_uuid: "1f442ef85029461091de847118c34c0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } health_report { overall_health: UNKNOWN } } }
I20260202 02:52:54.051596 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5c0e9f5b3d643659cdde15cbde856ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.051754 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71dd6890e0e948d09d102296d1bd6464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.051939 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8d706b1bf49494c871aee3ddc16e40c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.052248 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dac0f74c3d3f410f8222764ba9cd7f98. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:54.030196 715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 155385us
I20260202 02:52:54.055723 3040 raft_consensus.cc:493] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:54.055848 3040 raft_consensus.cc:515] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.056401 3040 leader_election.cc:290] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:54.058264 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84d68594516e4be4a3a1f4ebe9e217a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.060539 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa7ebfc320944c818cf7398729b3ae96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.062105 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5692ca1e36894c3194f0fcff18d014ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.063252 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00caa7d9a3ba445c97209bfb68de10d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.069204 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f2739c054ce4e6b87634eced940d548. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.070009 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 7709ea34429e4e91ad4922a3d73ebd14 (table default.Table_a9d14eea240d4bca96de4ea77b5b31f2 [id=c6e5418bf48e43e2adb6d1b60445f0b9]) successfully deleted
I20260202 02:52:54.070462 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet dc90eeb79ded4739883ad98928476680 (table default.Table_a9b816e5b5b84151b840bfad315deb70 [id=b46bd49d8fe14af094e69597521a1a22]) successfully deleted
I20260202 02:52:54.074293 1882 ts_tablet_manager.cc:1929] T 474df41e4fe24a818fa45a4ce84d5474 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:54.074857 1060 raft_consensus.cc:1240] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d [term 3 FOLLOWER]: Rejecting Update request from peer 1521a6d262b7451bbef8debe970dd6b5 for earlier term 2. Current term is 3. Ops: []
I20260202 02:52:54.080265 1882 log.cc:1199] T 474df41e4fe24a818fa45a4ce84d5474 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/474df41e4fe24a818fa45a4ce84d5474
I20260202 02:52:54.084550 1174 tablet_bootstrap.cc:492] T 25a407001f964da6b51f42375136c79a P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:54.084757 1174 ts_tablet_manager.cc:1403] T 25a407001f964da6b51f42375136c79a P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.065s user 0.006s sys 0.000s
I20260202 02:52:54.085670 3040 consensus_queue.cc:1048] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260202 02:52:54.087591 1173 tablet_bootstrap.cc:654] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:54.087838 715 leader_election.cc:304] T 2b352d9a815e4bccafb7e2251500c5bc P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:54.088488 773 raft_consensus.cc:1240] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [term 3 LEADER]: Rejecting Update request from peer 1521a6d262b7451bbef8debe970dd6b5 for earlier term 2. Current term is 3. Ops: []
I20260202 02:52:54.089103 2881 raft_consensus.cc:2804] T 2b352d9a815e4bccafb7e2251500c5bc P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:54.089217 2881 raft_consensus.cc:697] T 2b352d9a815e4bccafb7e2251500c5bc P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:54.089254 715 leader_election.cc:304] T f3eebefa74584c1791d52377f5ba1290 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:54.089458 2881 consensus_queue.cc:237] T 2b352d9a815e4bccafb7e2251500c5bc P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.091220 2911 raft_consensus.cc:2804] T f3eebefa74584c1791d52377f5ba1290 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:54.091301 2911 raft_consensus.cc:493] T f3eebefa74584c1791d52377f5ba1290 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:54.091354 2911 raft_consensus.cc:3060] T f3eebefa74584c1791d52377f5ba1290 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:54.091465 715 leader_election.cc:304] T ac2ab12f5040488aa306cb69b107b485 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:54.093163 759 tablet_service.cc:1552] Processing DeleteTablet for tablet 2616e1bbce4049dbb584550ccc150288 with delete_type TABLET_DATA_DELETED (replaced by tablet 8dd62de8e8234d6c9ce380b1aadc097d) from {username='slave'} at 127.0.0.1:42950
W20260202 02:52:54.093425 714 leader_election.cc:343] T ac2ab12f5040488aa306cb69b107b485 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: ac2ab12f5040488aa306cb69b107b485
W20260202 02:52:54.093701 714 leader_election.cc:343] T 02b434fa058b45529ff3938b053263c6 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 02b434fa058b45529ff3938b053263c6
I20260202 02:52:54.093784 714 leader_election.cc:304] T 02b434fa058b45529ff3938b053263c6 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:54.093984 3040 raft_consensus.cc:2804] T ac2ab12f5040488aa306cb69b107b485 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:54.094015 2911 raft_consensus.cc:515] T f3eebefa74584c1791d52377f5ba1290 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.094163 3040 raft_consensus.cc:697] T ac2ab12f5040488aa306cb69b107b485 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:54.094376 3040 consensus_queue.cc:237] T ac2ab12f5040488aa306cb69b107b485 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.094605 2911 leader_election.cc:290] T f3eebefa74584c1791d52377f5ba1290 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:54.099777 1174 raft_consensus.cc:359] T 25a407001f964da6b51f42375136c79a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.100934 1174 raft_consensus.cc:385] T 25a407001f964da6b51f42375136c79a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:54.101367 1174 raft_consensus.cc:740] T 25a407001f964da6b51f42375136c79a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:54.101922 1174 consensus_queue.cc:260] T 25a407001f964da6b51f42375136c79a P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
W20260202 02:52:54.104701 999 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 84598.2us
I20260202 02:52:54.105444 2794 raft_consensus.cc:493] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:54.105607 2794 raft_consensus.cc:515] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.106202 2794 leader_election.cc:290] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:54.106853 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dc90eeb79ded4739883ad98928476680" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:54.107066 773 raft_consensus.cc:2468] T dc90eeb79ded4739883ad98928476680 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:54.107532 998 leader_election.cc:304] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:54.107829 2794 raft_consensus.cc:2804] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:54.107879 2911 raft_consensus.cc:493] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:54.107905 2794 raft_consensus.cc:493] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:54.107959 2794 raft_consensus.cc:3060] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:54.107985 2911 raft_consensus.cc:515] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.108526 2911 leader_election.cc:290] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:54.106202 1174 ts_tablet_manager.cc:1434] T 25a407001f964da6b51f42375136c79a P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.021s user 0.002s sys 0.000s
I20260202 02:52:54.109624 1174 tablet_bootstrap.cc:492] T 07580ad1d049475a9645acfad0b1d219 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:54.110668 2794 raft_consensus.cc:515] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.111771 2794 leader_election.cc:290] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:54.112084 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dc90eeb79ded4739883ad98928476680" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:54.114199 773 raft_consensus.cc:3060] T dc90eeb79ded4739883ad98928476680 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:54.116645 1061 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "51ba0bcd498043c383c493f45ad1fe17" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:54.116839 1061 raft_consensus.cc:3060] T 51ba0bcd498043c383c493f45ad1fe17 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:54.116894 773 raft_consensus.cc:2468] T dc90eeb79ded4739883ad98928476680 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:54.117374 998 leader_election.cc:304] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:54.117755 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "99e0b5e9a9874aec886155e0d614dec2" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:54.117944 1059 raft_consensus.cc:2468] T 99e0b5e9a9874aec886155e0d614dec2 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:54.118203 2979 raft_consensus.cc:2804] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:54.118392 2979 raft_consensus.cc:697] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:54.118678 2979 consensus_queue.cc:237] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.119617 1061 raft_consensus.cc:2468] T 51ba0bcd498043c383c493f45ad1fe17 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:54.120203 1882 ts_tablet_manager.cc:1950] T 474df41e4fe24a818fa45a4ce84d5474 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:54.120887 1061 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5e20842de8cb4355b732685605c88834" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:54.121481 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 770) took 4704 ms. Trace:
I20260202 02:52:54.121584 1882 rpcz_store.cc:276] 0202 02:52:49.416520 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:49.416899 (+ 379us) service_pool.cc:225] Handling call
0202 02:52:53.797533 (+4380634us) ts_tablet_manager.cc:1151] Deleting tablet 474df41e4fe24a818fa45a4ce84d5474
0202 02:52:53.797544 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:54.022060 (+224516us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:54.074176 (+ 52116us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:54.121463 (+ 47287us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4380498}
I20260202 02:52:54.121747 1061 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6a3dac5cd1ce47488a50b4c32eca67c7" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:54.122088 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 474df41e4fe24a818fa45a4ce84d5474 (table default.Table_44f181eda9a64d218b9bac7f62831d07 [id=14bbe505bc8b4a4bbde264a2c0dcdf50]) successfully deleted
I20260202 02:52:54.124765 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 771) took 4708 ms. Trace:
I20260202 02:52:54.126401 1882 rpcz_store.cc:276] 0202 02:52:49.416622 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:49.417045 (+ 423us) service_pool.cc:225] Handling call
0202 02:52:54.124671 (+4707626us) ts_tablet_manager.cc:1151] Deleting tablet d8418a9dd0944456b4d4820dde0cde02
0202 02:52:54.124681 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:54.124745 (+ 64us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4707489}
I20260202 02:52:54.129779 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ee04563d3b8e4cb2b989bb8d6ee9b20e" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:54.129840 515 catalog_manager.cc:4952] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet d8418a9dd0944456b4d4820dde0cde02 because the tablet was not found. No further retry: Not found: Tablet not found: d8418a9dd0944456b4d4820dde0cde02
I20260202 02:52:54.131889 3041 raft_consensus.cc:2749] T 02b434fa058b45529ff3938b053263c6 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260202 02:52:54.132501 714 consensus_peers.cc:597] T 8bb14bf551f441dcbaaff7057b51b2be P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8bb14bf551f441dcbaaff7057b51b2be. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:54.132853 714 leader_election.cc:343] T ac2ab12f5040488aa306cb69b107b485 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: ac2ab12f5040488aa306cb69b107b485
I20260202 02:52:54.136224 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f3eebefa74584c1791d52377f5ba1290" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:54.137494 1060 raft_consensus.cc:3060] T f3eebefa74584c1791d52377f5ba1290 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:54.138074 1882 tablet_replica.cc:333] T 072bb7f0e4ab4935be49461bc8a3c163 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:54.138283 1882 raft_consensus.cc:2243] T 072bb7f0e4ab4935be49461bc8a3c163 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:54.138461 1882 raft_consensus.cc:2272] T 072bb7f0e4ab4935be49461bc8a3c163 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:54.139050 1882 ts_tablet_manager.cc:1916] T 072bb7f0e4ab4935be49461bc8a3c163 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:54.145427 1174 tablet_bootstrap.cc:654] T 07580ad1d049475a9645acfad0b1d219 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:54.147678 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet 67a8c9a1998b47b3b214a06ce62381e9 (table default.Table_1fa8930369e844ca8384126bf8f22e66 [id=116247212fca4c0a95ad1bdbbac4787f]) was not created within the allowed timeout. Replacing with a new tablet 32ee0da311f1453d8a2d30f9d8be6702
I20260202 02:52:54.150533 2911 raft_consensus.cc:493] T 25a407001f964da6b51f42375136c79a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:54.150645 715 leader_election.cc:304] T 99e0b5e9a9874aec886155e0d614dec2 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:54.150691 2911 raft_consensus.cc:515] T 25a407001f964da6b51f42375136c79a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.151455 2911 leader_election.cc:290] T 25a407001f964da6b51f42375136c79a P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:54.155077 2911 raft_consensus.cc:2804] T 99e0b5e9a9874aec886155e0d614dec2 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:54.155254 2911 raft_consensus.cc:493] T 99e0b5e9a9874aec886155e0d614dec2 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:54.155449 2911 raft_consensus.cc:3060] T 99e0b5e9a9874aec886155e0d614dec2 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:54.158047 2911 raft_consensus.cc:515] T 99e0b5e9a9874aec886155e0d614dec2 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.158761 2911 leader_election.cc:290] T 99e0b5e9a9874aec886155e0d614dec2 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:54.158895 1173 tablet_bootstrap.cc:492] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:54.159046 1173 ts_tablet_manager.cc:1403] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.175s user 0.004s sys 0.004s
I20260202 02:52:54.155429 715 leader_election.cc:304] T 51ba0bcd498043c383c493f45ad1fe17 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:54.159850 1173 raft_consensus.cc:359] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.160032 1173 raft_consensus.cc:385] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:54.160095 1173 raft_consensus.cc:740] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:54.160211 2911 raft_consensus.cc:2804] T 51ba0bcd498043c383c493f45ad1fe17 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:54.160290 1173 consensus_queue.cc:260] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.160852 1173 ts_tablet_manager.cc:1434] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:54.161012 2911 raft_consensus.cc:697] T 51ba0bcd498043c383c493f45ad1fe17 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:54.161132 1173 tablet_bootstrap.cc:492] T 528d4269f91a44679acfad16e9fbbe32 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:54.161417 2911 consensus_queue.cc:237] T 51ba0bcd498043c383c493f45ad1fe17 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.163143 1173 tablet_bootstrap.cc:654] T 528d4269f91a44679acfad16e9fbbe32 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:54.159958 715 leader_election.cc:343] T 5e20842de8cb4355b732685605c88834 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
W20260202 02:52:54.163666 715 leader_election.cc:343] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:54.017076 915 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "20ea39bc5d3d46f5a4ba427277d6c6a3" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:54.018101 911 raft_consensus.cc:1275] T e1284fde67bc4d65b067a54c416ccc33 P 1521a6d262b7451bbef8debe970dd6b5 [term 3 FOLLOWER]: Refusing update from remote peer 1f442ef85029461091de847118c34c0d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260202 02:52:54.166805 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet cef87d5f74f14a6d86150958de7b1ace (table default.Table_6831562d21db463986dc58b0100e9792 [id=50e58db611f247ee9dcd5b31776ace05]) was not created within the allowed timeout. Replacing with a new tablet cc7cb7a8cadc4c90a2449f9d7183aa9a
W20260202 02:52:54.167155 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet 1959438c70c544e69422f7354082cf89 (table default.Table_eeb507a1a49d407196bff72270b0834f [id=a18136bbb87a47d69600f926a474da24]) was not created within the allowed timeout. Replacing with a new tablet 223bf8b5e2684e49a316a3a654a0bf1a
I20260202 02:52:54.075470 2964 consensus_queue.cc:1059] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260202 02:52:54.167623 2964 raft_consensus.cc:3055] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [term 2 LEADER]: Stepping down as leader of term 2
I20260202 02:52:54.167789 2964 raft_consensus.cc:740] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:54.168107 2964 consensus_queue.cc:260] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:54.168504 2964 raft_consensus.cc:3060] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Advancing to term 3
W20260202 02:52:54.171986 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet 9f11a68d10374d06bc4a756a1d15bf09 (table default.Table_509184aa45184b7c8228aba2de1c528a [id=d3627805d70e4e269f073250e883f3e3]) was not created within the allowed timeout. Replacing with a new tablet da9d3228ad734065bfc640833c988c92
I20260202 02:52:54.078146 890 tablet_service.cc:1505] Processing CreateTablet for tablet 22c31519d7e642118ff1218e2a1b7621 (DEFAULT_TABLE table=default.Table_a9d14eea240d4bca96de4ea77b5b31f2 [id=c6e5418bf48e43e2adb6d1b60445f0b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.172652 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22c31519d7e642118ff1218e2a1b7621. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:54.172648 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet bbcc723a8079443db455ae20fb0d585d (table default.Table_93c8b5e956214e88abc353bfb46c8757 [id=1cb84b11b2f34287bbade3af6cd4df67]) was not created within the allowed timeout. Replacing with a new tablet 39d9e3fef89d4c1cb053278e25796eaf
W20260202 02:52:54.172772 715 leader_election.cc:343] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Not found: Tablet not found: 026e6ee0b3ae4d938fdf1550044ed1fa
W20260202 02:52:54.172909 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet 9a1d7d173c41462c8e32a95b494bb708 (table default.Table_28e7d9aa8131413e9a174756c246d164 [id=ec07c59893cf45c9b2a5eece556d10c5]) was not created within the allowed timeout. Replacing with a new tablet 862047d463374985ae0562f86ca2089d
W20260202 02:52:54.173192 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet 7f323175ee254df789bcb04e65047dff (table default.Table_c08aceb74c164c06a3a149c454c9c59e [id=0d6674bc4f264a36aad4ea7962a9e0e7]) was not created within the allowed timeout. Replacing with a new tablet 5a60685d08b249bfbc80da8c2ac9b28f
W20260202 02:52:54.174042 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet 4a518e8ad2e34ec98a0425c7ed3a470a (table default.Table_3dcd1dea44ed4e54a89f4413e7f17dbe [id=f868f7fd84f247c89a963da591f16c2d]) was not created within the allowed timeout. Replacing with a new tablet 6bce8dce844341f4982a44e1e90002e5
W20260202 02:52:54.174366 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet c17c85759e964f0ba3abcaf5fe938fba (table default.Table_67001aa310ae44c0bb9e2b24bb0b57b8 [id=c17d62a148094547af24de85773a1b8d]) was not created within the allowed timeout. Replacing with a new tablet e2fb216334a245e08617d18c836f1fbf
W20260202 02:52:54.174620 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet 604b3b863c9540d3b55381dbfe1afdc3 (table default.Table_756b4deb63c645e79a6fbb8f8f131ee0 [id=ce752170017a4e119a8c42c73fd95302]) was not created within the allowed timeout. Replacing with a new tablet 63c8ecaa42224d2ba0cc9cf26a05e698
W20260202 02:52:54.174899 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet 09504070c9ec4bcfbb59e3fb4c189775 (table default.Table_abd89cee70a04e2aa3a0d59b7eb943c0 [id=4c17da59a0554cee93ce4e626838a2c6]) was not created within the allowed timeout. Replacing with a new tablet 6c654857b0e74e399b16601c9ebf692e
W20260202 02:52:54.175362 715 leader_election.cc:343] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:54.078651 894 tablet_service.cc:1505] Processing CreateTablet for tablet 6f2739c054ce4e6b87634eced940d548 (DEFAULT_TABLE table=default.Table_cdcc140c2b8d4aabae614d58873dc22a [id=c2c454eb645044839fa81b33b6460558]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.177114 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f2739c054ce4e6b87634eced940d548. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.180784 3081 raft_consensus.cc:493] T 7f9dfac19c04475aadb45e95a6474eb4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:54.180931 3081 raft_consensus.cc:515] T 7f9dfac19c04475aadb45e95a6474eb4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.181514 3081 leader_election.cc:290] T 7f9dfac19c04475aadb45e95a6474eb4 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:54.182035 1882 ts_tablet_manager.cc:1929] T 072bb7f0e4ab4935be49461bc8a3c163 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:54.182178 1882 log.cc:1199] T 072bb7f0e4ab4935be49461bc8a3c163 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/072bb7f0e4ab4935be49461bc8a3c163
I20260202 02:52:54.182546 1882 ts_tablet_manager.cc:1950] T 072bb7f0e4ab4935be49461bc8a3c163 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:54.183332 1054 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "25a407001f964da6b51f42375136c79a" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:54.183633 1054 raft_consensus.cc:2468] T 25a407001f964da6b51f42375136c79a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:54.183934 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7f9dfac19c04475aadb45e95a6474eb4" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:54.079111 899 tablet_service.cc:1505] Processing CreateTablet for tablet 84d68594516e4be4a3a1f4ebe9e217a5 (DEFAULT_TABLE table=default.Table_33fc32ec03094e3c8fe78de4f6a26777 [id=85340c1716bd4235bdad7042b53ce58c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.079608 898 tablet_service.cc:1505] Processing CreateTablet for tablet dac0f74c3d3f410f8222764ba9cd7f98 (DEFAULT_TABLE table=default.Table_619d2fcf726a46bfb5e939bbe6f55612 [id=038692c44a0043bab69ab29f77aa13be]), partition=RANGE (key) PARTITION UNBOUNDED
W20260202 02:52:54.184481 859 leader_election.cc:343] T 7f9dfac19c04475aadb45e95a6474eb4 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:54.184643 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dac0f74c3d3f410f8222764ba9cd7f98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.185200 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84d68594516e4be4a3a1f4ebe9e217a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.079838 900 tablet_service.cc:1505] Processing CreateTablet for tablet aa7ebfc320944c818cf7398729b3ae96 (DEFAULT_TABLE table=default.Table_d866f96ecc014b50a4c39f4f93653c9e [id=df985a2551e1405a8b9b02adf4043e7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.079946 886 tablet_service.cc:1505] Processing CreateTablet for tablet d8d706b1bf49494c871aee3ddc16e40c (DEFAULT_TABLE table=default.Table_2ee0b536f96e44e4a3a4a9439c7bccc9 [id=d95486b11c494863be7380146ebcf0c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.080236 901 tablet_service.cc:1505] Processing CreateTablet for tablet 5692ca1e36894c3194f0fcff18d014ba (DEFAULT_TABLE table=default.Table_9d4fbf619567432a88aa9a0c1b19cb3e [id=fff85ebd1bd1415c932f7c9c5a0da42a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.080255 895 tablet_service.cc:1505] Processing CreateTablet for tablet b3e095157aec4096a2ea5db96872ae12 (DEFAULT_TABLE table=default.Table_48226e7c77da41cdba56e04221eb9689 [id=05e52ce8e94e4975b51849059b904272]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.191370 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8d706b1bf49494c871aee3ddc16e40c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.191695 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3e095157aec4096a2ea5db96872ae12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.081141 896 tablet_service.cc:1505] Processing CreateTablet for tablet f5c0e9f5b3d643659cdde15cbde856ab (DEFAULT_TABLE table=default.Table_0af7d8f82d2b4503aca40ada95dde3fc [id=90956c2303384b6cb0a7f4771e19a9b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.191920 1173 tablet_bootstrap.cc:492] T 528d4269f91a44679acfad16e9fbbe32 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:54.192023 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5c0e9f5b3d643659cdde15cbde856ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.192055 1173 ts_tablet_manager.cc:1403] T 528d4269f91a44679acfad16e9fbbe32 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.031s user 0.002s sys 0.001s
I20260202 02:52:54.192934 1173 raft_consensus.cc:359] T 528d4269f91a44679acfad16e9fbbe32 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.193106 1173 raft_consensus.cc:385] T 528d4269f91a44679acfad16e9fbbe32 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:54.193149 1173 raft_consensus.cc:740] T 528d4269f91a44679acfad16e9fbbe32 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:54.193368 1173 consensus_queue.cc:260] T 528d4269f91a44679acfad16e9fbbe32 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.193943 1173 ts_tablet_manager.cc:1434] T 528d4269f91a44679acfad16e9fbbe32 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260202 02:52:54.194211 1173 tablet_bootstrap.cc:492] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:54.088758 1182 ts_tablet_manager.cc:1929] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:54.195639 1182 log.cc:1199] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/6a3dac5cd1ce47488a50b4c32eca67c7
I20260202 02:52:54.195752 1173 tablet_bootstrap.cc:654] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:54.183436 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 785) took 4705 ms. Trace:
I20260202 02:52:54.195963 1182 ts_tablet_manager.cc:1950] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:54.195915 1882 rpcz_store.cc:276] 0202 02:52:49.478051 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:49.478123 (+ 72us) service_pool.cc:225] Handling call
0202 02:52:54.130195 (+4652072us) ts_tablet_manager.cc:1151] Deleting tablet 072bb7f0e4ab4935be49461bc8a3c163
0202 02:52:54.138022 (+ 7827us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:54.168879 (+ 30857us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:54.181920 (+ 13041us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:54.183417 (+ 1497us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4651821}
I20260202 02:52:54.196264 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 898) took 1486 ms. Trace:
I20260202 02:52:54.196475 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5692ca1e36894c3194f0fcff18d014ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.196370 1182 rpcz_store.cc:276] 0202 02:52:52.709753 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.710578 (+ 825us) service_pool.cc:225] Handling call
0202 02:52:53.967887 (+1257309us) ts_tablet_manager.cc:1151] Deleting tablet 6a3dac5cd1ce47488a50b4c32eca67c7
0202 02:52:53.967897 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:53.978863 (+ 10966us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:54.088640 (+109777us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:54.196246 (+107606us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1257194}
I20260202 02:52:54.196523 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 072bb7f0e4ab4935be49461bc8a3c163 (table default.Table_959d3e3da2814b3a986d29864e47246c [id=bdf2f19a439d4be3aa2362cfde38ab7f]) successfully deleted
I20260202 02:52:54.196714 1182 tablet_replica.cc:333] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:54.196909 1182 raft_consensus.cc:2243] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:54.197208 1182 raft_consensus.cc:2272] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:54.197578 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 787) took 4644 ms. Trace:
I20260202 02:52:54.197666 1882 rpcz_store.cc:276] 0202 02:52:49.553544 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:49.553600 (+ 56us) service_pool.cc:225] Handling call
0202 02:52:54.197364 (+4643764us) ts_tablet_manager.cc:1151] Deleting tablet c03a12f911084222be15ea6f482ebe32
0202 02:52:54.197376 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:54.197553 (+ 177us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4643478}
I20260202 02:52:54.146659 3076 raft_consensus.cc:493] T 3dcf4f794ec14b2f9d181d015fc437fb P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260202 02:52:54.198171 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet c03a12f911084222be15ea6f482ebe32 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c03a12f911084222be15ea6f482ebe32 already in progress: creating tablet
I20260202 02:52:54.198271 3076 raft_consensus.cc:515] T 3dcf4f794ec14b2f9d181d015fc437fb P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:54.198930 3076 leader_election.cc:290] T 3dcf4f794ec14b2f9d181d015fc437fb P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:54.080936 897 tablet_service.cc:1505] Processing CreateTablet for tablet 00caa7d9a3ba445c97209bfb68de10d4 (DEFAULT_TABLE table=default.Table_86f302ed9e76457888b61469aa6b2e7f [id=a16d364dde354c58a0cdc51be094b09f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.199831 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00caa7d9a3ba445c97209bfb68de10d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.200582 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 788) took 4646 ms. Trace:
I20260202 02:52:54.200665 1882 rpcz_store.cc:276] 0202 02:52:49.553947 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:49.553988 (+ 41us) service_pool.cc:225] Handling call
0202 02:52:54.200377 (+4646389us) ts_tablet_manager.cc:1151] Deleting tablet 4482b24826884d0da0d34f43f69e1646
0202 02:52:54.200393 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:54.200559 (+ 166us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4646002}
W20260202 02:52:54.201146 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 4482b24826884d0da0d34f43f69e1646 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4482b24826884d0da0d34f43f69e1646 already in progress: creating tablet
I20260202 02:52:54.204089 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 789) took 4649 ms. Trace:
I20260202 02:52:54.204176 1882 rpcz_store.cc:276] 0202 02:52:49.554284 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:49.554422 (+ 138us) service_pool.cc:225] Handling call
0202 02:52:54.203938 (+4649516us) ts_tablet_manager.cc:1151] Deleting tablet 00f2d3bafb2d472aaffc0a7ba926c825
0202 02:52:54.203950 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:54.204064 (+ 114us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4649402}
W20260202 02:52:54.204679 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 00f2d3bafb2d472aaffc0a7ba926c825 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 00f2d3bafb2d472aaffc0a7ba926c825 already in progress: creating tablet
I20260202 02:52:54.206245 3076 consensus_queue.cc:1048] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:54.210520 1182 ts_tablet_manager.cc:1916] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:54.213069 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 790) took 4643 ms. Trace:
I20260202 02:52:54.213196 1882 rpcz_store.cc:276] 0202 02:52:49.569236 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:49.569854 (+ 618us) service_pool.cc:225] Handling call
0202 02:52:54.212893 (+4643039us) ts_tablet_manager.cc:1151] Deleting tablet c707180fc9c5457292a8f43bf676f4f9
0202 02:52:54.212908 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:54.213039 (+ 131us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4642832}
I20260202 02:52:54.218317 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3dcf4f794ec14b2f9d181d015fc437fb" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:54.220082 859 leader_election.cc:343] T 3dcf4f794ec14b2f9d181d015fc437fb P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:54.222702 1882 tablet_replica.cc:333] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:54.224695 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa7ebfc320944c818cf7398729b3ae96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.224916 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "99e0b5e9a9874aec886155e0d614dec2" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:54.225203 1059 raft_consensus.cc:3060] T 99e0b5e9a9874aec886155e0d614dec2 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:54.225244 1882 raft_consensus.cc:2243] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:54.226655 1882 raft_consensus.cc:2272] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:54.227407 1173 tablet_bootstrap.cc:492] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:54.227756 1059 raft_consensus.cc:2468] T 99e0b5e9a9874aec886155e0d614dec2 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:54.227859 1173 ts_tablet_manager.cc:1403] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.034s user 0.001s sys 0.003s
I20260202 02:52:54.228699 1173 raft_consensus.cc:359] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.228888 1173 raft_consensus.cc:385] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:54.228948 1173 raft_consensus.cc:740] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:54.229214 1882 ts_tablet_manager.cc:1916] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:54.230616 1182 ts_tablet_manager.cc:1929] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:54.230813 1182 log.cc:1199] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/c8a273ac8cd344719a48fd81945bed0a
I20260202 02:52:54.231184 1182 ts_tablet_manager.cc:1950] T c8a273ac8cd344719a48fd81945bed0a P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:54.229175 1173 consensus_queue.cc:260] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.231617 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 903) took 1392 ms. Trace:
I20260202 02:52:54.231735 1182 rpcz_store.cc:276] 0202 02:52:52.838910 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.838972 (+ 62us) service_pool.cc:225] Handling call
0202 02:52:54.196650 (+1357678us) ts_tablet_manager.cc:1151] Deleting tablet c8a273ac8cd344719a48fd81945bed0a
0202 02:52:54.196661 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:54.213811 (+ 17150us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:54.230508 (+ 16697us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:54.231601 (+ 1093us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1357500}
I20260202 02:52:54.231886 1173 ts_tablet_manager.cc:1434] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260202 02:52:54.232142 1182 tablet_replica.cc:333] T f5727f9b9bbb4e77abfef065c51606a1 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:54.232173 1173 tablet_bootstrap.cc:492] T ee04563d3b8e4cb2b989bb8d6ee9b20e P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:54.232395 1182 raft_consensus.cc:2243] T f5727f9b9bbb4e77abfef065c51606a1 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:54.232583 1182 raft_consensus.cc:2272] T f5727f9b9bbb4e77abfef065c51606a1 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:54.233146 1182 ts_tablet_manager.cc:1916] T f5727f9b9bbb4e77abfef065c51606a1 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:54.234486 1173 tablet_bootstrap.cc:654] T ee04563d3b8e4cb2b989bb8d6ee9b20e P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:54.250188 1173 tablet_bootstrap.cc:492] T ee04563d3b8e4cb2b989bb8d6ee9b20e P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:54.250355 1173 ts_tablet_manager.cc:1403] T ee04563d3b8e4cb2b989bb8d6ee9b20e P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.000s
I20260202 02:52:54.251281 1173 raft_consensus.cc:359] T ee04563d3b8e4cb2b989bb8d6ee9b20e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.251456 1173 raft_consensus.cc:385] T ee04563d3b8e4cb2b989bb8d6ee9b20e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:54.251503 1173 raft_consensus.cc:740] T ee04563d3b8e4cb2b989bb8d6ee9b20e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:54.251699 1173 consensus_queue.cc:260] T ee04563d3b8e4cb2b989bb8d6ee9b20e P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.252084 715 leader_election.cc:304] T 25a407001f964da6b51f42375136c79a P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:54.252264 1173 ts_tablet_manager.cc:1434] T ee04563d3b8e4cb2b989bb8d6ee9b20e P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:54.252344 2911 raft_consensus.cc:2804] T 25a407001f964da6b51f42375136c79a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:54.252409 2911 raft_consensus.cc:493] T 25a407001f964da6b51f42375136c79a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:54.252465 2911 raft_consensus.cc:3060] T 25a407001f964da6b51f42375136c79a P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:54.252580 1173 tablet_bootstrap.cc:492] T eea6bb04ddf24276b46270d009ed78b4 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:54.254191 1173 tablet_bootstrap.cc:654] T eea6bb04ddf24276b46270d009ed78b4 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:54.254739 1882 ts_tablet_manager.cc:1929] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260202 02:52:54.254841 1882 log.cc:1199] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/8bb14bf551f441dcbaaff7057b51b2be
I20260202 02:52:54.254947 2911 raft_consensus.cc:515] T 25a407001f964da6b51f42375136c79a P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.255216 1882 ts_tablet_manager.cc:1950] T 8bb14bf551f441dcbaaff7057b51b2be P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:54.255528 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 791) took 4685 ms. Trace:
I20260202 02:52:54.255844 2911 leader_election.cc:290] T 25a407001f964da6b51f42375136c79a P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:54.260535 2911 consensus_queue.cc:1048] T 0138ba7326674741b0729324735f1de7 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:54.261226 715 leader_election.cc:304] T 99e0b5e9a9874aec886155e0d614dec2 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:54.261529 2911 raft_consensus.cc:2804] T 99e0b5e9a9874aec886155e0d614dec2 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:54.261771 2911 raft_consensus.cc:697] T 99e0b5e9a9874aec886155e0d614dec2 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:54.262059 2911 consensus_queue.cc:237] T 99e0b5e9a9874aec886155e0d614dec2 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.267858 526 master_service.cc:954] ReplaceTablet: received request to replace tablet 7c07195a46184d24bc709e4967945472 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:54.262451 2979 raft_consensus.cc:493] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:54.269342 2979 raft_consensus.cc:515] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.269701 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "25a407001f964da6b51f42375136c79a" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:54.270054 911 raft_consensus.cc:2468] T 25a407001f964da6b51f42375136c79a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:54.270112 2979 leader_election.cc:290] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:54.271286 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "59ad016e15ec4a488921eb564029b689" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:54.271275 1882 rpcz_store.cc:276] 0202 02:52:49.570063 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:49.570313 (+ 250us) service_pool.cc:225] Handling call
0202 02:52:54.222636 (+4652323us) ts_tablet_manager.cc:1151] Deleting tablet 8bb14bf551f441dcbaaff7057b51b2be
0202 02:52:54.222650 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:54.234194 (+ 11544us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:54.254624 (+ 20430us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:54.255508 (+ 884us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4652136}
I20260202 02:52:54.271492 773 raft_consensus.cc:2468] T 59ad016e15ec4a488921eb564029b689 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:54.271885 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 8bb14bf551f441dcbaaff7057b51b2be (table default.Table_7a314585590647ca8fd0f51e125b0622 [id=0e44d4587c694edc90c651e2f9db649b]) successfully deleted
I20260202 02:52:54.272286 998 leader_election.cc:304] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:54.272670 2979 raft_consensus.cc:2804] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:54.272749 2979 raft_consensus.cc:493] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:54.272802 2979 raft_consensus.cc:3060] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:54.272812 1174 tablet_bootstrap.cc:492] T 07580ad1d049475a9645acfad0b1d219 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:54.272956 1174 ts_tablet_manager.cc:1403] T 07580ad1d049475a9645acfad0b1d219 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.163s user 0.003s sys 0.000s
I20260202 02:52:54.273802 1174 raft_consensus.cc:359] T 07580ad1d049475a9645acfad0b1d219 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.273967 1174 raft_consensus.cc:385] T 07580ad1d049475a9645acfad0b1d219 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:54.274012 1174 raft_consensus.cc:740] T 07580ad1d049475a9645acfad0b1d219 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:54.274195 1174 consensus_queue.cc:260] T 07580ad1d049475a9645acfad0b1d219 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.275609 1173 tablet_bootstrap.cc:492] T eea6bb04ddf24276b46270d009ed78b4 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:54.275763 1173 ts_tablet_manager.cc:1403] T eea6bb04ddf24276b46270d009ed78b4 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.023s user 0.003s sys 0.000s
I20260202 02:52:54.276280 2979 raft_consensus.cc:515] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.276593 1173 raft_consensus.cc:359] T eea6bb04ddf24276b46270d009ed78b4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.276755 1173 raft_consensus.cc:385] T eea6bb04ddf24276b46270d009ed78b4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:54.276810 1173 raft_consensus.cc:740] T eea6bb04ddf24276b46270d009ed78b4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:54.277004 1173 consensus_queue.cc:260] T eea6bb04ddf24276b46270d009ed78b4 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.277483 1173 ts_tablet_manager.cc:1434] T eea6bb04ddf24276b46270d009ed78b4 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:54.277769 1173 tablet_bootstrap.cc:492] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:54.277953 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "59ad016e15ec4a488921eb564029b689" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:54.278136 773 raft_consensus.cc:3060] T 59ad016e15ec4a488921eb564029b689 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:54.280977 2979 leader_election.cc:290] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:54.281594 773 raft_consensus.cc:2468] T 59ad016e15ec4a488921eb564029b689 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:54.282464 998 leader_election.cc:304] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:54.283643 2979 raft_consensus.cc:2804] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:54.283799 2979 raft_consensus.cc:697] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:54.284014 2979 consensus_queue.cc:237] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.284611 1174 ts_tablet_manager.cc:1434] T 07580ad1d049475a9645acfad0b1d219 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.012s user 0.002s sys 0.000s
W20260202 02:52:54.288340 515 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 0883a26fce2540a5bc1c38e67b2f0dda (table default.Table_1b41581f01c94cc497e8af87b594b565 [id=ec4989eaa7f8482d8698a0e20d6e010f], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 0883a26fce2540a5bc1c38e67b2f0dda has no consensus state
I20260202 02:52:54.288740 1174 tablet_bootstrap.cc:492] T 7ff6d2fa085343a6b11be0898da6233d P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:54.289958 1173 tablet_bootstrap.cc:654] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:54.291569 3041 raft_consensus.cc:493] T 528d4269f91a44679acfad16e9fbbe32 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:54.291718 3041 raft_consensus.cc:515] T 528d4269f91a44679acfad16e9fbbe32 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.292313 3041 leader_election.cc:290] T 528d4269f91a44679acfad16e9fbbe32 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:54.295796 1182 ts_tablet_manager.cc:1929] T f5727f9b9bbb4e77abfef065c51606a1 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:54.296006 1182 log.cc:1199] T f5727f9b9bbb4e77abfef065c51606a1 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/f5727f9b9bbb4e77abfef065c51606a1
I20260202 02:52:54.296412 1182 ts_tablet_manager.cc:1950] T f5727f9b9bbb4e77abfef065c51606a1 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:54.296792 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 904) took 1431 ms. Trace:
I20260202 02:52:54.296960 1174 tablet_bootstrap.cc:654] T 7ff6d2fa085343a6b11be0898da6233d P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:54.296931 1182 rpcz_store.cc:276] 0202 02:52:52.865683 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.865769 (+ 86us) service_pool.cc:225] Handling call
0202 02:52:54.232101 (+1366332us) ts_tablet_manager.cc:1151] Deleting tablet f5727f9b9bbb4e77abfef065c51606a1
0202 02:52:54.232110 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:54.278714 (+ 46604us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:54.292645 (+ 13931us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:54.296775 (+ 4130us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1366129}
I20260202 02:52:54.297396 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 905) took 1431 ms. Trace:
I20260202 02:52:54.297554 1182 rpcz_store.cc:276] 0202 02:52:52.865786 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.865836 (+ 50us) service_pool.cc:225] Handling call
0202 02:52:54.297295 (+1431459us) ts_tablet_manager.cc:1151] Deleting tablet eea6bb04ddf24276b46270d009ed78b4
0202 02:52:54.297306 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:54.297381 (+ 75us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1431278}
I20260202 02:52:54.298014 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 908) took 1432 ms. Trace:
I20260202 02:52:54.300432 1182 rpcz_store.cc:276] 0202 02:52:52.866026 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.866064 (+ 38us) service_pool.cc:225] Handling call
0202 02:52:54.297909 (+1431845us) ts_tablet_manager.cc:1151] Deleting tablet fa57a267b0484262ac31039862f40a76
0202 02:52:54.297919 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:54.298000 (+ 81us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1431704}
I20260202 02:52:54.300957 1182 tablet_replica.cc:333] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:54.301394 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 786) took 4748 ms. Trace:
I20260202 02:52:54.301537 1882 rpcz_store.cc:276] 0202 02:52:49.553395 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:49.579194 (+ 25799us) service_pool.cc:225] Handling call
0202 02:52:54.301208 (+4722014us) ts_tablet_manager.cc:1151] Deleting tablet 6750a1212ceb4e6e82e4ffa93caab590
0202 02:52:54.301220 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:54.301370 (+ 150us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4721790}
W20260202 02:52:54.302162 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 6750a1212ceb4e6e82e4ffa93caab590 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6750a1212ceb4e6e82e4ffa93caab590 already in progress: creating tablet
I20260202 02:52:54.302651 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 798) took 4549 ms. Trace:
I20260202 02:52:54.303195 1882 rpcz_store.cc:276] 0202 02:52:49.752719 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:49.752783 (+ 64us) service_pool.cc:225] Handling call
0202 02:52:54.302522 (+4549739us) ts_tablet_manager.cc:1151] Deleting tablet 745fc215251e4ccb9a2abed02573dd84
0202 02:52:54.302534 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:54.302632 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4519084}
W20260202 02:52:54.304131 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 745fc215251e4ccb9a2abed02573dd84 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 745fc215251e4ccb9a2abed02573dd84 already in progress: creating tablet
I20260202 02:52:54.304857 1182 raft_consensus.cc:2243] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:54.305147 1182 raft_consensus.cc:2272] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:54.307392 1173 tablet_bootstrap.cc:492] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:54.307549 1173 ts_tablet_manager.cc:1403] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.030s user 0.003s sys 0.000s
I20260202 02:52:54.307543 1182 ts_tablet_manager.cc:1916] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:54.308683 715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 47474.7us
I20260202 02:52:54.309880 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "528d4269f91a44679acfad16e9fbbe32" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:54.310204 1062 raft_consensus.cc:2468] T 528d4269f91a44679acfad16e9fbbe32 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:54.311791 1175 tablet_bootstrap.cc:492] T 2f664f6119ba4d28b5ef1c212e611866 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:54.311738 1173 raft_consensus.cc:359] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.311901 1173 raft_consensus.cc:385] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:54.311952 1173 raft_consensus.cc:740] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:54.312127 1175 ts_tablet_manager.cc:1403] T 2f664f6119ba4d28b5ef1c212e611866 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.302s user 0.005s sys 0.000s
I20260202 02:52:54.312150 1173 consensus_queue.cc:260] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.312666 1173 ts_tablet_manager.cc:1434] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260202 02:52:54.313030 1175 raft_consensus.cc:359] T 2f664f6119ba4d28b5ef1c212e611866 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:54.313241 1175 raft_consensus.cc:385] T 2f664f6119ba4d28b5ef1c212e611866 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:54.313410 1175 raft_consensus.cc:740] T 2f664f6119ba4d28b5ef1c212e611866 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:54.313740 1175 consensus_queue.cc:260] T 2f664f6119ba4d28b5ef1c212e611866 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:54.314262 1175 ts_tablet_manager.cc:1434] T 2f664f6119ba4d28b5ef1c212e611866 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:54.315445 1175 tablet_bootstrap.cc:492] T 7921daa000f54c8dbed04473e21cf0a5 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:54.318586 1173 tablet_bootstrap.cc:492] T 5e20842de8cb4355b732685605c88834 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:54.319060 1061 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "25a407001f964da6b51f42375136c79a" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:54.319355 1061 raft_consensus.cc:3060] T 25a407001f964da6b51f42375136c79a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:54.319861 715 leader_election.cc:304] T 528d4269f91a44679acfad16e9fbbe32 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:54.320674 2881 raft_consensus.cc:2804] T 528d4269f91a44679acfad16e9fbbe32 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:54.320809 2881 raft_consensus.cc:493] T 528d4269f91a44679acfad16e9fbbe32 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:54.320915 2881 raft_consensus.cc:3060] T 528d4269f91a44679acfad16e9fbbe32 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:54.321385 2794 raft_consensus.cc:493] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:54.321527 2794 raft_consensus.cc:515] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.321934 1061 raft_consensus.cc:2468] T 25a407001f964da6b51f42375136c79a P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:54.322124 2794 leader_election.cc:290] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:54.323820 1173 tablet_bootstrap.cc:654] T 5e20842de8cb4355b732685605c88834 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:54.335338 1173 tablet_bootstrap.cc:492] T 5e20842de8cb4355b732685605c88834 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:54.335486 1173 ts_tablet_manager.cc:1403] T 5e20842de8cb4355b732685605c88834 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.001s
I20260202 02:52:54.336788 528 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: default.Table_f48ed1c7e02349adad7ce22c0e51ed81 [id=46b5d47377474ff9bac2acdb26b08c65] alter complete (version 3)
I20260202 02:52:54.336823 1173 raft_consensus.cc:359] T 5e20842de8cb4355b732685605c88834 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:54.336992 1173 raft_consensus.cc:385] T 5e20842de8cb4355b732685605c88834 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:54.337038 1173 raft_consensus.cc:740] T 5e20842de8cb4355b732685605c88834 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:54.337227 520 master_service.cc:954] ReplaceTablet: received request to replace tablet cb634d4bae554ef3b39eeaa9958adc6c from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:54.339069 715 leader_election.cc:304] T 25a407001f964da6b51f42375136c79a P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:54.283634 1060 raft_consensus.cc:2468] T f3eebefa74584c1791d52377f5ba1290 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:54.343142 1173 consensus_queue.cc:260] T 5e20842de8cb4355b732685605c88834 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:54.343680 1173 ts_tablet_manager.cc:1434] T 5e20842de8cb4355b732685605c88834 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260202 02:52:54.343961 1173 tablet_bootstrap.cc:492] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:54.345623 1173 tablet_bootstrap.cc:654] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:54.350172 715 leader_election.cc:304] T f3eebefa74584c1791d52377f5ba1290 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:54.354091 2979 raft_consensus.cc:493] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:54.354324 2979 raft_consensus.cc:515] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.355396 2979 leader_election.cc:290] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:54.357978 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 6a3dac5cd1ce47488a50b4c32eca67c7 (table default.Table_25f0023dd22a4bb0a7c2863da66d0002 [id=e24ffad12f514d60907b6cadfd09dbe6]) successfully deleted
I20260202 02:52:54.359457 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet c8a273ac8cd344719a48fd81945bed0a (table default.Table_5a703dc84eb34674bdc5793ac7295098 [id=62c3c39fb73249349cee49b86d535612]) successfully deleted
I20260202 02:52:54.359608 520 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet cb634d4bae554ef3b39eeaa9958adc6c
I20260202 02:52:54.360158 520 catalog_manager.cc:6724] ReplaceTablet: tablet cb634d4bae554ef3b39eeaa9958adc6c deleted and replaced by tablet 3cb014efbe9e46eab97e71fff0ff2887
I20260202 02:52:54.361356 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 67a8c9a1998b47b3b214a06ce62381e9
I20260202 02:52:54.361845 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet cef87d5f74f14a6d86150958de7b1ace
I20260202 02:52:54.362200 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 1959438c70c544e69422f7354082cf89
I20260202 02:52:54.362557 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 9f11a68d10374d06bc4a756a1d15bf09
I20260202 02:52:54.362951 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet bbcc723a8079443db455ae20fb0d585d
I20260202 02:52:54.363343 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 9a1d7d173c41462c8e32a95b494bb708
I20260202 02:52:54.363682 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 7f323175ee254df789bcb04e65047dff
I20260202 02:52:54.364114 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 4a518e8ad2e34ec98a0425c7ed3a470a
I20260202 02:52:54.365926 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet cef87d5f74f14a6d86150958de7b1ace with delete_type TABLET_DATA_DELETED (Replaced by cc7cb7a8cadc4c90a2449f9d7183aa9a at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:54.366178 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 1959438c70c544e69422f7354082cf89 with delete_type TABLET_DATA_DELETED (Replaced by 223bf8b5e2684e49a316a3a654a0bf1a at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:54.366178 1032 tablet_service.cc:1552] Processing DeleteTablet for tablet abfe7f76b2424c4d9ab07b29b66ec103 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:47 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:54.366425 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 7f323175ee254df789bcb04e65047dff with delete_type TABLET_DATA_DELETED (Replaced by 5a60685d08b249bfbc80da8c2ac9b28f at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:54.366597 1032 tablet_service.cc:1552] Processing DeleteTablet for tablet 59ad016e15ec4a488921eb564029b689 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:21 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:54.366672 1044 tablet_service.cc:1552] Processing DeleteTablet for tablet 9f11a68d10374d06bc4a756a1d15bf09 with delete_type TABLET_DATA_DELETED (Replaced by da9d3228ad734065bfc640833c988c92 at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:54.366745 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 4a518e8ad2e34ec98a0425c7ed3a470a with delete_type TABLET_DATA_DELETED (Replaced by 6bce8dce844341f4982a44e1e90002e5 at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:54.366859 1035 tablet_service.cc:1552] Processing DeleteTablet for tablet 8bb14bf551f441dcbaaff7057b51b2be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:54.366907 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet 9a1d7d173c41462c8e32a95b494bb708 with delete_type TABLET_DATA_DELETED (Replaced by 862047d463374985ae0562f86ca2089d at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:54.366984 1031 tablet_service.cc:1552] Processing DeleteTablet for tablet 93f958924885457ab15f2aac5509e0aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:25 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:54.367126 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet dc90eeb79ded4739883ad98928476680 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:54.368162 418 master-stress-test.cc:579] Killed test thread 21/25
I20260202 02:52:54.368253 418 master-stress-test.cc:577] Killing test thread 22/25
I20260202 02:52:54.368266 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet c17c85759e964f0ba3abcaf5fe938fba
I20260202 02:52:54.368707 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 604b3b863c9540d3b55381dbfe1afdc3
I20260202 02:52:54.369020 1039 tablet_service.cc:1552] Processing DeleteTablet for tablet bbcc723a8079443db455ae20fb0d585d with delete_type TABLET_DATA_DELETED (Replaced by 39d9e3fef89d4c1cb053278e25796eaf at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:54.373302 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 09504070c9ec4bcfbb59e3fb4c189775
I20260202 02:52:54.374120 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet 604b3b863c9540d3b55381dbfe1afdc3 with delete_type TABLET_DATA_DELETED (Replaced by 63c8ecaa42224d2ba0cc9cf26a05e698 at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:54.374315 1039 tablet_service.cc:1552] Processing DeleteTablet for tablet c17c85759e964f0ba3abcaf5fe938fba with delete_type TABLET_DATA_DELETED (Replaced by e2fb216334a245e08617d18c836f1fbf at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:54.317876 1182 ts_tablet_manager.cc:1929] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:54.375275 1182 log.cc:1199] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/8b0368e03f644f1db28dbb57c95c48c1
I20260202 02:52:54.329692 3080 consensus_queue.cc:1059] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260202 02:52:54.375998 3076 raft_consensus.cc:3055] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [term 2 LEADER]: Stepping down as leader of term 2
I20260202 02:52:54.376086 3076 raft_consensus.cc:740] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Running, Role: LEADER
I20260202 02:52:54.376374 3076 consensus_queue.cc:260] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:54.376684 3076 raft_consensus.cc:3060] T cb634d4bae554ef3b39eeaa9958adc6c P 1521a6d262b7451bbef8debe970dd6b5 [term 2 FOLLOWER]: Advancing to term 3
I20260202 02:52:54.377084 1175 tablet_bootstrap.cc:654] T 7921daa000f54c8dbed04473e21cf0a5 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:54.155242 1180 ts_tablet_manager.cc:1929] T 79c73f4a92044bafb5b8384229cb3a4d P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:54.377902 1042 tablet_service.cc:1552] Processing DeleteTablet for tablet 67a8c9a1998b47b3b214a06ce62381e9 with delete_type TABLET_DATA_DELETED (Replaced by 32ee0da311f1453d8a2d30f9d8be6702 at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:54.377982 1180 log.cc:1199] T 79c73f4a92044bafb5b8384229cb3a4d P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/79c73f4a92044bafb5b8384229cb3a4d
I20260202 02:52:54.378432 1180 ts_tablet_manager.cc:1950] T 79c73f4a92044bafb5b8384229cb3a4d P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:54.378888 1042 tablet_service.cc:1505] Processing CreateTablet for tablet f415a9ff427447d8a71829a4878dd7ac (DEFAULT_TABLE table=default.Table_76012bec9c634dbb848a193ad3c6d4e0 [id=88ffd0beda5b4608af047540f269f472]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.379256 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f415a9ff427447d8a71829a4878dd7ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.379233 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 744) took 6436 ms. Trace:
I20260202 02:52:54.379310 1180 rpcz_store.cc:276] 0202 02:52:47.942647 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:47.964198 (+ 21551us) service_pool.cc:225] Handling call
0202 02:52:53.989593 (+6025395us) ts_tablet_manager.cc:1151] Deleting tablet 79c73f4a92044bafb5b8384229cb3a4d
0202 02:52:53.989605 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:54.152153 (+162548us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:54.155136 (+ 2983us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:54.379209 (+224073us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5905116}
I20260202 02:52:54.379805 1039 tablet_service.cc:1552] Processing DeleteTablet for tablet 09504070c9ec4bcfbb59e3fb4c189775 with delete_type TABLET_DATA_DELETED (Replaced by 6c654857b0e74e399b16601c9ebf692e at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:54.379858 1180 tablet_replica.cc:333] T 842e54adae2a452a98521de24326eaff P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:54.380074 1180 raft_consensus.cc:2243] T 842e54adae2a452a98521de24326eaff P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:54.380239 1180 raft_consensus.cc:2272] T 842e54adae2a452a98521de24326eaff P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:54.380455 1182 ts_tablet_manager.cc:1950] T 8b0368e03f644f1db28dbb57c95c48c1 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:54.380807 1180 ts_tablet_manager.cc:1916] T 842e54adae2a452a98521de24326eaff P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:54.380829 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 907) took 1514 ms. Trace:
I20260202 02:52:54.380983 1182 rpcz_store.cc:276] 0202 02:52:52.865948 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.866061 (+ 113us) service_pool.cc:225] Handling call
0202 02:52:54.300902 (+1434841us) ts_tablet_manager.cc:1151] Deleting tablet 8b0368e03f644f1db28dbb57c95c48c1
0202 02:52:54.300913 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:54.314717 (+ 13804us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:54.317753 (+ 3036us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:54.380812 (+ 63059us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1434668}
I20260202 02:52:54.381570 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 909) took 1515 ms. Trace:
I20260202 02:52:54.381696 1182 rpcz_store.cc:276] 0202 02:52:52.866154 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.866200 (+ 46us) service_pool.cc:225] Handling call
0202 02:52:54.381422 (+1515222us) ts_tablet_manager.cc:1151] Deleting tablet efe78242e0654a1c9bdbaf9cfa88b034
0202 02:52:54.381433 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:54.381554 (+ 121us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1515098}
I20260202 02:52:54.382016 1182 tablet_replica.cc:333] T b427c7e897c64977b924e7d586420e69 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:54.382242 1182 raft_consensus.cc:2243] T b427c7e897c64977b924e7d586420e69 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:54.382439 1182 raft_consensus.cc:2272] T b427c7e897c64977b924e7d586420e69 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:54.383024 1182 ts_tablet_manager.cc:1916] T b427c7e897c64977b924e7d586420e69 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:54.384452 999 leader_election.cc:304] T 25a407001f964da6b51f42375136c79a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters: 1f442ef85029461091de847118c34c0d
I20260202 02:52:54.385030 1040 tablet_service.cc:1552] Processing DeleteTablet for tablet cb634d4bae554ef3b39eeaa9958adc6c with delete_type TABLET_DATA_DELETED (replaced by tablet 3cb014efbe9e46eab97e71fff0ff2887) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:54.385497 999 leader_election.cc:343] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: dc90eeb79ded4739883ad98928476680
W20260202 02:52:54.385979 999 leader_election.cc:343] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: dc90eeb79ded4739883ad98928476680
W20260202 02:52:54.386413 999 leader_election.cc:343] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 59ad016e15ec4a488921eb564029b689
I20260202 02:52:54.387153 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet f5727f9b9bbb4e77abfef065c51606a1 (table default.Table_4e6c3886f58b49698a00d45b957f4c9a [id=b5cc03b77f414304b29c5fb4bb7dc3ee]) successfully deleted
I20260202 02:52:54.387250 1040 tablet_service.cc:1505] Processing CreateTablet for tablet ff19d6c3a53a446ea9798c3c160ad7c6 (DEFAULT_TABLE table=default.Table_070200300d5d4886b185f9a99aa77100 [id=3630f9706deb453990898d296903ac1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.387658 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff19d6c3a53a446ea9798c3c160ad7c6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:54.387727 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet eea6bb04ddf24276b46270d009ed78b4 because the tablet was not found. No further retry: Not found: Tablet not found: eea6bb04ddf24276b46270d009ed78b4
W20260202 02:52:54.388151 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet fa57a267b0484262ac31039862f40a76 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fa57a267b0484262ac31039862f40a76 already in progress: creating tablet
I20260202 02:52:54.388460 2979 raft_consensus.cc:2764] T 25a407001f964da6b51f42375136c79a P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260202 02:52:54.389183 1182 ts_tablet_manager.cc:1929] T b427c7e897c64977b924e7d586420e69 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:54.390558 1182 log.cc:1199] T b427c7e897c64977b924e7d586420e69 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/b427c7e897c64977b924e7d586420e69
I20260202 02:52:54.392190 1182 ts_tablet_manager.cc:1950] T b427c7e897c64977b924e7d586420e69 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:54.392948 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 906) took 1527 ms. Trace:
I20260202 02:52:54.393477 1182 rpcz_store.cc:276] 0202 02:52:52.865869 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.866212 (+ 343us) service_pool.cc:225] Handling call
0202 02:52:54.381972 (+1515760us) ts_tablet_manager.cc:1151] Deleting tablet b427c7e897c64977b924e7d586420e69
0202 02:52:54.381982 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:54.386088 (+ 4106us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:54.389075 (+ 2987us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:54.392930 (+ 3855us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1515614}
I20260202 02:52:54.395226 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 910) took 1513 ms. Trace:
I20260202 02:52:54.395478 765 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6a3dac5cd1ce47488a50b4c32eca67c7" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:54.395768 765 raft_consensus.cc:2468] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:54.395687 1182 rpcz_store.cc:276] 0202 02:52:52.881294 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.881357 (+ 63us) service_pool.cc:225] Handling call
0202 02:52:54.395108 (+1513751us) ts_tablet_manager.cc:1151] Deleting tablet 7709ea34429e4e91ad4922a3d73ebd14
0202 02:52:54.395120 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:54.395205 (+ 85us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1513559}
I20260202 02:52:54.395963 1182 tablet_replica.cc:333] T 528d4269f91a44679acfad16e9fbbe32 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
W20260202 02:52:54.396001 999 leader_election.cc:343] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 59ad016e15ec4a488921eb564029b689
I20260202 02:52:54.396518 1182 raft_consensus.cc:2243] T 528d4269f91a44679acfad16e9fbbe32 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:54.396667 3080 raft_consensus.cc:493] T 09504070c9ec4bcfbb59e3fb4c189775 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:54.397483 3080 raft_consensus.cc:515] T 09504070c9ec4bcfbb59e3fb4c189775 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:54.398085 3080 leader_election.cc:290] T 09504070c9ec4bcfbb59e3fb4c189775 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:54.398420 1040 tablet_service.cc:1505] Processing CreateTablet for tablet 9bae6604ddc54fc5bf3423930fa44d9a (DEFAULT_TABLE table=default.Table_8aca64ada6af469b9946c447af43c260 [id=0ef28837669f4d48b1fa4ecf318ddff9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.396986 2964 raft_consensus.cc:493] T bbcc723a8079443db455ae20fb0d585d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:54.398734 1061 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "09504070c9ec4bcfbb59e3fb4c189775" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:54.398769 915 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5e20842de8cb4355b732685605c88834" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:54.399026 915 raft_consensus.cc:3060] T 5e20842de8cb4355b732685605c88834 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:54.399040 2964 raft_consensus.cc:515] T bbcc723a8079443db455ae20fb0d585d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
W20260202 02:52:54.399232 859 leader_election.cc:343] T 09504070c9ec4bcfbb59e3fb4c189775 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:54.399741 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bae6604ddc54fc5bf3423930fa44d9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.400137 1061 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bbcc723a8079443db455ae20fb0d585d" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:54.400872 859 leader_election.cc:343] T bbcc723a8079443db455ae20fb0d585d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:54.401682 915 raft_consensus.cc:2468] T 5e20842de8cb4355b732685605c88834 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:54.403376 2964 leader_election.cc:290] T bbcc723a8079443db455ae20fb0d585d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:54.404042 1182 raft_consensus.cc:2272] T 528d4269f91a44679acfad16e9fbbe32 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:54.404989 1182 ts_tablet_manager.cc:1916] T 528d4269f91a44679acfad16e9fbbe32 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:54.405925 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 797) took 4663 ms. Trace:
I20260202 02:52:54.406067 1882 rpcz_store.cc:276] 0202 02:52:49.742169 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:49.742999 (+ 830us) service_pool.cc:225] Handling call
0202 02:52:54.405772 (+4662773us) ts_tablet_manager.cc:1151] Deleting tablet 42321f4e77484027a485fdd584de3ae3
0202 02:52:54.405784 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:54.405902 (+ 118us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4622267}
I20260202 02:52:54.406916 1882 tablet_replica.cc:333] T 6df3d11cf578404ca41807f0777b6c53 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:54.407186 1882 raft_consensus.cc:2243] T 6df3d11cf578404ca41807f0777b6c53 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:54.407438 1882 raft_consensus.cc:2272] T 6df3d11cf578404ca41807f0777b6c53 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:54.408185 1882 ts_tablet_manager.cc:1916] T 6df3d11cf578404ca41807f0777b6c53 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:54.408977 1039 tablet_service.cc:1505] Processing CreateTablet for tablet f26e31992cc544ffa97ba78e8e47537c (DEFAULT_TABLE table=default.Table_8c3c9c0b564041f99e2006f890fc7559 [id=ffb64501dfa04daabe4aa55567608b77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.409339 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f26e31992cc544ffa97ba78e8e47537c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.409467 1175 tablet_bootstrap.cc:492] T 7921daa000f54c8dbed04473e21cf0a5 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:54.409670 1175 ts_tablet_manager.cc:1403] T 7921daa000f54c8dbed04473e21cf0a5 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.094s user 0.001s sys 0.002s
I20260202 02:52:54.410523 1175 raft_consensus.cc:359] T 7921daa000f54c8dbed04473e21cf0a5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:54.410802 1175 raft_consensus.cc:385] T 7921daa000f54c8dbed04473e21cf0a5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:54.410889 1175 raft_consensus.cc:740] T 7921daa000f54c8dbed04473e21cf0a5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:54.411149 1175 consensus_queue.cc:260] T 7921daa000f54c8dbed04473e21cf0a5 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:54.411888 1175 ts_tablet_manager.cc:1434] T 7921daa000f54c8dbed04473e21cf0a5 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:54.411979 915 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "51ba0bcd498043c383c493f45ad1fe17" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:54.412137 915 raft_consensus.cc:3060] T 51ba0bcd498043c383c493f45ad1fe17 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:54.412213 1175 tablet_bootstrap.cc:492] T 1959438c70c544e69422f7354082cf89 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:54.414264 1175 tablet_bootstrap.cc:654] T 1959438c70c544e69422f7354082cf89 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:54.414273 2881 raft_consensus.cc:515] T 528d4269f91a44679acfad16e9fbbe32 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.415242 2881 leader_election.cc:290] T 528d4269f91a44679acfad16e9fbbe32 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:54.415457 2881 raft_consensus.cc:2804] T 25a407001f964da6b51f42375136c79a P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:54.415616 2881 raft_consensus.cc:697] T 25a407001f964da6b51f42375136c79a P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:54.415910 2881 consensus_queue.cc:237] T 25a407001f964da6b51f42375136c79a P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.415156 915 raft_consensus.cc:2468] T 51ba0bcd498043c383c493f45ad1fe17 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:54.419596 3143 tablet.cc:1722] T 44c8d5b74c8e4c499f7f68e8185efc17 P 1521a6d262b7451bbef8debe970dd6b5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260202 02:52:54.428771 1039 tablet_service.cc:1505] Processing CreateTablet for tablet 9990414ac8654f9a8cc25a19f96b5e5e (DEFAULT_TABLE table=default.Table_320a9a8a16b949a8a4dfe8fec6a9e5a4 [id=e4f93251a3a24760b93813e187f0f6be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.429895 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9990414ac8654f9a8cc25a19f96b5e5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.431016 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet 7ff6d2fa085343a6b11be0898da6233d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:54.431914 1031 tablet_service.cc:1552] Processing DeleteTablet for tablet 5e20842de8cb4355b732685605c88834 with delete_type TABLET_DATA_DELETED (replaced by tablet 34a01a4ce1be4ad08d911763de158073) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:54.432098 1038 tablet_service.cc:1552] Processing DeleteTablet for tablet b4a4c75a906f40beb896bf6fff705ae6 with delete_type TABLET_DATA_DELETED (replaced by tablet af29c7a2ad9d4797b08052606151a7d5) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:54.432355 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a3dac5cd1ce47488a50b4c32eca67c7 with delete_type TABLET_DATA_DELETED (replaced by tablet 365417e050304073b5de887082fc699b) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:54.436672 1175 tablet_bootstrap.cc:492] T 1959438c70c544e69422f7354082cf89 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:54.440070 526 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 7c07195a46184d24bc709e4967945472
I20260202 02:52:54.440775 526 catalog_manager.cc:6724] ReplaceTablet: tablet 7c07195a46184d24bc709e4967945472 deleted and replaced by tablet 4d37d9e85b7345379e6bbeaa8cabd182
I20260202 02:52:54.441867 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet 7c07195a46184d24bc709e4967945472 with delete_type TABLET_DATA_DELETED (replaced by tablet 4d37d9e85b7345379e6bbeaa8cabd182) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:54.445276 1175 ts_tablet_manager.cc:1403] T 1959438c70c544e69422f7354082cf89 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.033s user 0.003s sys 0.000s
I20260202 02:52:54.445420 418 master-stress-test.cc:579] Killed test thread 22/25
I20260202 02:52:54.445528 418 master-stress-test.cc:577] Killing test thread 23/25
I20260202 02:52:54.446470 1175 raft_consensus.cc:359] T 1959438c70c544e69422f7354082cf89 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:54.448101 1027 tablet_service.cc:1505] Processing CreateTablet for tablet a5aa442344734b0f98f8053bafae3c50 (DEFAULT_TABLE table=default.Table_bd2d37a87601403eb2a45621e900472b [id=7fbbecd4c36c45a0987b9a5559c452c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.448529 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5aa442344734b0f98f8053bafae3c50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.448827 1039 tablet_service.cc:1505] Processing CreateTablet for tablet 31a6358735ab477eb1229be3264b82a6 (DEFAULT_TABLE table=default.Table_20f2fd4d31db4d66be77799ede02027e [id=cfcc4121e4f54862b31b9e643a842b74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.454393 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 8b0368e03f644f1db28dbb57c95c48c1 (table default.Table_f8d8615b34774b738faeb83039f9601a [id=8882e1fbac2949838c99adb1290f273e]) successfully deleted
W20260202 02:52:54.455060 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet efe78242e0654a1c9bdbaf9cfa88b034 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet efe78242e0654a1c9bdbaf9cfa88b034 already in progress: creating tablet
I20260202 02:52:54.461373 1174 tablet_bootstrap.cc:492] T 7ff6d2fa085343a6b11be0898da6233d P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:54.461881 759 tablet_service.cc:1552] Processing DeleteTablet for tablet b4a4c75a906f40beb896bf6fff705ae6 with delete_type TABLET_DATA_DELETED (replaced by tablet af29c7a2ad9d4797b08052606151a7d5) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:54.462111 740 tablet_service.cc:1552] Processing DeleteTablet for tablet cb634d4bae554ef3b39eeaa9958adc6c with delete_type TABLET_DATA_DELETED (replaced by tablet 3cb014efbe9e46eab97e71fff0ff2887) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:54.462165 1174 ts_tablet_manager.cc:1403] T 7ff6d2fa085343a6b11be0898da6233d P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.174s user 0.011s sys 0.011s
I20260202 02:52:54.469146 765 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7f9dfac19c04475aadb45e95a6474eb4" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:54.469254 1175 raft_consensus.cc:385] T 1959438c70c544e69422f7354082cf89 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:54.469334 1175 raft_consensus.cc:740] T 1959438c70c544e69422f7354082cf89 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:54.471298 1175 consensus_queue.cc:260] T 1959438c70c544e69422f7354082cf89 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:54.472101 1175 ts_tablet_manager.cc:1434] T 1959438c70c544e69422f7354082cf89 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.026s user 0.002s sys 0.000s
I20260202 02:52:54.473778 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 19491b92d4b044b9a47e0c72d634660d (DEFAULT_TABLE table=default.Table_78bb96bdb6a1489f94628205c7ccacfd [id=1f18a45584174b10878d792cb3e3ab97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.451913 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "25a407001f964da6b51f42375136c79a" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:54.474408 913 raft_consensus.cc:3060] T 25a407001f964da6b51f42375136c79a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:54.476011 1040 tablet_service.cc:1505] Processing CreateTablet for tablet 70540beb5e924667b61fe432bda26537 (DEFAULT_TABLE table=default.Table_4d795c06694e40cf9bf6b637784d3ea4 [id=e8b303fff78c46d58d66701d8a93cae3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.477171 913 raft_consensus.cc:2468] T 25a407001f964da6b51f42375136c79a P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:54.450986 915 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "25a407001f964da6b51f42375136c79a" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:54.477826 915 raft_consensus.cc:2376] T 25a407001f964da6b51f42375136c79a P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1. Re-sending same reply.
I20260202 02:52:54.479131 1042 tablet_service.cc:1505] Processing CreateTablet for tablet df5a29f2cd144373a75eb0853410e056 (DEFAULT_TABLE table=default.Table_b64e770cf31548baa0b24b084ea9f939 [id=c5b2ccfa53ac4e32ad22423db2e13116]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.481869 745 tablet_service.cc:1552] Processing DeleteTablet for tablet f1c8f7e1dc9644a6b6dd2bc264cce8ef with delete_type TABLET_DATA_DELETED (replaced by tablet 21fe651f45bf43609cf972bef5eca149) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:54.482116 740 tablet_service.cc:1552] Processing DeleteTablet for tablet 67a8c9a1998b47b3b214a06ce62381e9 with delete_type TABLET_DATA_DELETED (Replaced by 32ee0da311f1453d8a2d30f9d8be6702 at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:54.485698 1038 tablet_service.cc:1505] Processing CreateTablet for tablet ad244ef168034c3fb8fd86d2e732f1ef (DEFAULT_TABLE table=default.Table_92f45fe4aded4c79816e651a2be7ff27 [id=47aa39ab2d934b34b4251c06977086ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.486917 740 tablet_service.cc:1552] Processing DeleteTablet for tablet cef87d5f74f14a6d86150958de7b1ace with delete_type TABLET_DATA_DELETED (Replaced by cc7cb7a8cadc4c90a2449f9d7183aa9a at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:54.488306 740 tablet_service.cc:1552] Processing DeleteTablet for tablet 1959438c70c544e69422f7354082cf89 with delete_type TABLET_DATA_DELETED (Replaced by 223bf8b5e2684e49a316a3a654a0bf1a at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:54.488734 745 tablet_service.cc:1552] Processing DeleteTablet for tablet 9f11a68d10374d06bc4a756a1d15bf09 with delete_type TABLET_DATA_DELETED (Replaced by da9d3228ad734065bfc640833c988c92 at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:54.488979 1031 tablet_service.cc:1505] Processing CreateTablet for tablet 600c8f073d0f465fad7b69adb7dfff4e (DEFAULT_TABLE table=default.Table_768da74ea5aa47e187113ae83e1a5b72 [id=e0863fb79e6f4b378d4f54719b67003e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.489408 759 tablet_service.cc:1552] Processing DeleteTablet for tablet 7f323175ee254df789bcb04e65047dff with delete_type TABLET_DATA_DELETED (Replaced by 5a60685d08b249bfbc80da8c2ac9b28f at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:54.489606 740 tablet_service.cc:1552] Processing DeleteTablet for tablet 9a1d7d173c41462c8e32a95b494bb708 with delete_type TABLET_DATA_DELETED (Replaced by 862047d463374985ae0562f86ca2089d at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:54.490077 745 tablet_service.cc:1552] Processing DeleteTablet for tablet bbcc723a8079443db455ae20fb0d585d with delete_type TABLET_DATA_DELETED (Replaced by 39d9e3fef89d4c1cb053278e25796eaf at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:54.492820 1175 tablet_bootstrap.cc:492] T 842197c689064660b791b40e521d346b P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:54.493592 740 tablet_service.cc:1552] Processing DeleteTablet for tablet c17c85759e964f0ba3abcaf5fe938fba with delete_type TABLET_DATA_DELETED (Replaced by e2fb216334a245e08617d18c836f1fbf at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:54.493878 745 tablet_service.cc:1552] Processing DeleteTablet for tablet 4a518e8ad2e34ec98a0425c7ed3a470a with delete_type TABLET_DATA_DELETED (Replaced by 6bce8dce844341f4982a44e1e90002e5 at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:54.494493 1175 tablet_bootstrap.cc:654] T 842197c689064660b791b40e521d346b P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:54.494853 745 tablet_service.cc:1552] Processing DeleteTablet for tablet 604b3b863c9540d3b55381dbfe1afdc3 with delete_type TABLET_DATA_DELETED (Replaced by 63c8ecaa42224d2ba0cc9cf26a05e698 at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:54.495362 3144 raft_consensus.cc:2804] T f3eebefa74584c1791d52377f5ba1290 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:54.498132 527 master_service.cc:954] ReplaceTablet: received request to replace tablet dc90eeb79ded4739883ad98928476680 from {username='slave'} at 127.0.0.1:47430
I20260202 02:52:54.499531 2977 raft_consensus.cc:493] T abfe7f76b2424c4d9ab07b29b66ec103 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:54.499872 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet b427c7e897c64977b924e7d586420e69 (table default.Table_616a8ade4b5e481b8d4906b7a2a026d0 [id=98ee1a7044cd46d8b39dfda755ed8a7d]) successfully deleted
I20260202 02:52:54.500028 2977 raft_consensus.cc:515] T abfe7f76b2424c4d9ab07b29b66ec103 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
W20260202 02:52:54.500411 514 catalog_manager.cc:4952] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 7709ea34429e4e91ad4922a3d73ebd14 because the tablet was not found. No further retry: Not found: Tablet not found: 7709ea34429e4e91ad4922a3d73ebd14
I20260202 02:52:54.501983 2977 leader_election.cc:290] T abfe7f76b2424c4d9ab07b29b66ec103 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:54.502725 418 master-stress-test.cc:579] Killed test thread 23/25
I20260202 02:52:54.502833 418 master-stress-test.cc:577] Killing test thread 24/25
I20260202 02:52:54.502918 418 master-stress-test.cc:579] Killed test thread 24/25
I20260202 02:52:54.502975 418 master-stress-test.cc:577] Killing test thread 25/25
I20260202 02:52:54.503051 418 master-stress-test.cc:579] Killed test thread 25/25
I20260202 02:52:54.503208 418 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskYXxklX/build/asan/bin/kudu pid 696
I20260202 02:52:54.505641 753 tablet_service.cc:1552] Processing DeleteTablet for tablet 7c07195a46184d24bc709e4967945472 with delete_type TABLET_DATA_DELETED (replaced by tablet 4d37d9e85b7345379e6bbeaa8cabd182) from {username='slave'} at 127.0.0.1:42950
W20260202 02:52:54.506243 999 leader_election.cc:343] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 6a3dac5cd1ce47488a50b4c32eca67c7
I20260202 02:52:54.506258 753 tablet_service.cc:1505] Processing CreateTablet for tablet 70540beb5e924667b61fe432bda26537 (DEFAULT_TABLE table=default.Table_4d795c06694e40cf9bf6b637784d3ea4 [id=e8b303fff78c46d58d66701d8a93cae3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.506673 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70540beb5e924667b61fe432bda26537. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:54.506903 999 leader_election.cc:343] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: b4a4c75a906f40beb896bf6fff705ae6
I20260202 02:52:54.507020 748 tablet_service.cc:1505] Processing CreateTablet for tablet 9bae6604ddc54fc5bf3423930fa44d9a (DEFAULT_TABLE table=default.Table_8aca64ada6af469b9946c447af43c260 [id=0ef28837669f4d48b1fa4ecf318ddff9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.507378 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bae6604ddc54fc5bf3423930fa44d9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.507345 899 tablet_service.cc:1552] Processing DeleteTablet for tablet cb634d4bae554ef3b39eeaa9958adc6c with delete_type TABLET_DATA_DELETED (replaced by tablet 3cb014efbe9e46eab97e71fff0ff2887) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:54.507694 899 tablet_service.cc:1552] Processing DeleteTablet for tablet 67a8c9a1998b47b3b214a06ce62381e9 with delete_type TABLET_DATA_DELETED (Replaced by 32ee0da311f1453d8a2d30f9d8be6702 at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:54.507957 899 tablet_service.cc:1552] Processing DeleteTablet for tablet cef87d5f74f14a6d86150958de7b1ace with delete_type TABLET_DATA_DELETED (Replaced by cc7cb7a8cadc4c90a2449f9d7183aa9a at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:54.508297 899 tablet_service.cc:1552] Processing DeleteTablet for tablet 1959438c70c544e69422f7354082cf89 with delete_type TABLET_DATA_DELETED (Replaced by 223bf8b5e2684e49a316a3a654a0bf1a at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:54.508618 899 tablet_service.cc:1552] Processing DeleteTablet for tablet 9f11a68d10374d06bc4a756a1d15bf09 with delete_type TABLET_DATA_DELETED (Replaced by da9d3228ad734065bfc640833c988c92 at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:54.508944 899 tablet_service.cc:1552] Processing DeleteTablet for tablet bbcc723a8079443db455ae20fb0d585d with delete_type TABLET_DATA_DELETED (Replaced by 39d9e3fef89d4c1cb053278e25796eaf at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:54.509255 899 tablet_service.cc:1552] Processing DeleteTablet for tablet 9a1d7d173c41462c8e32a95b494bb708 with delete_type TABLET_DATA_DELETED (Replaced by 862047d463374985ae0562f86ca2089d at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:54.509569 899 tablet_service.cc:1552] Processing DeleteTablet for tablet 7f323175ee254df789bcb04e65047dff with delete_type TABLET_DATA_DELETED (Replaced by 5a60685d08b249bfbc80da8c2ac9b28f at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:54.509893 899 tablet_service.cc:1552] Processing DeleteTablet for tablet 4a518e8ad2e34ec98a0425c7ed3a470a with delete_type TABLET_DATA_DELETED (Replaced by 6bce8dce844341f4982a44e1e90002e5 at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:54.509985 745 tablet_service.cc:1552] Processing DeleteTablet for tablet 09504070c9ec4bcfbb59e3fb4c189775 with delete_type TABLET_DATA_DELETED (Replaced by 6c654857b0e74e399b16601c9ebf692e at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:54.510233 899 tablet_service.cc:1552] Processing DeleteTablet for tablet c17c85759e964f0ba3abcaf5fe938fba with delete_type TABLET_DATA_DELETED (Replaced by e2fb216334a245e08617d18c836f1fbf at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:54.510618 899 tablet_service.cc:1552] Processing DeleteTablet for tablet 604b3b863c9540d3b55381dbfe1afdc3 with delete_type TABLET_DATA_DELETED (Replaced by 63c8ecaa42224d2ba0cc9cf26a05e698 at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:54.510725 755 tablet_service.cc:1505] Processing CreateTablet for tablet f26e31992cc544ffa97ba78e8e47537c (DEFAULT_TABLE table=default.Table_8c3c9c0b564041f99e2006f890fc7559 [id=ffb64501dfa04daabe4aa55567608b77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.510963 899 tablet_service.cc:1552] Processing DeleteTablet for tablet 09504070c9ec4bcfbb59e3fb4c189775 with delete_type TABLET_DATA_DELETED (Replaced by 6c654857b0e74e399b16601c9ebf692e at 2026-02-02 02:52:54 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:54.511111 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f26e31992cc544ffa97ba78e8e47537c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.511566 899 tablet_service.cc:1505] Processing CreateTablet for tablet f415a9ff427447d8a71829a4878dd7ac (DEFAULT_TABLE table=default.Table_76012bec9c634dbb848a193ad3c6d4e0 [id=88ffd0beda5b4608af047540f269f472]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.511914 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f415a9ff427447d8a71829a4878dd7ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.512132 744 tablet_service.cc:1505] Processing CreateTablet for tablet 19491b92d4b044b9a47e0c72d634660d (DEFAULT_TABLE table=default.Table_78bb96bdb6a1489f94628205c7ccacfd [id=1f18a45584174b10878d792cb3e3ab97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.512403 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31a6358735ab477eb1229be3264b82a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.512480 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19491b92d4b044b9a47e0c72d634660d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.514003 757 tablet_service.cc:1505] Processing CreateTablet for tablet 600c8f073d0f465fad7b69adb7dfff4e (DEFAULT_TABLE table=default.Table_768da74ea5aa47e187113ae83e1a5b72 [id=e0863fb79e6f4b378d4f54719b67003e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.514323 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 600c8f073d0f465fad7b69adb7dfff4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.514940 759 tablet_service.cc:1505] Processing CreateTablet for tablet ff19d6c3a53a446ea9798c3c160ad7c6 (DEFAULT_TABLE table=default.Table_070200300d5d4886b185f9a99aa77100 [id=3630f9706deb453990898d296903ac1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.515244 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff19d6c3a53a446ea9798c3c160ad7c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.515525 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 79c73f4a92044bafb5b8384229cb3a4d (table default.Table_a15c97bb32284dcfb36cdc47921b2346 [id=dc77705cb5704fd58af1eed3edd45675]) successfully deleted
W20260202 02:52:54.516682 714 leader_election.cc:343] T f3eebefa74584c1791d52377f5ba1290 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: f3eebefa74584c1791d52377f5ba1290
W20260202 02:52:54.517357 714 leader_election.cc:343] T 99e0b5e9a9874aec886155e0d614dec2 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 99e0b5e9a9874aec886155e0d614dec2
I20260202 02:52:54.518075 1174 raft_consensus.cc:359] T 7ff6d2fa085343a6b11be0898da6233d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.518244 1174 raft_consensus.cc:385] T 7ff6d2fa085343a6b11be0898da6233d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:54.518290 1174 raft_consensus.cc:740] T 7ff6d2fa085343a6b11be0898da6233d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:54.518451 1174 consensus_queue.cc:260] T 7ff6d2fa085343a6b11be0898da6233d P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.518898 1174 ts_tablet_manager.cc:1434] T 7ff6d2fa085343a6b11be0898da6233d P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.056s user 0.001s sys 0.001s
I20260202 02:52:54.519163 1174 tablet_bootstrap.cc:492] T f5727f9b9bbb4e77abfef065c51606a1 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:54.520531 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 600c8f073d0f465fad7b69adb7dfff4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.520992 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df5a29f2cd144373a75eb0853410e056. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.524861 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad244ef168034c3fb8fd86d2e732f1ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.530633 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19491b92d4b044b9a47e0c72d634660d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.530974 741 tablet_service.cc:1505] Processing CreateTablet for tablet 31a6358735ab477eb1229be3264b82a6 (DEFAULT_TABLE table=default.Table_20f2fd4d31db4d66be77799ede02027e [id=cfcc4121e4f54862b31b9e643a842b74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.531365 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31a6358735ab477eb1229be3264b82a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.533720 749 tablet_service.cc:1505] Processing CreateTablet for tablet 9990414ac8654f9a8cc25a19f96b5e5e (DEFAULT_TABLE table=default.Table_320a9a8a16b949a8a4dfe8fec6a9e5a4 [id=e4f93251a3a24760b93813e187f0f6be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.534132 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9990414ac8654f9a8cc25a19f96b5e5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.534336 746 tablet_service.cc:1505] Processing CreateTablet for tablet a5aa442344734b0f98f8053bafae3c50 (DEFAULT_TABLE table=default.Table_bd2d37a87601403eb2a45621e900472b [id=7fbbecd4c36c45a0987b9a5559c452c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.534617 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5aa442344734b0f98f8053bafae3c50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.535861 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70540beb5e924667b61fe432bda26537. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.536278 918 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "528d4269f91a44679acfad16e9fbbe32" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
W20260202 02:52:54.542037 714 leader_election.cc:343] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 026e6ee0b3ae4d938fdf1550044ed1fa
I20260202 02:52:54.543910 899 tablet_service.cc:1505] Processing CreateTablet for tablet ff19d6c3a53a446ea9798c3c160ad7c6 (DEFAULT_TABLE table=default.Table_070200300d5d4886b185f9a99aa77100 [id=3630f9706deb453990898d296903ac1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.544457 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff19d6c3a53a446ea9798c3c160ad7c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.545773 1182 ts_tablet_manager.cc:1929] T 528d4269f91a44679acfad16e9fbbe32 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:54.549855 3041 raft_consensus.cc:493] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:54.553848 1882 ts_tablet_manager.cc:1929] T 6df3d11cf578404ca41807f0777b6c53 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:54.554459 1882 log.cc:1199] T 6df3d11cf578404ca41807f0777b6c53 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/6df3d11cf578404ca41807f0777b6c53
I20260202 02:52:54.554594 1182 log.cc:1199] T 528d4269f91a44679acfad16e9fbbe32 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/528d4269f91a44679acfad16e9fbbe32
I20260202 02:52:54.555086 1182 ts_tablet_manager.cc:1950] T 528d4269f91a44679acfad16e9fbbe32 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:54.555922 1882 ts_tablet_manager.cc:1950] T 6df3d11cf578404ca41807f0777b6c53 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:54.555979 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 911) took 1674 ms. Trace:
I20260202 02:52:54.556116 1182 rpcz_store.cc:276] 0202 02:52:52.881398 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.881443 (+ 45us) service_pool.cc:225] Handling call
0202 02:52:54.395918 (+1514475us) ts_tablet_manager.cc:1151] Deleting tablet 528d4269f91a44679acfad16e9fbbe32
0202 02:52:54.395927 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:54.448145 (+ 52218us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:54.545668 (+ 97523us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:54.555955 (+ 10287us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1514315}
I20260202 02:52:54.556593 1182 tablet_replica.cc:333] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:54.556806 1182 raft_consensus.cc:2243] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:54.557015 1182 raft_consensus.cc:2272] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:54.557365 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 792) took 4867 ms. Trace:
I20260202 02:52:54.557621 1182 ts_tablet_manager.cc:1916] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:54.558251 890 tablet_service.cc:1505] Processing CreateTablet for tablet 9bae6604ddc54fc5bf3423930fa44d9a (DEFAULT_TABLE table=default.Table_8aca64ada6af469b9946c447af43c260 [id=0ef28837669f4d48b1fa4ecf318ddff9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.558740 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bae6604ddc54fc5bf3423930fa44d9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.562187 1882 rpcz_store.cc:276] 0202 02:52:49.690308 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:49.690355 (+ 47us) service_pool.cc:225] Handling call
0202 02:52:54.406866 (+4716511us) ts_tablet_manager.cc:1151] Deleting tablet 6df3d11cf578404ca41807f0777b6c53
0202 02:52:54.406876 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:54.458843 (+ 51967us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:54.553724 (+ 94881us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:54.557345 (+ 3621us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4606172}
I20260202 02:52:54.564410 1182 ts_tablet_manager.cc:1929] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:54.565337 742 tablet_service.cc:1505] Processing CreateTablet for tablet ad244ef168034c3fb8fd86d2e732f1ef (DEFAULT_TABLE table=default.Table_92f45fe4aded4c79816e651a2be7ff27 [id=47aa39ab2d934b34b4251c06977086ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.565758 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad244ef168034c3fb8fd86d2e732f1ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.566056 758 tablet_service.cc:1505] Processing CreateTablet for tablet df5a29f2cd144373a75eb0853410e056 (DEFAULT_TABLE table=default.Table_b64e770cf31548baa0b24b084ea9f939 [id=c5b2ccfa53ac4e32ad22423db2e13116]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.566360 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df5a29f2cd144373a75eb0853410e056. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.566852 740 tablet_service.cc:1505] Processing CreateTablet for tablet f415a9ff427447d8a71829a4878dd7ac (DEFAULT_TABLE table=default.Table_76012bec9c634dbb848a193ad3c6d4e0 [id=88ffd0beda5b4608af047540f269f472]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.567147 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f415a9ff427447d8a71829a4878dd7ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.550068 3041 raft_consensus.cc:515] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.575407 1182 log.cc:1199] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/1d8143f529b74e6ab779dd695920413c
I20260202 02:52:54.575505 3041 leader_election.cc:290] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:54.566902 890 tablet_service.cc:1505] Processing CreateTablet for tablet f26e31992cc544ffa97ba78e8e47537c (DEFAULT_TABLE table=default.Table_8c3c9c0b564041f99e2006f890fc7559 [id=ffb64501dfa04daabe4aa55567608b77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.570982 1175 tablet_bootstrap.cc:492] T 842197c689064660b791b40e521d346b P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:54.572026 900 tablet_service.cc:1505] Processing CreateTablet for tablet 9990414ac8654f9a8cc25a19f96b5e5e (DEFAULT_TABLE table=default.Table_320a9a8a16b949a8a4dfe8fec6a9e5a4 [id=e4f93251a3a24760b93813e187f0f6be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.572557 894 tablet_service.cc:1505] Processing CreateTablet for tablet 31a6358735ab477eb1229be3264b82a6 (DEFAULT_TABLE table=default.Table_20f2fd4d31db4d66be77799ede02027e [id=cfcc4121e4f54862b31b9e643a842b74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.575699 3041 consensus_queue.cc:1048] T e1284fde67bc4d65b067a54c416ccc33 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260202 02:52:54.573066 897 tablet_service.cc:1552] Processing DeleteTablet for tablet 7c07195a46184d24bc709e4967945472 with delete_type TABLET_DATA_DELETED (replaced by tablet 4d37d9e85b7345379e6bbeaa8cabd182) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:54.575908 1182 ts_tablet_manager.cc:1950] T 1d8143f529b74e6ab779dd695920413c P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:54.573566 896 tablet_service.cc:1505] Processing CreateTablet for tablet 19491b92d4b044b9a47e0c72d634660d (DEFAULT_TABLE table=default.Table_78bb96bdb6a1489f94628205c7ccacfd [id=1f18a45584174b10878d792cb3e3ab97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.573798 886 tablet_service.cc:1505] Processing CreateTablet for tablet a5aa442344734b0f98f8053bafae3c50 (DEFAULT_TABLE table=default.Table_bd2d37a87601403eb2a45621e900472b [id=7fbbecd4c36c45a0987b9a5559c452c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.576085 1175 ts_tablet_manager.cc:1403] T 842197c689064660b791b40e521d346b P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.083s user 0.004s sys 0.002s
I20260202 02:52:54.574015 895 tablet_service.cc:1505] Processing CreateTablet for tablet 70540beb5e924667b61fe432bda26537 (DEFAULT_TABLE table=default.Table_4d795c06694e40cf9bf6b637784d3ea4 [id=e8b303fff78c46d58d66701d8a93cae3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.576413 3041 raft_consensus.cc:493] T b4a4c75a906f40beb896bf6fff705ae6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:54.576681 3041 raft_consensus.cc:515] T b4a4c75a906f40beb896bf6fff705ae6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.576882 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5aa442344734b0f98f8053bafae3c50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.577083 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19491b92d4b044b9a47e0c72d634660d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.577288 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70540beb5e924667b61fe432bda26537. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.577389 3041 leader_election.cc:290] T b4a4c75a906f40beb896bf6fff705ae6 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:54.577497 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31a6358735ab477eb1229be3264b82a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.578235 1180 ts_tablet_manager.cc:1929] T 842e54adae2a452a98521de24326eaff P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:54.578330 1180 log.cc:1199] T 842e54adae2a452a98521de24326eaff P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/842e54adae2a452a98521de24326eaff
I20260202 02:52:54.578583 1180 ts_tablet_manager.cc:1950] T 842e54adae2a452a98521de24326eaff P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:54.579327 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 745) took 6480 ms. Trace:
I20260202 02:52:54.579407 1180 rpcz_store.cc:276] 0202 02:52:48.098962 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:48.099033 (+ 71us) service_pool.cc:225] Handling call
0202 02:52:54.379810 (+6280777us) ts_tablet_manager.cc:1151] Deleting tablet 842e54adae2a452a98521de24326eaff
0202 02:52:54.379820 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:54.494302 (+114482us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:54.578127 (+ 83825us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:54.579303 (+ 1176us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6280567}
I20260202 02:52:54.579720 1180 tablet_replica.cc:333] T abfe7f76b2424c4d9ab07b29b66ec103 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:54.579896 1180 raft_consensus.cc:2243] T abfe7f76b2424c4d9ab07b29b66ec103 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:54.580034 1180 raft_consensus.cc:2272] T abfe7f76b2424c4d9ab07b29b66ec103 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:54.580528 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9990414ac8654f9a8cc25a19f96b5e5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.583427 1175 raft_consensus.cc:359] T 842197c689064660b791b40e521d346b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:54.583642 1175 raft_consensus.cc:385] T 842197c689064660b791b40e521d346b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:54.583719 1175 raft_consensus.cc:740] T 842197c689064660b791b40e521d346b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:54.583930 1175 consensus_queue.cc:260] T 842197c689064660b791b40e521d346b P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:54.584628 1175 ts_tablet_manager.cc:1434] T 842197c689064660b791b40e521d346b P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260202 02:52:54.585057 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f26e31992cc544ffa97ba78e8e47537c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.542204 714 leader_election.cc:304] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
W20260202 02:52:54.590768 714 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 48743.4us
I20260202 02:52:54.585440 1175 tablet_bootstrap.cc:492] T 604b3b863c9540d3b55381dbfe1afdc3 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:54.597404 1175 tablet_bootstrap.cc:654] T 604b3b863c9540d3b55381dbfe1afdc3 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:54.592212 3157 raft_consensus.cc:2749] T 026e6ee0b3ae4d938fdf1550044ed1fa P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260202 02:52:54.598141 3043 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s user 0.015s sys 0.023s
W20260202 02:52:54.599319 3043 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.503s user 0.015s sys 0.023s
I20260202 02:52:54.608677 765 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b4a4c75a906f40beb896bf6fff705ae6" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:54.608901 765 raft_consensus.cc:2468] T b4a4c75a906f40beb896bf6fff705ae6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:54.613719 1175 tablet_bootstrap.cc:492] T 604b3b863c9540d3b55381dbfe1afdc3 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:54.613986 1175 ts_tablet_manager.cc:1403] T 604b3b863c9540d3b55381dbfe1afdc3 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.004s
I20260202 02:52:54.614934 1175 raft_consensus.cc:359] T 604b3b863c9540d3b55381dbfe1afdc3 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:54.615238 1175 raft_consensus.cc:385] T 604b3b863c9540d3b55381dbfe1afdc3 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:54.615346 1175 raft_consensus.cc:740] T 604b3b863c9540d3b55381dbfe1afdc3 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:54.615739 1175 consensus_queue.cc:260] T 604b3b863c9540d3b55381dbfe1afdc3 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:54.616297 1175 ts_tablet_manager.cc:1434] T 604b3b863c9540d3b55381dbfe1afdc3 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:54.618047 1175 tablet_bootstrap.cc:492] T c9ef08b5830147a3a55049285a40535b P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:54.622360 1175 tablet_bootstrap.cc:654] T c9ef08b5830147a3a55049285a40535b P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:54.625587 901 tablet_service.cc:1505] Processing CreateTablet for tablet df5a29f2cd144373a75eb0853410e056 (DEFAULT_TABLE table=default.Table_b64e770cf31548baa0b24b084ea9f939 [id=c5b2ccfa53ac4e32ad22423db2e13116]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.629372 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df5a29f2cd144373a75eb0853410e056. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.631671 1174 tablet_bootstrap.cc:654] T f5727f9b9bbb4e77abfef065c51606a1 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:54.626209 894 tablet_service.cc:1505] Processing CreateTablet for tablet 600c8f073d0f465fad7b69adb7dfff4e (DEFAULT_TABLE table=default.Table_768da74ea5aa47e187113ae83e1a5b72 [id=e0863fb79e6f4b378d4f54719b67003e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.637519 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 600c8f073d0f465fad7b69adb7dfff4e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:54.638214 3040 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s user 0.011s sys 0.022s
W20260202 02:52:54.639428 3040 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.544s user 0.011s sys 0.022s
I20260202 02:52:54.638365 3144 raft_consensus.cc:697] T f3eebefa74584c1791d52377f5ba1290 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:54.626505 896 tablet_service.cc:1552] Processing DeleteTablet for tablet 7921daa000f54c8dbed04473e21cf0a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:25 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:54.642977 3144 consensus_queue.cc:237] T f3eebefa74584c1791d52377f5ba1290 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.626718 900 tablet_service.cc:1552] Processing DeleteTablet for tablet 2f664f6119ba4d28b5ef1c212e611866 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:54.627224 897 tablet_service.cc:1505] Processing CreateTablet for tablet b17a32ac48c848c79518917e041172b1 (DEFAULT_TABLE table=default.Table_58d515e7e01b4109b06e1e027ec60e3a [id=51f6afe54fec49b79e14b2d55387ce53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.627194 895 tablet_service.cc:1505] Processing CreateTablet for tablet 6993b8ddf6cb403689d6cc99c860f1d1 (DEFAULT_TABLE table=default.Table_ee948408dac543078c1e7453d5930acc [id=3be73250c4994e279db99053c2185ca5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.627628 898 tablet_service.cc:1505] Processing CreateTablet for tablet 13f53365036d4c57b520f8e282442a20 (DEFAULT_TABLE table=default.Table_0dd06fff80bc41c78d6f707fc8c0bee3 [id=2be6211dbde24f8db41e4fc1d02673fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.627740 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 3dcf4f794ec14b2f9d181d015fc437fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:30 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:54.628158 884 tablet_service.cc:1505] Processing CreateTablet for tablet 4735b8c368244bc8956c74a5bdef6658 (DEFAULT_TABLE table=default.Table_f479891b2cc74fc68dbdbd2a63e870ee [id=3f54b77e4da54574b6f707051d60c6ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.643656 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b17a32ac48c848c79518917e041172b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.643872 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6993b8ddf6cb403689d6cc99c860f1d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.628259 892 tablet_service.cc:1505] Processing CreateTablet for tablet adf26e4901dc4ddda6de50a87855cc9d (DEFAULT_TABLE table=default.Table_a64a2e7c03c74179a133b3bcd1da1d27 [id=b355b1e7bfcd4164800f8da86d341cf9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.644026 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4735b8c368244bc8956c74a5bdef6658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.644194 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13f53365036d4c57b520f8e282442a20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.628448 902 tablet_service.cc:1505] Processing CreateTablet for tablet 902883ef12dc4c82b051634fb04feb19 (DEFAULT_TABLE table=default.Table_ab37f999ca1f4033ac2a25bdf3f8bbf6 [id=2fdf8e379824433196848d0ab4330877]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.628692 885 tablet_service.cc:1505] Processing CreateTablet for tablet e670b8b95a60482896450fb231bd4169 (DEFAULT_TABLE table=default.Table_32235d7fa05f42efb99094f7b8d3c93b [id=932164e1140849f193982a139df92b19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.644379 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adf26e4901dc4ddda6de50a87855cc9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.644531 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e670b8b95a60482896450fb231bd4169. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.645364 765 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "abfe7f76b2424c4d9ab07b29b66ec103" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:54.648146 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 902883ef12dc4c82b051634fb04feb19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.666990 1180 ts_tablet_manager.cc:1916] T abfe7f76b2424c4d9ab07b29b66ec103 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:54.564406 1042 tablet_service.cc:1505] Processing CreateTablet for tablet 6993b8ddf6cb403689d6cc99c860f1d1 (DEFAULT_TABLE table=default.Table_ee948408dac543078c1e7453d5930acc [id=3be73250c4994e279db99053c2185ca5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.668859 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6993b8ddf6cb403689d6cc99c860f1d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.671774 1175 tablet_bootstrap.cc:492] T c9ef08b5830147a3a55049285a40535b P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:54.671916 1175 ts_tablet_manager.cc:1403] T c9ef08b5830147a3a55049285a40535b P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.054s user 0.004s sys 0.000s
I20260202 02:52:54.565220 998 leader_election.cc:304] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
W20260202 02:52:54.672498 998 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 107303us
I20260202 02:52:54.672892 1175 raft_consensus.cc:359] T c9ef08b5830147a3a55049285a40535b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:54.673058 1175 raft_consensus.cc:385] T c9ef08b5830147a3a55049285a40535b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:54.673101 1175 raft_consensus.cc:740] T c9ef08b5830147a3a55049285a40535b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:54.673097 998 leader_election.cc:304] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5
I20260202 02:52:54.673336 1175 consensus_queue.cc:260] T c9ef08b5830147a3a55049285a40535b P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
W20260202 02:52:54.673650 998 leader_election.cc:343] T abfe7f76b2424c4d9ab07b29b66ec103 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:54.566094 1027 tablet_service.cc:1505] Processing CreateTablet for tablet b17a32ac48c848c79518917e041172b1 (DEFAULT_TABLE table=default.Table_58d515e7e01b4109b06e1e027ec60e3a [id=51f6afe54fec49b79e14b2d55387ce53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.674324 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b17a32ac48c848c79518917e041172b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.674741 892 tablet_service.cc:1505] Processing CreateTablet for tablet 82efe2179fe54c01bf41630fca03802a (DEFAULT_TABLE table=default.Table_c8d5b0bad2904df78c01b87bbd5b8e7f [id=ff99fedab8cb41dc8b94941890c8f923]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.675180 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82efe2179fe54c01bf41630fca03802a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.676048 885 tablet_service.cc:1505] Processing CreateTablet for tablet 98ee9e01f0354a2abdbc1c45f1e6acdf (DEFAULT_TABLE table=default.Table_f61ca1b0d7424868993ccd4657d50429 [id=3c38914fb1644aa79f39fb66939ebc92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.676440 3032 raft_consensus.cc:2804] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:54.676517 3032 raft_consensus.cc:493] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:54.676571 3032 raft_consensus.cc:3060] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:54.676568 902 tablet_service.cc:1505] Processing CreateTablet for tablet 25a54991224a4c8da9e0e278a054e7b0 (DEFAULT_TABLE table=default.Table_7f1d13e7a79e4a86ad883eb6e5d214e0 [id=2c2446c1766b4e4c9d5eead5cd9b9290]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.677140 884 tablet_service.cc:1505] Processing CreateTablet for tablet 3590bfe57ac346b48e059a8c2deaafd9 (DEFAULT_TABLE table=default.Table_2d519304c313459a8b38c0fe0c65d21f [id=022bce98e93d4821a275d6a26dd033aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.677470 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98ee9e01f0354a2abdbc1c45f1e6acdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.678996 3032 raft_consensus.cc:515] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.680495 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3590bfe57ac346b48e059a8c2deaafd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.680712 899 tablet_service.cc:1505] Processing CreateTablet for tablet ad244ef168034c3fb8fd86d2e732f1ef (DEFAULT_TABLE table=default.Table_92f45fe4aded4c79816e651a2be7ff27 [id=47aa39ab2d934b34b4251c06977086ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.681056 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad244ef168034c3fb8fd86d2e732f1ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.681795 2979 raft_consensus.cc:493] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:54.681934 2979 raft_consensus.cc:515] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.682523 2979 leader_election.cc:290] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:54.682668 2979 raft_consensus.cc:2804] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:54.682722 2979 raft_consensus.cc:493] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:54.682786 2979 raft_consensus.cc:3060] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:54.684464 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25a54991224a4c8da9e0e278a054e7b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.684837 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 6df3d11cf578404ca41807f0777b6c53 (table default.Table_d1815e87de114d408122224276266f69 [id=4a43e638e8854fcda2702463684c08d7]) successfully deleted
I20260202 02:52:54.569232 1040 tablet_service.cc:1505] Processing CreateTablet for tablet 13f53365036d4c57b520f8e282442a20 (DEFAULT_TABLE table=default.Table_0dd06fff80bc41c78d6f707fc8c0bee3 [id=2be6211dbde24f8db41e4fc1d02673fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.685287 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13f53365036d4c57b520f8e282442a20. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:54.685719 715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.131:35257, user_credentials={real_user=slave}} blocked reactor thread for 335552us
I20260202 02:52:54.686079 895 tablet_service.cc:1505] Processing CreateTablet for tablet b4dddc2235ab4c9d898f5e7f3a2ac983 (DEFAULT_TABLE table=default.Table_74a1a2c121f1430d87ced4156ee01e53 [id=8a048083482c400b9dada08379e26471]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.686414 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4dddc2235ab4c9d898f5e7f3a2ac983. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.686468 1054 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "528d4269f91a44679acfad16e9fbbe32" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:54.686654 1054 raft_consensus.cc:3060] T 528d4269f91a44679acfad16e9fbbe32 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:54.569867 2977 raft_consensus.cc:493] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:54.688426 2977 raft_consensus.cc:515] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.688980 2977 leader_election.cc:290] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:54.573447 1039 tablet_service.cc:1505] Processing CreateTablet for tablet 4735b8c368244bc8956c74a5bdef6658 (DEFAULT_TABLE table=default.Table_f479891b2cc74fc68dbdbd2a63e870ee [id=3f54b77e4da54574b6f707051d60c6ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.689116 1054 raft_consensus.cc:2468] T 528d4269f91a44679acfad16e9fbbe32 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:54.689348 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4735b8c368244bc8956c74a5bdef6658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.587193 1031 tablet_service.cc:1505] Processing CreateTablet for tablet 902883ef12dc4c82b051634fb04feb19 (DEFAULT_TABLE table=default.Table_ab37f999ca1f4033ac2a25bdf3f8bbf6 [id=2fdf8e379824433196848d0ab4330877]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.692629 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 902883ef12dc4c82b051634fb04feb19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.692538 895 tablet_service.cc:1505] Processing CreateTablet for tablet ca5e638942e14b829b80d99a5884305c (DEFAULT_TABLE table=default.Table_8161c74bbc974802bd0baade2f1fb103 [id=adae2af9c45e4cda8f87cb6fcf9001ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.692894 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca5e638942e14b829b80d99a5884305c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.589033 1038 tablet_service.cc:1505] Processing CreateTablet for tablet adf26e4901dc4ddda6de50a87855cc9d (DEFAULT_TABLE table=default.Table_a64a2e7c03c74179a133b3bcd1da1d27 [id=b355b1e7bfcd4164800f8da86d341cf9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.696677 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adf26e4901dc4ddda6de50a87855cc9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.696936 1174 tablet_bootstrap.cc:492] T f5727f9b9bbb4e77abfef065c51606a1 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:54.697067 1174 ts_tablet_manager.cc:1403] T f5727f9b9bbb4e77abfef065c51606a1 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.178s user 0.003s sys 0.008s
I20260202 02:52:54.696995 897 tablet_service.cc:1505] Processing CreateTablet for tablet 32ee0da311f1453d8a2d30f9d8be6702 (DEFAULT_TABLE table=default.Table_1fa8930369e844ca8384126bf8f22e66 [id=116247212fca4c0a95ad1bdbbac4787f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.697331 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32ee0da311f1453d8a2d30f9d8be6702. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.697849 1174 raft_consensus.cc:359] T f5727f9b9bbb4e77abfef065c51606a1 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:54.698009 1174 raft_consensus.cc:385] T f5727f9b9bbb4e77abfef065c51606a1 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:54.698053 1174 raft_consensus.cc:740] T f5727f9b9bbb4e77abfef065c51606a1 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:54.591183 1035 tablet_service.cc:1505] Processing CreateTablet for tablet e670b8b95a60482896450fb231bd4169 (DEFAULT_TABLE table=default.Table_32235d7fa05f42efb99094f7b8d3c93b [id=932164e1140849f193982a139df92b19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.700655 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e670b8b95a60482896450fb231bd4169. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.703953 1054 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f1c8f7e1dc9644a6b6dd2bc264cce8ef" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:54.704156 1054 raft_consensus.cc:2468] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:54.591995 1173 tablet_bootstrap.cc:492] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:54.704752 1173 ts_tablet_manager.cc:1403] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.361s user 0.024s sys 0.000s
I20260202 02:52:54.704885 1054 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b4a4c75a906f40beb896bf6fff705ae6" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:54.705407 1173 raft_consensus.cc:359] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:54.705552 1173 raft_consensus.cc:385] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:54.705595 1173 raft_consensus.cc:740] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:54.705766 1173 consensus_queue.cc:260] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:54.705792 902 tablet_service.cc:1505] Processing CreateTablet for tablet cc7cb7a8cadc4c90a2449f9d7183aa9a (DEFAULT_TABLE table=default.Table_6831562d21db463986dc58b0100e9792 [id=50e58db611f247ee9dcd5b31776ace05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.705864 3160 raft_consensus.cc:493] T 07580ad1d049475a9645acfad0b1d219 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:54.705978 3160 raft_consensus.cc:515] T 07580ad1d049475a9645acfad0b1d219 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.706177 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc7cb7a8cadc4c90a2449f9d7183aa9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.706233 1173 ts_tablet_manager.cc:1434] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260202 02:52:54.706503 3160 leader_election.cc:290] T 07580ad1d049475a9645acfad0b1d219 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:54.706559 1173 tablet_bootstrap.cc:492] T c17c85759e964f0ba3abcaf5fe938fba P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:54.708073 1173 tablet_bootstrap.cc:654] T c17c85759e964f0ba3abcaf5fe938fba P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:54.600381 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 8ee843650a344e99a1c9d46b1c523ac9 (DEFAULT_TABLE table=default.Table_bc3f75a4a12c4f92a4bbf19395b7800a [id=4ab3c9611a2c4d1280ae18acb2507c90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.605517 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 98ee9e01f0354a2abdbc1c45f1e6acdf (DEFAULT_TABLE table=default.Table_f61ca1b0d7424868993ccd4657d50429 [id=3c38914fb1644aa79f39fb66939ebc92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.708674 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ee843650a344e99a1c9d46b1c523ac9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.708873 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98ee9e01f0354a2abdbc1c45f1e6acdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.610010 1032 tablet_service.cc:1505] Processing CreateTablet for tablet 25a54991224a4c8da9e0e278a054e7b0 (DEFAULT_TABLE table=default.Table_7f1d13e7a79e4a86ad883eb6e5d214e0 [id=2c2446c1766b4e4c9d5eead5cd9b9290]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.617445 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 3590bfe57ac346b48e059a8c2deaafd9 (DEFAULT_TABLE table=default.Table_2d519304c313459a8b38c0fe0c65d21f [id=022bce98e93d4821a275d6a26dd033aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.709542 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25a54991224a4c8da9e0e278a054e7b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.710723 1175 ts_tablet_manager.cc:1434] T c9ef08b5830147a3a55049285a40535b P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.039s user 0.001s sys 0.001s
I20260202 02:52:54.711016 1175 tablet_bootstrap.cc:492] T a81e1d10ca49459989260fbd6a2b8645 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:54.710963 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "09504070c9ec4bcfbb59e3fb4c189775" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:54.711156 3032 leader_election.cc:290] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:54.711400 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bbcc723a8079443db455ae20fb0d585d" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:54.625084 1043 tablet_service.cc:1505] Processing CreateTablet for tablet 82efe2179fe54c01bf41630fca03802a (DEFAULT_TABLE table=default.Table_c8d5b0bad2904df78c01b87bbd5b8e7f [id=ff99fedab8cb41dc8b94941890c8f923]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.712177 765 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3dcf4f794ec14b2f9d181d015fc437fb" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:54.712791 1175 tablet_bootstrap.cc:654] T a81e1d10ca49459989260fbd6a2b8645 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:54.714359 715 leader_election.cc:304] T 528d4269f91a44679acfad16e9fbbe32 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:54.714780 3144 raft_consensus.cc:2804] T 528d4269f91a44679acfad16e9fbbe32 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:54.714883 3144 raft_consensus.cc:697] T 528d4269f91a44679acfad16e9fbbe32 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:54.656561 1028 tablet_service.cc:1505] Processing CreateTablet for tablet b4dddc2235ab4c9d898f5e7f3a2ac983 (DEFAULT_TABLE table=default.Table_74a1a2c121f1430d87ced4156ee01e53 [id=8a048083482c400b9dada08379e26471]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.658236 1030 tablet_service.cc:1505] Processing CreateTablet for tablet ca5e638942e14b829b80d99a5884305c (DEFAULT_TABLE table=default.Table_8161c74bbc974802bd0baade2f1fb103 [id=adae2af9c45e4cda8f87cb6fcf9001ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.660271 1036 tablet_service.cc:1505] Processing CreateTablet for tablet ec5f5e47588d46da98d4b7e340b5138a (DEFAULT_TABLE table=default.Table_42bf0269c9e64c62adfa02c34d69459e [id=ac38ffe9c0a144a98b1f723643011f71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.664805 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 32ee0da311f1453d8a2d30f9d8be6702 (DEFAULT_TABLE table=default.Table_1fa8930369e844ca8384126bf8f22e66 [id=116247212fca4c0a95ad1bdbbac4787f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.666608 1034 tablet_service.cc:1505] Processing CreateTablet for tablet cc7cb7a8cadc4c90a2449f9d7183aa9a (DEFAULT_TABLE table=default.Table_6831562d21db463986dc58b0100e9792 [id=50e58db611f247ee9dcd5b31776ace05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.715097 3144 consensus_queue.cc:237] T 528d4269f91a44679acfad16e9fbbe32 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.715876 890 tablet_service.cc:1505] Processing CreateTablet for tablet 8ee843650a344e99a1c9d46b1c523ac9 (DEFAULT_TABLE table=default.Table_bc3f75a4a12c4f92a4bbf19395b7800a [id=4ab3c9611a2c4d1280ae18acb2507c90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.716499 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ee843650a344e99a1c9d46b1c523ac9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.716543 898 tablet_service.cc:1505] Processing CreateTablet for tablet ec5f5e47588d46da98d4b7e340b5138a (DEFAULT_TABLE table=default.Table_42bf0269c9e64c62adfa02c34d69459e [id=ac38ffe9c0a144a98b1f723643011f71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.716838 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec5f5e47588d46da98d4b7e340b5138a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.716941 894 tablet_service.cc:1552] Processing DeleteTablet for tablet 842197c689064660b791b40e521d346b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:28 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:54.718389 1174 consensus_queue.cc:260] T f5727f9b9bbb4e77abfef065c51606a1 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:54.718981 1174 ts_tablet_manager.cc:1434] T f5727f9b9bbb4e77abfef065c51606a1 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.022s user 0.000s sys 0.002s
I20260202 02:52:54.725596 2977 raft_consensus.cc:493] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:54.725737 2977 raft_consensus.cc:515] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:54.726264 2977 leader_election.cc:290] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:54.730481 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "07580ad1d049475a9645acfad0b1d219" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:54.730681 1060 raft_consensus.cc:2468] T 07580ad1d049475a9645acfad0b1d219 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 0.
I20260202 02:52:54.733258 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3590bfe57ac346b48e059a8c2deaafd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.739953 1174 tablet_bootstrap.cc:492] T c17c85759e964f0ba3abcaf5fe938fba P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:54.741011 1175 tablet_bootstrap.cc:492] T a81e1d10ca49459989260fbd6a2b8645 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:54.741240 1175 ts_tablet_manager.cc:1403] T a81e1d10ca49459989260fbd6a2b8645 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.030s user 0.004s sys 0.000s
W20260202 02:52:54.731757 715 consensus_peers.cc:597] T 474df41e4fe24a818fa45a4ce84d5474 P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 474df41e4fe24a818fa45a4ce84d5474. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:54.741737 715 leader_election.cc:304] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:54.742092 3160 raft_consensus.cc:2804] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:54.742009 1175 raft_consensus.cc:359] T a81e1d10ca49459989260fbd6a2b8645 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:54.742180 1175 raft_consensus.cc:385] T a81e1d10ca49459989260fbd6a2b8645 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:54.742225 1175 raft_consensus.cc:740] T a81e1d10ca49459989260fbd6a2b8645 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:54.742336 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82efe2179fe54c01bf41630fca03802a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.742413 1175 consensus_queue.cc:260] T a81e1d10ca49459989260fbd6a2b8645 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:54.742570 3160 raft_consensus.cc:493] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:54.742688 3160 raft_consensus.cc:3060] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:54.745273 3160 raft_consensus.cc:515] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.746098 3160 leader_election.cc:290] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:54.746867 715 leader_election.cc:304] T 07580ad1d049475a9645acfad0b1d219 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:54.747056 2979 raft_consensus.cc:515] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.747566 1054 raft_consensus.cc:2393] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f442ef85029461091de847118c34c0d in current term 1: Already voted for candidate b1f05820b6034228bdad884cc348716d in this term.
I20260202 02:52:54.748117 1175 ts_tablet_manager.cc:1434] T a81e1d10ca49459989260fbd6a2b8645 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20260202 02:52:54.748435 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4dddc2235ab4c9d898f5e7f3a2ac983. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.748870 2944 raft_consensus.cc:2804] T 07580ad1d049475a9645acfad0b1d219 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:54.748944 2944 raft_consensus.cc:493] T 07580ad1d049475a9645acfad0b1d219 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:54.749001 2944 raft_consensus.cc:3060] T 07580ad1d049475a9645acfad0b1d219 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:54.750914 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca5e638942e14b829b80d99a5884305c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.751626 1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec5f5e47588d46da98d4b7e340b5138a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.751787 2944 raft_consensus.cc:515] T 07580ad1d049475a9645acfad0b1d219 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.752243 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32ee0da311f1453d8a2d30f9d8be6702. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.752437 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc7cb7a8cadc4c90a2449f9d7183aa9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.752869 2944 leader_election.cc:290] T 07580ad1d049475a9645acfad0b1d219 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Requested vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:54.752904 1175 tablet_bootstrap.cc:492] T f6c2bc531d194a4b8f3062c9b40e3da6 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:54.754989 1060 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "07580ad1d049475a9645acfad0b1d219" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:54.755168 1060 raft_consensus.cc:3060] T 07580ad1d049475a9645acfad0b1d219 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:54.756556 1174 tablet_bootstrap.cc:654] T c17c85759e964f0ba3abcaf5fe938fba P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:54.759438 1059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f1c8f7e1dc9644a6b6dd2bc264cce8ef" candidate_uuid: "1f442ef85029461091de847118c34c0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d"
I20260202 02:52:54.759668 1059 raft_consensus.cc:3060] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:54.761241 1175 tablet_bootstrap.cc:654] T f6c2bc531d194a4b8f3062c9b40e3da6 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:54.764299 2979 leader_election.cc:290] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:54.761394 1180 ts_tablet_manager.cc:1929] T abfe7f76b2424c4d9ab07b29b66ec103 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:54.769258 1180 log.cc:1199] T abfe7f76b2424c4d9ab07b29b66ec103 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/abfe7f76b2424c4d9ab07b29b66ec103
I20260202 02:52:54.769717 1180 ts_tablet_manager.cc:1950] T abfe7f76b2424c4d9ab07b29b66ec103 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:54.770644 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 746) took 6671 ms. Trace:
I20260202 02:52:54.770833 1180 rpcz_store.cc:276] 0202 02:52:48.099067 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:48.099216 (+ 149us) service_pool.cc:225] Handling call
0202 02:52:54.579663 (+6480447us) ts_tablet_manager.cc:1151] Deleting tablet abfe7f76b2424c4d9ab07b29b66ec103
0202 02:52:54.579673 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:54.684072 (+104399us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:54.761278 (+ 77206us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:54.770624 (+ 9346us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6480267}
I20260202 02:52:54.771392 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 747) took 6672 ms. Trace:
I20260202 02:52:54.771565 1180 rpcz_store.cc:276] 0202 02:52:48.099160 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:48.099282 (+ 122us) service_pool.cc:225] Handling call
0202 02:52:54.771257 (+6671975us) ts_tablet_manager.cc:1151] Deleting tablet dd8f36d7a82f47adae1bd3c1f8628b6e
0202 02:52:54.771268 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:54.771377 (+ 109us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6671739}
I20260202 02:52:54.771935 1180 tablet_replica.cc:333] T 02b434fa058b45529ff3938b053263c6 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:54.772156 1180 raft_consensus.cc:2243] T 02b434fa058b45529ff3938b053263c6 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:54.772403 1180 raft_consensus.cc:2272] T 02b434fa058b45529ff3938b053263c6 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:54.773084 1180 ts_tablet_manager.cc:1916] T 02b434fa058b45529ff3938b053263c6 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:54.778542 1060 raft_consensus.cc:2468] T 07580ad1d049475a9645acfad0b1d219 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:54.779268 715 leader_election.cc:304] T 07580ad1d049475a9645acfad0b1d219 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:54.780069 3161 raft_consensus.cc:493] T 26b890ed87c3471b9fc0b172c90e0a11 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1f05820b6034228bdad884cc348716d)
I20260202 02:52:54.780269 3161 raft_consensus.cc:515] T 26b890ed87c3471b9fc0b172c90e0a11 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.781037 3161 leader_election.cc:290] T 26b890ed87c3471b9fc0b172c90e0a11 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b1f05820b6034228bdad884cc348716d (127.0.104.131:35257), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:54.781252 3161 raft_consensus.cc:2804] T 07580ad1d049475a9645acfad0b1d219 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:54.781486 3161 raft_consensus.cc:697] T 07580ad1d049475a9645acfad0b1d219 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:54.781831 3161 consensus_queue.cc:237] T 07580ad1d049475a9645acfad0b1d219 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.793865 765 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "93f958924885457ab15f2aac5509e0aa" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:54.794142 765 raft_consensus.cc:2468] T 93f958924885457ab15f2aac5509e0aa P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:54.794414 778 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f5727f9b9bbb4e77abfef065c51606a1" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:54.794554 778 raft_consensus.cc:2468] T f5727f9b9bbb4e77abfef065c51606a1 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:54.794698 764 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b4a4c75a906f40beb896bf6fff705ae6" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:54.794875 764 raft_consensus.cc:3060] T b4a4c75a906f40beb896bf6fff705ae6 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:54.795164 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7ff6d2fa085343a6b11be0898da6233d" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:54.795307 775 raft_consensus.cc:2468] T 7ff6d2fa085343a6b11be0898da6233d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:54.795509 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6a3dac5cd1ce47488a50b4c32eca67c7" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:54.795617 766 raft_consensus.cc:3060] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:54.797430 764 raft_consensus.cc:2468] T b4a4c75a906f40beb896bf6fff705ae6 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:54.798357 766 raft_consensus.cc:2468] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:54.799254 998 leader_election.cc:304] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:54.800637 714 leader_election.cc:304] T 5e20842de8cb4355b732685605c88834 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d; no voters: b1f05820b6034228bdad884cc348716d
I20260202 02:52:54.801965 998 leader_election.cc:304] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:54.802444 998 leader_election.cc:304] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:54.802762 2794 raft_consensus.cc:2804] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:54.802832 2794 raft_consensus.cc:493] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:54.802887 2794 raft_consensus.cc:3060] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:54.805420 2794 raft_consensus.cc:515] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.806052 2794 leader_election.cc:290] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:54.806788 2794 raft_consensus.cc:2804] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:54.807282 2794 raft_consensus.cc:493] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:54.807559 2794 raft_consensus.cc:3060] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:54.808686 2977 raft_consensus.cc:2804] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:54.808782 2977 raft_consensus.cc:493] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:54.808848 2977 raft_consensus.cc:3060] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:54.810650 998 leader_election.cc:304] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:54.811216 1035 tablet_service.cc:1505] Processing CreateTablet for tablet 4381644c4d874bd7b1d62537edb0bbf1 (DEFAULT_TABLE table=default.Table_b5055a016bd5483980012ad46c8c68c9 [id=4203187b5306439fba0774f9d0cd4bfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.813126 1175 tablet_bootstrap.cc:492] T f6c2bc531d194a4b8f3062c9b40e3da6 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:54.813346 1175 ts_tablet_manager.cc:1403] T f6c2bc531d194a4b8f3062c9b40e3da6 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.061s user 0.020s sys 0.017s
I20260202 02:52:54.814126 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4381644c4d874bd7b1d62537edb0bbf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.814361 1882 tablet_replica.cc:333] T 3284213e4b0447b49a7718ba896d3fa8 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:54.814291 1175 raft_consensus.cc:359] T f6c2bc531d194a4b8f3062c9b40e3da6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:54.814458 1175 raft_consensus.cc:385] T f6c2bc531d194a4b8f3062c9b40e3da6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:54.814524 1175 raft_consensus.cc:740] T f6c2bc531d194a4b8f3062c9b40e3da6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:54.814761 1175 consensus_queue.cc:260] T f6c2bc531d194a4b8f3062c9b40e3da6 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:54.815073 1882 raft_consensus.cc:2243] T 3284213e4b0447b49a7718ba896d3fa8 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:54.802855 1174 tablet_bootstrap.cc:492] T c17c85759e964f0ba3abcaf5fe938fba P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:54.818791 1174 ts_tablet_manager.cc:1403] T c17c85759e964f0ba3abcaf5fe938fba P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.079s user 0.003s sys 0.001s
I20260202 02:52:54.819691 1174 raft_consensus.cc:359] T c17c85759e964f0ba3abcaf5fe938fba P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.819970 1174 raft_consensus.cc:385] T c17c85759e964f0ba3abcaf5fe938fba P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:54.820067 1174 raft_consensus.cc:740] T c17c85759e964f0ba3abcaf5fe938fba P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:54.820287 1175 ts_tablet_manager.cc:1434] T f6c2bc531d194a4b8f3062c9b40e3da6 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260202 02:52:54.820312 1174 consensus_queue.cc:260] T c17c85759e964f0ba3abcaf5fe938fba P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.820775 998 leader_election.cc:304] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:54.820955 1174 ts_tablet_manager.cc:1434] T c17c85759e964f0ba3abcaf5fe938fba P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260202 02:52:54.821213 2979 raft_consensus.cc:2804] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:54.821437 1174 tablet_bootstrap.cc:492] T 7f323175ee254df789bcb04e65047dff P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:54.821573 3032 consensus_queue.cc:1048] T b427c7e897c64977b924e7d586420e69 P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260202 02:52:54.812713 715 leader_election.cc:343] T 26b890ed87c3471b9fc0b172c90e0a11 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Not found: Tablet not found: 26b890ed87c3471b9fc0b172c90e0a11
I20260202 02:52:54.821982 3032 raft_consensus.cc:2804] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:54.822090 3032 raft_consensus.cc:697] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:54.822321 3032 consensus_queue.cc:237] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.823588 1174 tablet_bootstrap.cc:654] T 7f323175ee254df789bcb04e65047dff P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:54.826433 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 2740246e8fad411abb09db0fd00beda0 (DEFAULT_TABLE table=default.Table_b6f0e40295b247c493d61b5369e218f3 [id=0efa2fe908a443f6b44ccb1bc13da814]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.826879 1175 tablet_bootstrap.cc:492] T dc4c3f52f4f44a84b0d0e3c9a4a49f09 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:54.826972 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2740246e8fad411abb09db0fd00beda0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:54.827083 999 leader_election.cc:343] T abfe7f76b2424c4d9ab07b29b66ec103 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: abfe7f76b2424c4d9ab07b29b66ec103
I20260202 02:52:54.827185 999 leader_election.cc:304] T abfe7f76b2424c4d9ab07b29b66ec103 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:54.827884 1059 raft_consensus.cc:2468] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f442ef85029461091de847118c34c0d in term 1.
I20260202 02:52:54.831332 715 leader_election.cc:304] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:54.832582 1175 tablet_bootstrap.cc:654] T dc4c3f52f4f44a84b0d0e3c9a4a49f09 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:54.834196 3161 raft_consensus.cc:2804] T 5e20842de8cb4355b732685605c88834 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:54.834343 3161 raft_consensus.cc:697] T 5e20842de8cb4355b732685605c88834 P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:54.834585 3161 consensus_queue.cc:237] T 5e20842de8cb4355b732685605c88834 P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:54.835850 2944 raft_consensus.cc:2804] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:54.836102 2944 raft_consensus.cc:697] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1f442ef85029461091de847118c34c0d [term 1 LEADER]: Becoming Leader. State: Replica: 1f442ef85029461091de847118c34c0d, State: Running, Role: LEADER
I20260202 02:52:54.836411 2944 consensus_queue.cc:237] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1f442ef85029461091de847118c34c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.837652 2794 raft_consensus.cc:515] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:54.839128 899 tablet_service.cc:1552] Processing DeleteTablet for tablet c9ef08b5830147a3a55049285a40535b with delete_type TABLET_DATA_DELETED (replaced by tablet 5d2fc0a74684434d85b9922cb4e32fdc) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:54.839322 890 tablet_service.cc:1505] Processing CreateTablet for tablet 4381644c4d874bd7b1d62537edb0bbf1 (DEFAULT_TABLE table=default.Table_b5055a016bd5483980012ad46c8c68c9 [id=4203187b5306439fba0774f9d0cd4bfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.839744 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4381644c4d874bd7b1d62537edb0bbf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.848279 898 tablet_service.cc:1505] Processing CreateTablet for tablet 2740246e8fad411abb09db0fd00beda0 (DEFAULT_TABLE table=default.Table_b6f0e40295b247c493d61b5369e218f3 [id=0efa2fe908a443f6b44ccb1bc13da814]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.848706 2794 leader_election.cc:290] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:54.848889 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2740246e8fad411abb09db0fd00beda0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.849238 2794 raft_consensus.cc:2749] T abfe7f76b2424c4d9ab07b29b66ec103 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:54.852682 2977 raft_consensus.cc:515] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
W20260202 02:52:54.852967 714 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 52351.5us
I20260202 02:52:54.854010 2977 leader_election.cc:290] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:54.856102 714 leader_election.cc:343] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 6a3dac5cd1ce47488a50b4c32eca67c7
I20260202 02:52:54.856429 714 leader_election.cc:304] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:54.857345 898 tablet_service.cc:1552] Processing DeleteTablet for tablet a81e1d10ca49459989260fbd6a2b8645 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:54.857697 857 leader_election.cc:343] T 7f9dfac19c04475aadb45e95a6474eb4 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:54.858160 857 leader_election.cc:304] T 7f9dfac19c04475aadb45e95a6474eb4 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:54.858561 3080 raft_consensus.cc:2749] T 7f9dfac19c04475aadb45e95a6474eb4 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:54.867739 898 tablet_service.cc:1552] Processing DeleteTablet for tablet f6c2bc531d194a4b8f3062c9b40e3da6 with delete_type TABLET_DATA_DELETED (replaced by tablet 29cde4226db6428987d4a39cde804a5c) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:54.868700 3160 raft_consensus.cc:2749] T 6a3dac5cd1ce47488a50b4c32eca67c7 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:54.859184 1174 tablet_bootstrap.cc:492] T 7f323175ee254df789bcb04e65047dff P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
I20260202 02:52:54.869351 1174 ts_tablet_manager.cc:1403] T 7f323175ee254df789bcb04e65047dff P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.048s user 0.004s sys 0.002s
I20260202 02:52:54.870321 1174 raft_consensus.cc:359] T 7f323175ee254df789bcb04e65047dff P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.870610 1174 raft_consensus.cc:385] T 7f323175ee254df789bcb04e65047dff P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:54.870709 1174 raft_consensus.cc:740] T 7f323175ee254df789bcb04e65047dff P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:54.870999 1174 consensus_queue.cc:260] T 7f323175ee254df789bcb04e65047dff P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.871959 1174 ts_tablet_manager.cc:1434] T 7f323175ee254df789bcb04e65047dff P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:54.875257 1174 tablet_bootstrap.cc:492] T 7f9dfac19c04475aadb45e95a6474eb4 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:54.877734 1174 tablet_bootstrap.cc:654] T 7f9dfac19c04475aadb45e95a6474eb4 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:54.884012 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 912) took 2002 ms. Trace:
I20260202 02:52:54.874607 1180 ts_tablet_manager.cc:1929] T 02b434fa058b45529ff3938b053263c6 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260202 02:52:54.872550 714 leader_election.cc:343] T f3eebefa74584c1791d52377f5ba1290 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: f3eebefa74584c1791d52377f5ba1290
W20260202 02:52:54.887871 714 consensus_peers.cc:597] T 0138ba7326674741b0729324735f1de7 P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0138ba7326674741b0729324735f1de7. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:54.888448 714 leader_election.cc:343] T 99e0b5e9a9874aec886155e0d614dec2 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 99e0b5e9a9874aec886155e0d614dec2
I20260202 02:52:54.889019 1182 rpcz_store.cc:276] 0202 02:52:52.881498 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.881591 (+ 93us) service_pool.cc:225] Handling call
0202 02:52:54.556505 (+1674914us) ts_tablet_manager.cc:1151] Deleting tablet 1d8143f529b74e6ab779dd695920413c
0202 02:52:54.556530 (+ 25us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:54.561107 (+ 4577us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:54.564245 (+ 3138us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:54.883983 (+319738us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1674786}
W20260202 02:52:54.889389 714 leader_election.cc:343] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: ee04563d3b8e4cb2b989bb8d6ee9b20e
I20260202 02:52:54.889567 714 leader_election.cc:304] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:54.889838 3160 raft_consensus.cc:2749] T ee04563d3b8e4cb2b989bb8d6ee9b20e P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:54.889951 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 1d8143f529b74e6ab779dd695920413c (table default.Table_33fc32ec03094e3c8fe78de4f6a26777 [id=85340c1716bd4235bdad7042b53ce58c]) successfully deleted
I20260202 02:52:54.886778 1180 log.cc:1199] T 02b434fa058b45529ff3938b053263c6 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/02b434fa058b45529ff3938b053263c6
I20260202 02:52:54.890372 1180 ts_tablet_manager.cc:1950] T 02b434fa058b45529ff3938b053263c6 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:54.890658 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 913) took 1969 ms. Trace:
I20260202 02:52:54.891217 1182 rpcz_store.cc:276] 0202 02:52:52.921257 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.921431 (+ 174us) service_pool.cc:225] Handling call
0202 02:52:54.890496 (+1969065us) ts_tablet_manager.cc:1151] Deleting tablet 6f2739c054ce4e6b87634eced940d548
0202 02:52:54.890512 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:54.890639 (+ 127us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1968838}
W20260202 02:52:54.891040 714 leader_election.cc:343] T 528d4269f91a44679acfad16e9fbbe32 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Illegal state: Tablet not RUNNING: STOPPED
W20260202 02:52:54.891873 714 leader_election.cc:343] T 528d4269f91a44679acfad16e9fbbe32 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 528d4269f91a44679acfad16e9fbbe32
W20260202 02:52:54.891974 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 6f2739c054ce4e6b87634eced940d548 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6f2739c054ce4e6b87634eced940d548 already in progress: creating tablet
I20260202 02:52:54.892622 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 914) took 1971 ms. Trace:
I20260202 02:52:54.892447 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 752) took 6735 ms. Trace:
I20260202 02:52:54.893103 1182 rpcz_store.cc:276] 0202 02:52:52.921380 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.924773 (+ 3393us) service_pool.cc:225] Handling call
0202 02:52:54.892491 (+1967718us) ts_tablet_manager.cc:1151] Deleting tablet 3857680ccbdc4dd4ad8247da96254434
0202 02:52:54.892505 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:54.892605 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1967508}
W20260202 02:52:54.893482 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 3857680ccbdc4dd4ad8247da96254434 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3857680ccbdc4dd4ad8247da96254434 already in progress: creating tablet
I20260202 02:52:54.893110 1180 rpcz_store.cc:276] 0202 02:52:48.156987 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:48.157213 (+ 226us) service_pool.cc:225] Handling call
0202 02:52:54.771887 (+6614674us) ts_tablet_manager.cc:1151] Deleting tablet 02b434fa058b45529ff3938b053263c6
0202 02:52:54.771896 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:54.817333 (+ 45437us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:54.874501 (+ 57168us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:54.892427 (+ 17926us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6614453}
I20260202 02:52:54.894199 1180 tablet_replica.cc:333] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1f442ef85029461091de847118c34c0d: stopping tablet replica
I20260202 02:52:54.894609 1182 tablet_replica.cc:333] T 7a44585a217b48f39f247bae3ca2e255 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:54.895207 1182 raft_consensus.cc:2243] T 7a44585a217b48f39f247bae3ca2e255 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:54.895002 1180 raft_consensus.cc:2243] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:54.896113 1180 raft_consensus.cc:2272] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:54.896456 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 842e54adae2a452a98521de24326eaff (table default.Table_c6e682a233904983978e7abb23565e41 [id=0d7624b55c1a4bf0a449f651ac419d77]) successfully deleted
I20260202 02:52:54.896648 1882 raft_consensus.cc:2272] T 3284213e4b0447b49a7718ba896d3fa8 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:54.897379 1882 ts_tablet_manager.cc:1916] T 3284213e4b0447b49a7718ba896d3fa8 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:54.897832 1180 ts_tablet_manager.cc:1916] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1f442ef85029461091de847118c34c0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:54.901046 1054 raft_consensus.cc:1275] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P b1f05820b6034228bdad884cc348716d [term 3 FOLLOWER]: Refusing update from remote peer 1f442ef85029461091de847118c34c0d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260202 02:52:54.901750 3161 consensus_queue.cc:1048] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260202 02:52:54.919003 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet abfe7f76b2424c4d9ab07b29b66ec103 (table default.Table_ff54266e8b5a4334ad1fbeda5270f0f2 [id=b6c09db68b4542a3bb11bf20b1d54571]) successfully deleted
W20260202 02:52:54.919981 513 catalog_manager.cc:4965] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): delete failed for tablet dd8f36d7a82f47adae1bd3c1f8628b6e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dd8f36d7a82f47adae1bd3c1f8628b6e already in progress: creating tablet
I20260202 02:52:54.923453 1043 tablet_service.cc:1505] Processing CreateTablet for tablet 223bf8b5e2684e49a316a3a654a0bf1a (DEFAULT_TABLE table=default.Table_eeb507a1a49d407196bff72270b0834f [id=a18136bbb87a47d69600f926a474da24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.924122 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 223bf8b5e2684e49a316a3a654a0bf1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.926252 749 tablet_service.cc:1505] Processing CreateTablet for tablet 6993b8ddf6cb403689d6cc99c860f1d1 (DEFAULT_TABLE table=default.Table_ee948408dac543078c1e7453d5930acc [id=3be73250c4994e279db99053c2185ca5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.926712 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6993b8ddf6cb403689d6cc99c860f1d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.928666 746 tablet_service.cc:1505] Processing CreateTablet for tablet b17a32ac48c848c79518917e041172b1 (DEFAULT_TABLE table=default.Table_58d515e7e01b4109b06e1e027ec60e3a [id=51f6afe54fec49b79e14b2d55387ce53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.929026 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b17a32ac48c848c79518917e041172b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.932706 740 tablet_service.cc:1505] Processing CreateTablet for tablet 13f53365036d4c57b520f8e282442a20 (DEFAULT_TABLE table=default.Table_0dd06fff80bc41c78d6f707fc8c0bee3 [id=2be6211dbde24f8db41e4fc1d02673fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.933126 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13f53365036d4c57b520f8e282442a20. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:54.934298 715 consensus_peers.cc:597] T 8bb14bf551f441dcbaaff7057b51b2be P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8bb14bf551f441dcbaaff7057b51b2be. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:54.934798 753 tablet_service.cc:1505] Processing CreateTablet for tablet 82efe2179fe54c01bf41630fca03802a (DEFAULT_TABLE table=default.Table_c8d5b0bad2904df78c01b87bbd5b8e7f [id=ff99fedab8cb41dc8b94941890c8f923]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.935148 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82efe2179fe54c01bf41630fca03802a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.936610 742 tablet_service.cc:1505] Processing CreateTablet for tablet 4735b8c368244bc8956c74a5bdef6658 (DEFAULT_TABLE table=default.Table_f479891b2cc74fc68dbdbd2a63e870ee [id=3f54b77e4da54574b6f707051d60c6ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.936945 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4735b8c368244bc8956c74a5bdef6658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.940672 755 tablet_service.cc:1505] Processing CreateTablet for tablet 902883ef12dc4c82b051634fb04feb19 (DEFAULT_TABLE table=default.Table_ab37f999ca1f4033ac2a25bdf3f8bbf6 [id=2fdf8e379824433196848d0ab4330877]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.941047 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 902883ef12dc4c82b051634fb04feb19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.944089 744 tablet_service.cc:1505] Processing CreateTablet for tablet e670b8b95a60482896450fb231bd4169 (DEFAULT_TABLE table=default.Table_32235d7fa05f42efb99094f7b8d3c93b [id=932164e1140849f193982a139df92b19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.944540 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e670b8b95a60482896450fb231bd4169. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.944566 753 tablet_service.cc:1505] Processing CreateTablet for tablet b4dddc2235ab4c9d898f5e7f3a2ac983 (DEFAULT_TABLE table=default.Table_74a1a2c121f1430d87ced4156ee01e53 [id=8a048083482c400b9dada08379e26471]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.944849 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4dddc2235ab4c9d898f5e7f3a2ac983. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.945013 757 tablet_service.cc:1505] Processing CreateTablet for tablet 8ee843650a344e99a1c9d46b1c523ac9 (DEFAULT_TABLE table=default.Table_bc3f75a4a12c4f92a4bbf19395b7800a [id=4ab3c9611a2c4d1280ae18acb2507c90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.945274 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ee843650a344e99a1c9d46b1c523ac9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.948647 759 tablet_service.cc:1505] Processing CreateTablet for tablet 98ee9e01f0354a2abdbc1c45f1e6acdf (DEFAULT_TABLE table=default.Table_f61ca1b0d7424868993ccd4657d50429 [id=3c38914fb1644aa79f39fb66939ebc92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.949011 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98ee9e01f0354a2abdbc1c45f1e6acdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.950372 2944 consensus_queue.cc:1048] T ac2ab12f5040488aa306cb69b107b485 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260202 02:52:54.932399 3032 raft_consensus.cc:493] T 5e20842de8cb4355b732685605c88834 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:54.951705 3032 raft_consensus.cc:515] T 5e20842de8cb4355b732685605c88834 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:54.951737 748 tablet_service.cc:1505] Processing CreateTablet for tablet 3590bfe57ac346b48e059a8c2deaafd9 (DEFAULT_TABLE table=default.Table_2d519304c313459a8b38c0fe0c65d21f [id=022bce98e93d4821a275d6a26dd033aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.952097 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3590bfe57ac346b48e059a8c2deaafd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.952234 3032 leader_election.cc:290] T 5e20842de8cb4355b732685605c88834 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:54.937667 998 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 116912us
I20260202 02:52:54.953001 766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "93f958924885457ab15f2aac5509e0aa" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:54.953182 766 raft_consensus.cc:3060] T 93f958924885457ab15f2aac5509e0aa P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:54.955356 745 tablet_service.cc:1505] Processing CreateTablet for tablet ec5f5e47588d46da98d4b7e340b5138a (DEFAULT_TABLE table=default.Table_42bf0269c9e64c62adfa02c34d69459e [id=ac38ffe9c0a144a98b1f723643011f71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.955715 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec5f5e47588d46da98d4b7e340b5138a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.955878 766 raft_consensus.cc:2468] T 93f958924885457ab15f2aac5509e0aa P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:54.957401 764 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f5727f9b9bbb4e77abfef065c51606a1" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:54.957602 764 raft_consensus.cc:3060] T f5727f9b9bbb4e77abfef065c51606a1 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:54.960073 764 raft_consensus.cc:2468] T f5727f9b9bbb4e77abfef065c51606a1 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:54.960922 764 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7ff6d2fa085343a6b11be0898da6233d" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d"
I20260202 02:52:54.961081 764 raft_consensus.cc:3060] T 7ff6d2fa085343a6b11be0898da6233d P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:54.961149 758 tablet_service.cc:1505] Processing CreateTablet for tablet adf26e4901dc4ddda6de50a87855cc9d (DEFAULT_TABLE table=default.Table_a64a2e7c03c74179a133b3bcd1da1d27 [id=b355b1e7bfcd4164800f8da86d341cf9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.961447 745 tablet_service.cc:1505] Processing CreateTablet for tablet 32ee0da311f1453d8a2d30f9d8be6702 (DEFAULT_TABLE table=default.Table_1fa8930369e844ca8384126bf8f22e66 [id=116247212fca4c0a95ad1bdbbac4787f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.961529 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adf26e4901dc4ddda6de50a87855cc9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.961782 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32ee0da311f1453d8a2d30f9d8be6702. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.961982 741 tablet_service.cc:1505] Processing CreateTablet for tablet 25a54991224a4c8da9e0e278a054e7b0 (DEFAULT_TABLE table=default.Table_7f1d13e7a79e4a86ad883eb6e5d214e0 [id=2c2446c1766b4e4c9d5eead5cd9b9290]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.962237 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25a54991224a4c8da9e0e278a054e7b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.963548 764 raft_consensus.cc:2468] T 7ff6d2fa085343a6b11be0898da6233d P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:54.964430 998 leader_election.cc:304] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:54.964740 998 leader_election.cc:304] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:54.965003 751 tablet_service.cc:1505] Processing CreateTablet for tablet cc7cb7a8cadc4c90a2449f9d7183aa9a (DEFAULT_TABLE table=default.Table_6831562d21db463986dc58b0100e9792 [id=50e58db611f247ee9dcd5b31776ace05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.965338 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc7cb7a8cadc4c90a2449f9d7183aa9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.966851 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5e20842de8cb4355b732685605c88834" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:54.967743 3032 raft_consensus.cc:2804] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:54.971933 3032 raft_consensus.cc:697] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:54.972406 3032 consensus_queue.cc:237] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.973625 3181 raft_consensus.cc:2804] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:54.973796 3181 raft_consensus.cc:697] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:54.974088 3181 consensus_queue.cc:237] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } }
I20260202 02:52:54.941783 2979 raft_consensus.cc:697] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:54.899025 1175 tablet_bootstrap.cc:492] T dc4c3f52f4f44a84b0d0e3c9a4a49f09 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:54.923372 898 tablet_service.cc:1505] Processing CreateTablet for tablet 223bf8b5e2684e49a316a3a654a0bf1a (DEFAULT_TABLE table=default.Table_eeb507a1a49d407196bff72270b0834f [id=a18136bbb87a47d69600f926a474da24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.941661 890 tablet_service.cc:1505] Processing CreateTablet for tablet 18a4930e3d5c42e9a9ac50052e7f38af (DEFAULT_TABLE table=default.Table_19135ff47c4d4097abe42c140d685d9b [id=2196e4a0c40648128b95c47f7b75c572]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.943326 899 tablet_service.cc:1505] Processing CreateTablet for tablet 66e0db2d16594503a39916c61ad84824 (DEFAULT_TABLE table=default.Table_1403dbd979384147afa3d43063fdc558 [id=4c42657a14e44293a534cb3b9c53427d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.975857 752 tablet_service.cc:1505] Processing CreateTablet for tablet 4381644c4d874bd7b1d62537edb0bbf1 (DEFAULT_TABLE table=default.Table_b5055a016bd5483980012ad46c8c68c9 [id=4203187b5306439fba0774f9d0cd4bfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.948840 3080 raft_consensus.cc:493] T 7921daa000f54c8dbed04473e21cf0a5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260202 02:52:54.950438 857 leader_election.cc:343] T 09504070c9ec4bcfbb59e3fb4c189775 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:54.975601 1175 ts_tablet_manager.cc:1403] T dc4c3f52f4f44a84b0d0e3c9a4a49f09 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.149s user 0.015s sys 0.015s
I20260202 02:52:54.976210 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4381644c4d874bd7b1d62537edb0bbf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.975800 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 223bf8b5e2684e49a316a3a654a0bf1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.976967 1175 raft_consensus.cc:359] T dc4c3f52f4f44a84b0d0e3c9a4a49f09 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:54.977109 1175 raft_consensus.cc:385] T dc4c3f52f4f44a84b0d0e3c9a4a49f09 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:54.977150 1175 raft_consensus.cc:740] T dc4c3f52f4f44a84b0d0e3c9a4a49f09 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:54.977309 1175 consensus_queue.cc:260] T dc4c3f52f4f44a84b0d0e3c9a4a49f09 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:54.977762 1175 ts_tablet_manager.cc:1434] T dc4c3f52f4f44a84b0d0e3c9a4a49f09 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:54.978035 1175 tablet_bootstrap.cc:492] T bb1005103c25423783736ace1866c328 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:54.979542 1175 tablet_bootstrap.cc:654] T bb1005103c25423783736ace1866c328 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:54.979698 2979 consensus_queue.cc:237] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.980466 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66e0db2d16594503a39916c61ad84824. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.980724 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18a4930e3d5c42e9a9ac50052e7f38af. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:54.941826 999 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.130:38539, user_credentials={real_user=slave}} blocked reactor thread for 112326us
W20260202 02:52:54.983157 999 leader_election.cc:343] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 93f958924885457ab15f2aac5509e0aa
W20260202 02:52:54.983464 999 leader_election.cc:343] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 7ff6d2fa085343a6b11be0898da6233d
W20260202 02:52:54.983745 999 leader_election.cc:343] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 6a3dac5cd1ce47488a50b4c32eca67c7
W20260202 02:52:54.984285 999 leader_election.cc:343] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: f5727f9b9bbb4e77abfef065c51606a1
I20260202 02:52:54.976071 3080 raft_consensus.cc:515] T 7921daa000f54c8dbed04473e21cf0a5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
W20260202 02:52:54.984622 999 leader_election.cc:343] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: b4a4c75a906f40beb896bf6fff705ae6
W20260202 02:52:54.984920 999 leader_election.cc:343] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 93f958924885457ab15f2aac5509e0aa
I20260202 02:52:54.985077 3080 leader_election.cc:290] T 7921daa000f54c8dbed04473e21cf0a5 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:54.985694 857 leader_election.cc:304] T 09504070c9ec4bcfbb59e3fb4c189775 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
W20260202 02:52:54.986146 857 leader_election.cc:343] T bbcc723a8079443db455ae20fb0d585d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:54.986322 857 leader_election.cc:304] T bbcc723a8079443db455ae20fb0d585d P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:54.986737 2964 raft_consensus.cc:2749] T bbcc723a8079443db455ae20fb0d585d P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:54.986853 857 leader_election.cc:343] T 3dcf4f794ec14b2f9d181d015fc437fb P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:54.986933 857 leader_election.cc:304] T 3dcf4f794ec14b2f9d181d015fc437fb P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:54.987110 2964 raft_consensus.cc:2749] T 3dcf4f794ec14b2f9d181d015fc437fb P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:54.986533 3080 raft_consensus.cc:2749] T 09504070c9ec4bcfbb59e3fb4c189775 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:54.948652 1035 tablet_service.cc:1505] Processing CreateTablet for tablet 18a4930e3d5c42e9a9ac50052e7f38af (DEFAULT_TABLE table=default.Table_19135ff47c4d4097abe42c140d685d9b [id=2196e4a0c40648128b95c47f7b75c572]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.989090 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18a4930e3d5c42e9a9ac50052e7f38af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.989840 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5e20842de8cb4355b732685605c88834" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:54.990067 911 raft_consensus.cc:2393] T 5e20842de8cb4355b732685605c88834 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1f05820b6034228bdad884cc348716d in current term 1: Already voted for candidate 1f442ef85029461091de847118c34c0d in this term.
I20260202 02:52:54.951360 1173 tablet_bootstrap.cc:492] T c17c85759e964f0ba3abcaf5fe938fba P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:54.991990 1173 ts_tablet_manager.cc:1403] T c17c85759e964f0ba3abcaf5fe938fba P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.286s user 0.006s sys 0.006s
I20260202 02:52:54.992800 1173 raft_consensus.cc:359] T c17c85759e964f0ba3abcaf5fe938fba P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.993038 1173 raft_consensus.cc:385] T c17c85759e964f0ba3abcaf5fe938fba P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:54.993139 1173 raft_consensus.cc:740] T c17c85759e964f0ba3abcaf5fe938fba P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:54.993408 890 tablet_service.cc:1552] Processing DeleteTablet for tablet dc4c3f52f4f44a84b0d0e3c9a4a49f09 with delete_type TABLET_DATA_DELETED (replaced by tablet 3191a80374724effa8389adc842e3448) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:54.993384 1173 consensus_queue.cc:260] T c17c85759e964f0ba3abcaf5fe938fba P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:54.994004 1173 ts_tablet_manager.cc:1434] T c17c85759e964f0ba3abcaf5fe938fba P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:54.994308 1173 tablet_bootstrap.cc:492] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:54.994570 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 07580ad1d049475a9645acfad0b1d219 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:42950
I20260202 02:52:54.995194 751 tablet_service.cc:1505] Processing CreateTablet for tablet 2740246e8fad411abb09db0fd00beda0 (DEFAULT_TABLE table=default.Table_b6f0e40295b247c493d61b5369e218f3 [id=0efa2fe908a443f6b44ccb1bc13da814]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:54.995687 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2740246e8fad411abb09db0fd00beda0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.995836 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7921daa000f54c8dbed04473e21cf0a5" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:54.996480 859 leader_election.cc:343] T 7921daa000f54c8dbed04473e21cf0a5 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:55.000809 998 leader_election.cc:304] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:55.001340 2977 raft_consensus.cc:2804] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:55.001451 2977 raft_consensus.cc:697] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:55.001693 2977 consensus_queue.cc:237] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:55.002898 2794 consensus_queue.cc:1048] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:55.004769 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 02b434fa058b45529ff3938b053263c6 (table default.Table_6133c617ff8d40eca5cbde96533acd82 [id=5a502f86d98b42c6a3e274de5bf08544]) successfully deleted
I20260202 02:52:55.005980 915 raft_consensus.cc:1275] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [term 3 FOLLOWER]: Refusing update from remote peer 1f442ef85029461091de847118c34c0d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 3. (term mismatch)
I20260202 02:52:55.006091 915 pending_rounds.cc:85] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5: Aborting all ops after (but not including) 2
I20260202 02:52:55.006138 915 pending_rounds.cc:107] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5: Aborting uncommitted NO_OP operation due to leader change: 2.3
I20260202 02:52:55.006230 915 raft_consensus.cc:2889] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1521a6d262b7451bbef8debe970dd6b5 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260202 02:52:54.974607 1182 raft_consensus.cc:2272] T 7a44585a217b48f39f247bae3ca2e255 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:55.008232 1182 ts_tablet_manager.cc:1916] T 7a44585a217b48f39f247bae3ca2e255 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:55.012013 759 tablet_service.cc:1505] Processing CreateTablet for tablet 18a4930e3d5c42e9a9ac50052e7f38af (DEFAULT_TABLE table=default.Table_19135ff47c4d4097abe42c140d685d9b [id=2196e4a0c40648128b95c47f7b75c572]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.012688 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18a4930e3d5c42e9a9ac50052e7f38af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.012769 751 tablet_service.cc:1505] Processing CreateTablet for tablet 223bf8b5e2684e49a316a3a654a0bf1a (DEFAULT_TABLE table=default.Table_eeb507a1a49d407196bff72270b0834f [id=a18136bbb87a47d69600f926a474da24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.013051 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 223bf8b5e2684e49a316a3a654a0bf1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.014436 2944 consensus_queue.cc:1048] T 7ea7a5d8a92b4ac0903483627258ae66 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:55.015802 2794 consensus_queue.cc:1048] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:52:55.016875 1174 tablet_bootstrap.cc:492] T 7f9dfac19c04475aadb45e95a6474eb4 P 1f442ef85029461091de847118c34c0d: No bootstrap required, opened a new log
W20260202 02:52:55.016917 999 consensus_peers.cc:597] T b427c7e897c64977b924e7d586420e69 P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b427c7e897c64977b924e7d586420e69. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:55.017154 1174 ts_tablet_manager.cc:1403] T 7f9dfac19c04475aadb45e95a6474eb4 P 1f442ef85029461091de847118c34c0d: Time spent bootstrapping tablet: real 0.142s user 0.005s sys 0.001s
I20260202 02:52:55.017922 1174 raft_consensus.cc:359] T 7f9dfac19c04475aadb45e95a6474eb4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:55.018167 1174 raft_consensus.cc:385] T 7f9dfac19c04475aadb45e95a6474eb4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260202 02:52:55.018150 999 leader_election.cc:343] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 7ff6d2fa085343a6b11be0898da6233d
I20260202 02:52:55.018250 1174 raft_consensus.cc:740] T 7f9dfac19c04475aadb45e95a6474eb4 P 1f442ef85029461091de847118c34c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f442ef85029461091de847118c34c0d, State: Initialized, Role: FOLLOWER
I20260202 02:52:55.018476 1174 consensus_queue.cc:260] T 7f9dfac19c04475aadb45e95a6474eb4 P 1f442ef85029461091de847118c34c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
W20260202 02:52:55.018599 999 leader_election.cc:343] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: f5727f9b9bbb4e77abfef065c51606a1
I20260202 02:52:55.019021 1174 ts_tablet_manager.cc:1434] T 7f9dfac19c04475aadb45e95a6474eb4 P 1f442ef85029461091de847118c34c0d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:55.019321 1174 tablet_bootstrap.cc:492] T 9a1d7d173c41462c8e32a95b494bb708 P 1f442ef85029461091de847118c34c0d: Bootstrap starting.
I20260202 02:52:55.020097 998 leader_election.cc:304] T 5e20842de8cb4355b732685605c88834 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:55.020479 2794 raft_consensus.cc:3060] T 5e20842de8cb4355b732685605c88834 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:55.022972 2794 raft_consensus.cc:2749] T 5e20842de8cb4355b732685605c88834 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:55.027127 899 tablet_service.cc:1505] Processing CreateTablet for tablet cd00299690ef49e185185c6f6107d221 (DEFAULT_TABLE table=default.Table_20da10c887054d929c1fcc0f81006103 [id=56c7af1ab763463984bf26f93cec4e0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.028635 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd00299690ef49e185185c6f6107d221. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:54.932313 1882 ts_tablet_manager.cc:1929] T 3284213e4b0447b49a7718ba896d3fa8 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:55.029419 1882 log.cc:1199] T 3284213e4b0447b49a7718ba896d3fa8 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/3284213e4b0447b49a7718ba896d3fa8
I20260202 02:52:55.029765 1882 ts_tablet_manager.cc:1950] T 3284213e4b0447b49a7718ba896d3fa8 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:55.030540 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 800) took 5164 ms. Trace:
I20260202 02:52:55.030629 1882 rpcz_store.cc:276] 0202 02:52:49.865912 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:49.885858 (+ 19946us) service_pool.cc:225] Handling call
0202 02:52:54.668640 (+4782782us) ts_tablet_manager.cc:1151] Deleting tablet 3284213e4b0447b49a7718ba896d3fa8
0202 02:52:54.814311 (+145671us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:54.929422 (+115111us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:54.932205 (+ 2783us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.030514 (+ 98309us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":81331,"tablet-delete.queue_time_us":4782516}
I20260202 02:52:55.031143 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 3284213e4b0447b49a7718ba896d3fa8 (table default.Table_e71a7347855d447cabe6ea2cc64b293a [id=e490dd882e8e4e51ad5a2073c891ba91]) successfully deleted
I20260202 02:52:55.031508 1882 tablet_replica.cc:333] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:55.031706 1882 raft_consensus.cc:2243] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:55.031973 1882 raft_consensus.cc:2272] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:55.032618 1882 ts_tablet_manager.cc:1916] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:55.013379 753 tablet_service.cc:1505] Processing CreateTablet for tablet ca5e638942e14b829b80d99a5884305c (DEFAULT_TABLE table=default.Table_8161c74bbc974802bd0baade2f1fb103 [id=adae2af9c45e4cda8f87cb6fcf9001ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.037024 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca5e638942e14b829b80d99a5884305c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.041359 1175 tablet_bootstrap.cc:492] T bb1005103c25423783736ace1866c328 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:55.041612 1175 ts_tablet_manager.cc:1403] T bb1005103c25423783736ace1866c328 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.064s user 0.005s sys 0.006s
I20260202 02:52:55.042555 1175 raft_consensus.cc:359] T bb1005103c25423783736ace1866c328 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.042858 1175 raft_consensus.cc:385] T bb1005103c25423783736ace1866c328 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:55.042954 1175 raft_consensus.cc:740] T bb1005103c25423783736ace1866c328 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:55.043257 1175 consensus_queue.cc:260] T bb1005103c25423783736ace1866c328 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
W20260202 02:52:55.037622 714 leader_election.cc:343] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: f1c8f7e1dc9644a6b6dd2bc264cce8ef
W20260202 02:52:55.045847 714 leader_election.cc:343] T b4a4c75a906f40beb896bf6fff705ae6 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: b4a4c75a906f40beb896bf6fff705ae6
I20260202 02:52:55.046499 714 leader_election.cc:304] T b4a4c75a906f40beb896bf6fff705ae6 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:55.009982 1180 ts_tablet_manager.cc:1929] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1f442ef85029461091de847118c34c0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:55.046988 1180 log.cc:1199] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1f442ef85029461091de847118c34c0d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/874ca4ba4ebc4721bb4b60cba8e01841
I20260202 02:52:55.047446 1180 ts_tablet_manager.cc:1950] T 874ca4ba4ebc4721bb4b60cba8e01841 P 1f442ef85029461091de847118c34c0d: Deleting consensus metadata
I20260202 02:52:55.049702 1180 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42950 (request call id 753) took 6892 ms. Trace:
I20260202 02:52:54.995922 1173 tablet_bootstrap.cc:654] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:55.047768 3144 raft_consensus.cc:2749] T b4a4c75a906f40beb896bf6fff705ae6 P 1f442ef85029461091de847118c34c0d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:55.048754 714 leader_election.cc:343] T 07580ad1d049475a9645acfad0b1d219 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 07580ad1d049475a9645acfad0b1d219
W20260202 02:52:55.061772 714 leader_election.cc:343] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: f1c8f7e1dc9644a6b6dd2bc264cce8ef
W20260202 02:52:55.062136 714 leader_election.cc:343] T 07580ad1d049475a9645acfad0b1d219 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 07580ad1d049475a9645acfad0b1d219
W20260202 02:52:55.062811 714 leader_election.cc:343] T 26b890ed87c3471b9fc0b172c90e0a11 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 26b890ed87c3471b9fc0b172c90e0a11
I20260202 02:52:55.062955 714 leader_election.cc:304] T 26b890ed87c3471b9fc0b172c90e0a11 P 1f442ef85029461091de847118c34c0d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f442ef85029461091de847118c34c0d; no voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d
I20260202 02:52:55.063443 3160 raft_consensus.cc:2749] T 26b890ed87c3471b9fc0b172c90e0a11 P 1f442ef85029461091de847118c34c0d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260202 02:52:55.038569 1174 tablet_bootstrap.cc:654] T 9a1d7d173c41462c8e32a95b494bb708 P 1f442ef85029461091de847118c34c0d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:55.056424 715 consensus_peers.cc:597] T 6df3d11cf578404ca41807f0777b6c53 P 1f442ef85029461091de847118c34c0d -> Peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Couldn't send request to peer b1f05820b6034228bdad884cc348716d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6df3d11cf578404ca41807f0777b6c53. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:55.064224 819 generic_service.cc:196] Checking for leaks (request via RPC)
I20260202 02:52:55.064440 1180 rpcz_store.cc:276] 0202 02:52:48.157088 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:48.157472 (+ 384us) service_pool.cc:225] Handling call
0202 02:52:54.894141 (+6736669us) ts_tablet_manager.cc:1151] Deleting tablet 874ca4ba4ebc4721bb4b60cba8e01841
0202 02:52:54.894154 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:54.915239 (+ 21085us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.009874 (+ 94635us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.049680 (+ 39806us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6736537}
W20260202 02:52:55.065527 714 consensus_peers.cc:597] T 7ea7a5d8a92b4ac0903483627258ae66 P 1f442ef85029461091de847118c34c0d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7ea7a5d8a92b4ac0903483627258ae66. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:55.059545 764 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7921daa000f54c8dbed04473e21cf0a5" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
I20260202 02:52:55.066267 3144 consensus_queue.cc:1048] T 20ea39bc5d3d46f5a4ba427277d6c6a3 P 1f442ef85029461091de847118c34c0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260202 02:52:55.070483 3080 raft_consensus.cc:493] T c9ef08b5830147a3a55049285a40535b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:55.070684 3080 raft_consensus.cc:515] T c9ef08b5830147a3a55049285a40535b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.071326 3080 leader_election.cc:290] T c9ef08b5830147a3a55049285a40535b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:55.071547 513 catalog_manager.cc:4985] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): tablet 874ca4ba4ebc4721bb4b60cba8e01841 (table default.Table_9913ba1a0b0c4f30bc3dd302ee61b92d [id=6777a441613b40cbb21d431744fca23f]) successfully deleted
I20260202 02:52:55.077229 3032 raft_consensus.cc:493] T ee04563d3b8e4cb2b989bb8d6ee9b20e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:55.077519 3032 raft_consensus.cc:515] T ee04563d3b8e4cb2b989bb8d6ee9b20e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:55.078305 3032 leader_election.cc:290] T ee04563d3b8e4cb2b989bb8d6ee9b20e P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:55.077301 999 consensus_peers.cc:597] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:55.084890 999 consensus_peers.cc:597] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2616e1bbce4049dbb584550ccc150288. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:55.084573 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c9ef08b5830147a3a55049285a40535b" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:55.088408 999 leader_election.cc:343] T ee04563d3b8e4cb2b989bb8d6ee9b20e P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: ee04563d3b8e4cb2b989bb8d6ee9b20e
W20260202 02:52:55.088562 859 leader_election.cc:343] T c9ef08b5830147a3a55049285a40535b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:55.102577 1173 tablet_bootstrap.cc:492] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:55.102866 1173 ts_tablet_manager.cc:1403] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.109s user 0.001s sys 0.003s
I20260202 02:52:55.103690 1173 raft_consensus.cc:359] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:55.103947 1173 raft_consensus.cc:385] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:55.104032 1173 raft_consensus.cc:740] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:55.104285 1173 consensus_queue.cc:260] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:55.104873 1173 ts_tablet_manager.cc:1434] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:55.105190 1173 tablet_bootstrap.cc:492] T 3dcf4f794ec14b2f9d181d015fc437fb P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:55.113862 1037 tablet_service.cc:1505] Processing CreateTablet for tablet cd00299690ef49e185185c6f6107d221 (DEFAULT_TABLE table=default.Table_20da10c887054d929c1fcc0f81006103 [id=56c7af1ab763463984bf26f93cec4e0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.114420 3181 raft_consensus.cc:493] T eea6bb04ddf24276b46270d009ed78b4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:55.114712 3181 raft_consensus.cc:515] T eea6bb04ddf24276b46270d009ed78b4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:55.115978 3181 leader_election.cc:290] T eea6bb04ddf24276b46270d009ed78b4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:55.119968 999 leader_election.cc:343] T eea6bb04ddf24276b46270d009ed78b4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: eea6bb04ddf24276b46270d009ed78b4
I20260202 02:52:55.122025 1043 tablet_service.cc:1505] Processing CreateTablet for tablet 66e0db2d16594503a39916c61ad84824 (DEFAULT_TABLE table=default.Table_1403dbd979384147afa3d43063fdc558 [id=4c42657a14e44293a534cb3b9c53427d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.123303 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd00299690ef49e185185c6f6107d221. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.125340 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c9ef08b5830147a3a55049285a40535b" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f442ef85029461091de847118c34c0d" is_pre_election: true
W20260202 02:52:55.125640 857 leader_election.cc:343] T 7921daa000f54c8dbed04473e21cf0a5 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:55.126005 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66e0db2d16594503a39916c61ad84824. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.125928 857 leader_election.cc:304] T 7921daa000f54c8dbed04473e21cf0a5 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
I20260202 02:52:55.126412 3080 raft_consensus.cc:2749] T 7921daa000f54c8dbed04473e21cf0a5 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:55.129330 1175 ts_tablet_manager.cc:1434] T bb1005103c25423783736ace1866c328 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.088s user 0.001s sys 0.001s
I20260202 02:52:55.130403 1175 tablet_bootstrap.cc:492] T 67a8c9a1998b47b3b214a06ce62381e9 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:55.139428 3080 raft_consensus.cc:493] T 842197c689064660b791b40e521d346b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:55.139601 3080 raft_consensus.cc:515] T 842197c689064660b791b40e521d346b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.140240 3080 leader_election.cc:290] T 842197c689064660b791b40e521d346b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:55.141716 1175 tablet_bootstrap.cc:654] T 67a8c9a1998b47b3b214a06ce62381e9 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:55.141713 1054 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "842197c689064660b791b40e521d346b" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:55.142977 859 leader_election.cc:343] T 842197c689064660b791b40e521d346b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:55.143142 1028 tablet_service.cc:1505] Processing CreateTablet for tablet a947a02cc9e8470dbb23dd86e9e87f52 (DEFAULT_TABLE table=default.Table_0990710bca2c4106a2a43ad8ca140c7c [id=fc787468e5e24682bf4b565883552093]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.143530 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a947a02cc9e8470dbb23dd86e9e87f52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.144701 899 tablet_service.cc:1505] Processing CreateTablet for tablet a947a02cc9e8470dbb23dd86e9e87f52 (DEFAULT_TABLE table=default.Table_0990710bca2c4106a2a43ad8ca140c7c [id=fc787468e5e24682bf4b565883552093]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.145187 1182 ts_tablet_manager.cc:1929] T 7a44585a217b48f39f247bae3ca2e255 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:55.145428 1182 log.cc:1199] T 7a44585a217b48f39f247bae3ca2e255 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/7a44585a217b48f39f247bae3ca2e255
I20260202 02:52:55.145539 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a947a02cc9e8470dbb23dd86e9e87f52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.145803 1182 ts_tablet_manager.cc:1950] T 7a44585a217b48f39f247bae3ca2e255 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:55.150385 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 917) took 2211 ms. Trace:
I20260202 02:52:55.150597 1182 rpcz_store.cc:276] 0202 02:52:52.938427 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.938495 (+ 68us) service_pool.cc:225] Handling call
0202 02:52:54.894562 (+1956067us) ts_tablet_manager.cc:1151] Deleting tablet 7a44585a217b48f39f247bae3ca2e255
0202 02:52:54.894572 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.135567 (+240995us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.145080 (+ 9513us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.150357 (+ 5277us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1955821}
I20260202 02:52:55.151124 1182 tablet_replica.cc:333] T 25a407001f964da6b51f42375136c79a P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:55.151278 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 7a44585a217b48f39f247bae3ca2e255 (table default.Table_2ee0b536f96e44e4a3a4a9439c7bccc9 [id=d95486b11c494863be7380146ebcf0c8]) successfully deleted
I20260202 02:52:55.151631 1182 raft_consensus.cc:2243] T 25a407001f964da6b51f42375136c79a P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:55.152045 1182 raft_consensus.cc:2272] T 25a407001f964da6b51f42375136c79a P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:55.153395 1182 ts_tablet_manager.cc:1916] T 25a407001f964da6b51f42375136c79a P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:55.152825 1175 tablet_bootstrap.cc:492] T 67a8c9a1998b47b3b214a06ce62381e9 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:55.153695 1175 ts_tablet_manager.cc:1403] T 67a8c9a1998b47b3b214a06ce62381e9 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.000s
I20260202 02:52:55.154556 1175 raft_consensus.cc:359] T 67a8c9a1998b47b3b214a06ce62381e9 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.154819 1175 raft_consensus.cc:385] T 67a8c9a1998b47b3b214a06ce62381e9 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:55.154923 1175 raft_consensus.cc:740] T 67a8c9a1998b47b3b214a06ce62381e9 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:55.155238 1175 consensus_queue.cc:260] T 67a8c9a1998b47b3b214a06ce62381e9 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.159391 1175 ts_tablet_manager.cc:1434] T 67a8c9a1998b47b3b214a06ce62381e9 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20260202 02:52:55.170781 890 tablet_service.cc:1505] Processing CreateTablet for tablet 4735b85a728d4d43b003214b06937f27 (DEFAULT_TABLE table=default.Table_66a25197aee2457cb4d2ace107cb7f01 [id=e0d5c4d052df4bca9efefd4006602cf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.170809 898 tablet_service.cc:1505] Processing CreateTablet for tablet 60feac4d4fb841c0834f030db0f599d1 (DEFAULT_TABLE table=default.Table_dd4c50f48678452cb127469d5a7bf548 [id=d20907a677b742ff8a793c13a1faf28d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.172443 1035 tablet_service.cc:1505] Processing CreateTablet for tablet 4735b85a728d4d43b003214b06937f27 (DEFAULT_TABLE table=default.Table_66a25197aee2457cb4d2ace107cb7f01 [id=e0d5c4d052df4bca9efefd4006602cf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.173008 1182 ts_tablet_manager.cc:1929] T 25a407001f964da6b51f42375136c79a P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:55.173481 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4735b85a728d4d43b003214b06937f27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.175551 1028 tablet_service.cc:1505] Processing CreateTablet for tablet 60feac4d4fb841c0834f030db0f599d1 (DEFAULT_TABLE table=default.Table_dd4c50f48678452cb127469d5a7bf548 [id=d20907a677b742ff8a793c13a1faf28d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.175966 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60feac4d4fb841c0834f030db0f599d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.177285 1175 tablet_bootstrap.cc:492] T e95153ee8f2d40de955e3389d3a58644 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:55.177553 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4735b85a728d4d43b003214b06937f27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.181613 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60feac4d4fb841c0834f030db0f599d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.183455 1026 tablet_service.cc:1505] Processing CreateTablet for tablet c8606fb177db49c7b2e2b16b18d4725c (DEFAULT_TABLE table=default.Table_6806575803c4468fa7b7ef990004b446 [id=944fbe1acd894017bae3c54f135ee9b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.188927 902 tablet_service.cc:1505] Processing CreateTablet for tablet f89c5ad8995044d7a5f6026c1a047fba (DEFAULT_TABLE table=default.Table_7b218d827e024dee86865d1c48ab6cf9 [id=5355076e6963472cb00450df549f62bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.189221 1037 tablet_service.cc:1505] Processing CreateTablet for tablet f89c5ad8995044d7a5f6026c1a047fba (DEFAULT_TABLE table=default.Table_7b218d827e024dee86865d1c48ab6cf9 [id=5355076e6963472cb00450df549f62bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.189347 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f89c5ad8995044d7a5f6026c1a047fba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.190835 895 tablet_service.cc:1505] Processing CreateTablet for tablet 9c1e408a75fe4d948a355f0dede46316 (DEFAULT_TABLE table=default.Table_75db64440693483083cf6cb51164e115 [id=d32c18fb23514443a7774a6bb56030ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.191185 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c1e408a75fe4d948a355f0dede46316. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.192919 1043 tablet_service.cc:1505] Processing CreateTablet for tablet 9c1e408a75fe4d948a355f0dede46316 (DEFAULT_TABLE table=default.Table_75db64440693483083cf6cb51164e115 [id=d32c18fb23514443a7774a6bb56030ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.193810 1175 tablet_bootstrap.cc:654] T e95153ee8f2d40de955e3389d3a58644 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:55.194753 1028 tablet_service.cc:1505] Processing CreateTablet for tablet da9d3228ad734065bfc640833c988c92 (DEFAULT_TABLE table=default.Table_509184aa45184b7c8228aba2de1c528a [id=d3627805d70e4e269f073250e883f3e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.196259 1034 tablet_service.cc:1505] Processing CreateTablet for tablet 07dd6e4d34b94162bba0eb2a4ca84708 (DEFAULT_TABLE table=default.Table_c59a4ef49a364dcd8acdee41798c5793 [id=201dfda859414ca8b7c7b4f0bbd62b42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.196743 3080 raft_consensus.cc:493] T a81e1d10ca49459989260fbd6a2b8645 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:55.196880 3080 raft_consensus.cc:515] T a81e1d10ca49459989260fbd6a2b8645 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.197464 3080 leader_election.cc:290] T a81e1d10ca49459989260fbd6a2b8645 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:55.198143 1030 tablet_service.cc:1505] Processing CreateTablet for tablet ecd7fccb38c64f73a08a7af4210e1982 (DEFAULT_TABLE table=default.Table_7ab92cab118c400ca54a1361f373133b [id=ce5e9796a5e74ecba86fae368ce30a26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.198287 894 tablet_service.cc:1505] Processing CreateTablet for tablet 07dd6e4d34b94162bba0eb2a4ca84708 (DEFAULT_TABLE table=default.Table_c59a4ef49a364dcd8acdee41798c5793 [id=201dfda859414ca8b7c7b4f0bbd62b42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.198625 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07dd6e4d34b94162bba0eb2a4ca84708. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.199932 1036 tablet_service.cc:1505] Processing CreateTablet for tablet d98b04c60e9845c3abb8a74c1ef9620b (DEFAULT_TABLE table=default.Table_c5b9c9e8e8d043bdbb55326857791c2a [id=01a0f786eaa94e4f8a46c239ba34c4b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.201566 1039 tablet_service.cc:1505] Processing CreateTablet for tablet b83c65eb64d447feb462a24255141cb2 (DEFAULT_TABLE table=default.Table_5d59e3cf9d5a41afa503a76353543218 [id=0f35ce1317bf4b85b328e67c8911dc89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.203291 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 3bad993086e14d7b944f32b515508fc0 (DEFAULT_TABLE table=default.Table_06e158b452e14d3eba4e5e9290d48c7c [id=43648b9d84e1492c971291bccb5e6167]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.188927 898 tablet_service.cc:1505] Processing CreateTablet for tablet c8606fb177db49c7b2e2b16b18d4725c (DEFAULT_TABLE table=default.Table_6806575803c4468fa7b7ef990004b446 [id=944fbe1acd894017bae3c54f135ee9b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.205315 1042 tablet_service.cc:1505] Processing CreateTablet for tablet 165320266c8e4a258372f225d0372daa (DEFAULT_TABLE table=default.Table_66adee49a1b045918360d13af6bca9a2 [id=4096475017214ffdba27d20aca4e8d64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.205734 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8606fb177db49c7b2e2b16b18d4725c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.207168 1038 tablet_service.cc:1505] Processing CreateTablet for tablet 39d9e3fef89d4c1cb053278e25796eaf (DEFAULT_TABLE table=default.Table_93c8b5e956214e88abc353bfb46c8757 [id=1cb84b11b2f34287bbade3af6cd4df67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.207336 902 tablet_service.cc:1505] Processing CreateTablet for tablet da9d3228ad734065bfc640833c988c92 (DEFAULT_TABLE table=default.Table_509184aa45184b7c8228aba2de1c528a [id=d3627805d70e4e269f073250e883f3e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.207702 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da9d3228ad734065bfc640833c988c92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.208886 1032 tablet_service.cc:1505] Processing CreateTablet for tablet 862047d463374985ae0562f86ca2089d (DEFAULT_TABLE table=default.Table_28e7d9aa8131413e9a174756c246d164 [id=ec07c59893cf45c9b2a5eece556d10c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.209136 892 tablet_service.cc:1505] Processing CreateTablet for tablet 3bad993086e14d7b944f32b515508fc0 (DEFAULT_TABLE table=default.Table_06e158b452e14d3eba4e5e9290d48c7c [id=43648b9d84e1492c971291bccb5e6167]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.209501 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3bad993086e14d7b944f32b515508fc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.209867 890 tablet_service.cc:1505] Processing CreateTablet for tablet ecd7fccb38c64f73a08a7af4210e1982 (DEFAULT_TABLE table=default.Table_7ab92cab118c400ca54a1361f373133b [id=ce5e9796a5e74ecba86fae368ce30a26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.210189 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecd7fccb38c64f73a08a7af4210e1982. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.210668 1040 tablet_service.cc:1505] Processing CreateTablet for tablet 5a60685d08b249bfbc80da8c2ac9b28f (DEFAULT_TABLE table=default.Table_c08aceb74c164c06a3a149c454c9c59e [id=0d6674bc4f264a36aad4ea7962a9e0e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.212227 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 92319d5707d14b029ce7ff1c24296662 (DEFAULT_TABLE table=default.Table_7f793bd143b642d98a4f147a032541d3 [id=fa472d30e4ec444aa660ddcc696dd93c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.212695 1054 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a81e1d10ca49459989260fbd6a2b8645" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:55.213234 859 leader_election.cc:343] T a81e1d10ca49459989260fbd6a2b8645 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:55.213696 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 6bce8dce844341f4982a44e1e90002e5 (DEFAULT_TABLE table=default.Table_3dcd1dea44ed4e54a89f4413e7f17dbe [id=f868f7fd84f247c89a963da591f16c2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.215422 1031 tablet_service.cc:1505] Processing CreateTablet for tablet e2fb216334a245e08617d18c836f1fbf (DEFAULT_TABLE table=default.Table_67001aa310ae44c0bb9e2b24bb0b57b8 [id=c17d62a148094547af24de85773a1b8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.215899 895 tablet_service.cc:1505] Processing CreateTablet for tablet d98b04c60e9845c3abb8a74c1ef9620b (DEFAULT_TABLE table=default.Table_c5b9c9e8e8d043bdbb55326857791c2a [id=01a0f786eaa94e4f8a46c239ba34c4b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.216586 894 tablet_service.cc:1505] Processing CreateTablet for tablet b83c65eb64d447feb462a24255141cb2 (DEFAULT_TABLE table=default.Table_5d59e3cf9d5a41afa503a76353543218 [id=0f35ce1317bf4b85b328e67c8911dc89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.216990 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b83c65eb64d447feb462a24255141cb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.217235 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 63c8ecaa42224d2ba0cc9cf26a05e698 (DEFAULT_TABLE table=default.Table_756b4deb63c645e79a6fbb8f8f131ee0 [id=ce752170017a4e119a8c42c73fd95302]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.219108 1033 tablet_service.cc:1505] Processing CreateTablet for tablet 6c654857b0e74e399b16601c9ebf692e (DEFAULT_TABLE table=default.Table_abd89cee70a04e2aa3a0d59b7eb943c0 [id=4c17da59a0554cee93ce4e626838a2c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.219830 1041 tablet_service.cc:1505] Processing CreateTablet for tablet 0883a26fce2540a5bc1c38e67b2f0dda (DEFAULT_TABLE table=default.Table_1b41581f01c94cc497e8af87b594b565 [id=ec4989eaa7f8482d8698a0e20d6e010f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.220183 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d98b04c60e9845c3abb8a74c1ef9620b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.220880 1182 log.cc:1199] T 25a407001f964da6b51f42375136c79a P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/25a407001f964da6b51f42375136c79a
I20260202 02:52:55.221210 1182 ts_tablet_manager.cc:1950] T 25a407001f964da6b51f42375136c79a P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:55.224959 884 tablet_service.cc:1505] Processing CreateTablet for tablet 39d9e3fef89d4c1cb053278e25796eaf (DEFAULT_TABLE table=default.Table_93c8b5e956214e88abc353bfb46c8757 [id=1cb84b11b2f34287bbade3af6cd4df67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.224998 885 tablet_service.cc:1505] Processing CreateTablet for tablet 862047d463374985ae0562f86ca2089d (DEFAULT_TABLE table=default.Table_28e7d9aa8131413e9a174756c246d164 [id=ec07c59893cf45c9b2a5eece556d10c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.225381 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39d9e3fef89d4c1cb053278e25796eaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.225615 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 862047d463374985ae0562f86ca2089d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.228569 3080 raft_consensus.cc:493] T 604b3b863c9540d3b55381dbfe1afdc3 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:55.228717 3080 raft_consensus.cc:515] T 604b3b863c9540d3b55381dbfe1afdc3 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.229307 3080 leader_election.cc:290] T 604b3b863c9540d3b55381dbfe1afdc3 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:55.232514 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 918) took 2293 ms. Trace:
I20260202 02:52:55.233613 1054 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "604b3b863c9540d3b55381dbfe1afdc3" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:55.233229 892 tablet_service.cc:1505] Processing CreateTablet for tablet 165320266c8e4a258372f225d0372daa (DEFAULT_TABLE table=default.Table_66adee49a1b045918360d13af6bca9a2 [id=4096475017214ffdba27d20aca4e8d64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.235682 900 tablet_service.cc:1505] Processing CreateTablet for tablet e2fb216334a245e08617d18c836f1fbf (DEFAULT_TABLE table=default.Table_67001aa310ae44c0bb9e2b24bb0b57b8 [id=c17d62a148094547af24de85773a1b8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.235067 884 tablet_service.cc:1505] Processing CreateTablet for tablet 92319d5707d14b029ce7ff1c24296662 (DEFAULT_TABLE table=default.Table_7f793bd143b642d98a4f147a032541d3 [id=fa472d30e4ec444aa660ddcc696dd93c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.235095 897 tablet_service.cc:1505] Processing CreateTablet for tablet 6bce8dce844341f4982a44e1e90002e5 (DEFAULT_TABLE table=default.Table_3dcd1dea44ed4e54a89f4413e7f17dbe [id=f868f7fd84f247c89a963da591f16c2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.235786 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 165320266c8e4a258372f225d0372daa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.236768 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bce8dce844341f4982a44e1e90002e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.236940 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92319d5707d14b029ce7ff1c24296662. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.237109 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2fb216334a245e08617d18c836f1fbf. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:55.234328 859 leader_election.cc:343] T 604b3b863c9540d3b55381dbfe1afdc3 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:55.244814 3080 raft_consensus.cc:493] T 67a8c9a1998b47b3b214a06ce62381e9 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260202 02:52:55.244882 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet 28f6161d0d0a4b41af4560f0f3bee0f7 (table default.Table_b5dcfcce852a48a0b1fbd3deed34f4d2 [id=a77fa0c30d27405294adaef676bc76f7]) was not created within the allowed timeout. Replacing with a new tablet bcee6b4b0b6f409790f3a5c30bbe85d5
I20260202 02:52:55.244958 3080 raft_consensus.cc:515] T 67a8c9a1998b47b3b214a06ce62381e9 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
W20260202 02:52:55.245476 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet 8d9ffdb5f10e4510888a90c49cd779bd (table default.Table_346e800aad794a65acc77ee0bfb0b82c [id=0dcc50bca61b416ca6d7f7b2d3da8582]) was not created within the allowed timeout. Replacing with a new tablet 0c55b4e75ae14b44a4955a3a236620a2
I20260202 02:52:55.245594 3080 leader_election.cc:290] T 67a8c9a1998b47b3b214a06ce62381e9 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
W20260202 02:52:55.245795 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet 5d2fc0a74684434d85b9922cb4e32fdc (table default.Table_ac6e92f3b1f540a7af72ae905e0481a7 [id=63269345f7e842dd9d4d250191e12c6e]) was not created within the allowed timeout. Replacing with a new tablet 67f279f55a2d42dc8cf5d62d392d3b68
W20260202 02:52:55.246060 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet e33f95e4f17a479c9acef5764c6c4c56 (table default.Table_1455877b4adc4c53a941d7fc145014b7 [id=5c2778a5ed864321bb59e445b4db671a]) was not created within the allowed timeout. Replacing with a new tablet ebdbfbea82494ba1b83daa5d93fa61c3
W20260202 02:52:55.246320 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet 4e90f273439d414caf16d63e1c605d3d (table default.Table_e11021f60bfe480ab07b2242bb36e94f [id=ae5d5815dcce405a897ffbce9d745591]) was not created within the allowed timeout. Replacing with a new tablet abd4d697cafa4e1a99fd37eb25f757dd
W20260202 02:52:55.246562 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet 455e7d1197c144eda1cf1ef405aeb262 (table default.Table_f4ed15aa08c44cb29cf820bbbac9c6df [id=d3fcc9cc54ba4938858eb27aa840f5d4]) was not created within the allowed timeout. Replacing with a new tablet 46e898063fb449dca45e69ab8045c3df
W20260202 02:52:55.246830 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet 7f227a89344b4aa6a62f6b58f679fbef (table default.Table_efacf68164014a2d8873859a3650a913 [id=baa4d4e86e9b475899c69cc47683f853]) was not created within the allowed timeout. Replacing with a new tablet 4e2fc5b9113447c193243ea028ce4956
W20260202 02:52:55.247071 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet 470f9ce5e38f4d22b4850e986a4d5c52 (table default.Table_e4bcea76324043f09b2a8d4cdec7f371 [id=4cb26dd223bc4f418094e3ec7c207998]) was not created within the allowed timeout. Replacing with a new tablet 6f03ed259a444605be1bab5125aed773
W20260202 02:52:55.247323 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet b34dc1cab5874fa6aee55d79c6e09ade (table default.Table_d376bfb6c4d8479b9a306b2f9c124fdd [id=2a2750072b914dc6b6f3adadd645206a]) was not created within the allowed timeout. Replacing with a new tablet 6ccb42ededd146a68dd7c9cc131fef58
W20260202 02:52:55.247550 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet bb145df67e1a44569a3dbc91c024b584 (table default.Table_8c1b93c2a1234d058590e574f89b770b [id=bf2ff444381f4d62a22258ffbfe1c040]) was not created within the allowed timeout. Replacing with a new tablet 8115d7ed07e94aa592ccbad4f096ba3b
W20260202 02:52:55.247813 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet 353bbad7f24f40e3b6316f619c749017 (table default.Table_44e1b09767d14dc899a9d956b5db9b02 [id=bb2dfdb752ee45aea4f8658346231390]) was not created within the allowed timeout. Replacing with a new tablet 737650f0494646ba822549abde78c846
W20260202 02:52:55.248031 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet 5ffdf97dff3e4dca89507f488c5b0f8e (table default.Table_18fec323d6b04e10a687de6361b643b2 [id=1ffcc11561524fe8a7b09a7ddc92d5b2]) was not created within the allowed timeout. Replacing with a new tablet 1a6720763aeb41fbbe2344b627994856
W20260202 02:52:55.248394 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet 916f52df8e0a4838b46772ecab4f852c (table default.Table_c70528658e2f49bcb97d3c9c815379ec [id=976c3ac7dde84abab2a5666eb46576e8]) was not created within the allowed timeout. Replacing with a new tablet e7937da857e14dd6b55c3b6f988dfa4d
I20260202 02:52:55.250170 3080 raft_consensus.cc:493] T 2f664f6119ba4d28b5ef1c212e611866 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:55.250337 3080 raft_consensus.cc:515] T 2f664f6119ba4d28b5ef1c212e611866 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.251021 3080 leader_election.cc:290] T 2f664f6119ba4d28b5ef1c212e611866 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:55.251820 1062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f664f6119ba4d28b5ef1c212e611866" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:55.252452 859 leader_election.cc:343] T 2f664f6119ba4d28b5ef1c212e611866 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:55.254601 885 tablet_service.cc:1505] Processing CreateTablet for tablet 5a60685d08b249bfbc80da8c2ac9b28f (DEFAULT_TABLE table=default.Table_c08aceb74c164c06a3a149c454c9c59e [id=0d6674bc4f264a36aad4ea7962a9e0e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.255082 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a60685d08b249bfbc80da8c2ac9b28f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.257459 1175 tablet_bootstrap.cc:492] T e95153ee8f2d40de955e3389d3a58644 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:55.257673 1175 ts_tablet_manager.cc:1403] T e95153ee8f2d40de955e3389d3a58644 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.080s user 0.003s sys 0.005s
I20260202 02:52:55.258510 1175 raft_consensus.cc:359] T e95153ee8f2d40de955e3389d3a58644 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.258766 1175 raft_consensus.cc:385] T e95153ee8f2d40de955e3389d3a58644 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:55.258855 1175 raft_consensus.cc:740] T e95153ee8f2d40de955e3389d3a58644 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:55.259117 1175 consensus_queue.cc:260] T e95153ee8f2d40de955e3389d3a58644 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.259763 1175 ts_tablet_manager.cc:1434] T e95153ee8f2d40de955e3389d3a58644 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:55.260089 1175 tablet_bootstrap.cc:492] T cef87d5f74f14a6d86150958de7b1ace P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:55.261914 1175 tablet_bootstrap.cc:654] T cef87d5f74f14a6d86150958de7b1ace P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:55.262184 1054 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "67a8c9a1998b47b3b214a06ce62381e9" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:55.267115 1173 tablet_bootstrap.cc:654] T 3dcf4f794ec14b2f9d181d015fc437fb P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:55.271090 3122 raft_consensus.cc:493] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:55.271266 3122 raft_consensus.cc:515] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:55.271919 3122 leader_election.cc:290] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:55.262887 859 leader_election.cc:343] T 67a8c9a1998b47b3b214a06ce62381e9 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:55.264222 896 tablet_service.cc:1505] Processing CreateTablet for tablet 54e596b4df1247b5b6ed270f92f9d389 (DEFAULT_TABLE table=default.Table_75aa5c9f446e4b6898acfc002ee4e4a9 [id=c600409762e44da6b398161b4829e932]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.275279 884 tablet_service.cc:1505] Processing CreateTablet for tablet 6c654857b0e74e399b16601c9ebf692e (DEFAULT_TABLE table=default.Table_abd89cee70a04e2aa3a0d59b7eb943c0 [id=4c17da59a0554cee93ce4e626838a2c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.275305 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54e596b4df1247b5b6ed270f92f9d389. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.275657 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c654857b0e74e399b16601c9ebf692e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.277928 902 tablet_service.cc:1505] Processing CreateTablet for tablet 4e348f6a09cf4e40b7debeafcbae83ee (DEFAULT_TABLE table=default.Table_619bb68d71834b3a9f5e0b2b0fdaed09 [id=616ea3074afc41bdae8f70c0d38bb591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.278306 902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e348f6a09cf4e40b7debeafcbae83ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.279281 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92319d5707d14b029ce7ff1c24296662. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.279503 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3bad993086e14d7b944f32b515508fc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.279901 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c654857b0e74e399b16601c9ebf692e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.280084 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bce8dce844341f4982a44e1e90002e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.280509 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2fb216334a245e08617d18c836f1fbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.280906 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63c8ecaa42224d2ba0cc9cf26a05e698. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.281309 1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0883a26fce2540a5bc1c38e67b2f0dda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.264693 900 tablet_service.cc:1505] Processing CreateTablet for tablet 63c8ecaa42224d2ba0cc9cf26a05e698 (DEFAULT_TABLE table=default.Table_756b4deb63c645e79a6fbb8f8f131ee0 [id=ce752170017a4e119a8c42c73fd95302]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.264730 901 tablet_service.cc:1505] Processing CreateTablet for tablet 44c10a6f003946f3971a4fcea535bc67 (DEFAULT_TABLE table=default.Table_c04e684df4804dfda34db628838a00c3 [id=7a4a9741f03548e49e4ef91e9073b747]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.282263 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44c10a6f003946f3971a4fcea535bc67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.284579 1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b83c65eb64d447feb462a24255141cb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.287699 1044 tablet_service.cc:1505] Processing CreateTablet for tablet cd2e710eebfe4a8f9ec20a24c1374356 (DEFAULT_TABLE table=default.Table_9584e635380e470f877ad287cbad051d [id=ed6c57a8715a49ffa4f6d1c99dca078e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.288115 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd2e710eebfe4a8f9ec20a24c1374356. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.288635 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a60685d08b249bfbc80da8c2ac9b28f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.289076 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63c8ecaa42224d2ba0cc9cf26a05e698. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.292356 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07dd6e4d34b94162bba0eb2a4ca84708. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.296501 1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d98b04c60e9845c3abb8a74c1ef9620b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.297084 1175 tablet_bootstrap.cc:492] T cef87d5f74f14a6d86150958de7b1ace P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:55.297268 1175 ts_tablet_manager.cc:1403] T cef87d5f74f14a6d86150958de7b1ace P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.037s user 0.005s sys 0.000s
I20260202 02:52:55.298044 1175 raft_consensus.cc:359] T cef87d5f74f14a6d86150958de7b1ace P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.298310 1175 raft_consensus.cc:385] T cef87d5f74f14a6d86150958de7b1ace P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:55.298408 1175 raft_consensus.cc:740] T cef87d5f74f14a6d86150958de7b1ace P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:55.298578 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 4e348f6a09cf4e40b7debeafcbae83ee (DEFAULT_TABLE table=default.Table_619bb68d71834b3a9f5e0b2b0fdaed09 [id=616ea3074afc41bdae8f70c0d38bb591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.298731 1175 consensus_queue.cc:260] T cef87d5f74f14a6d86150958de7b1ace P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.299245 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e348f6a09cf4e40b7debeafcbae83ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.299454 1175 ts_tablet_manager.cc:1434] T cef87d5f74f14a6d86150958de7b1ace P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:55.299793 1175 tablet_bootstrap.cc:492] T 9f11a68d10374d06bc4a756a1d15bf09 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:55.300572 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da9d3228ad734065bfc640833c988c92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.304484 1175 tablet_bootstrap.cc:654] T 9f11a68d10374d06bc4a756a1d15bf09 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:55.304517 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39d9e3fef89d4c1cb053278e25796eaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.308086 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f89c5ad8995044d7a5f6026c1a047fba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.311793 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8606fb177db49c7b2e2b16b18d4725c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.313617 1175 tablet_bootstrap.cc:492] T 9f11a68d10374d06bc4a756a1d15bf09 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:55.313875 1175 ts_tablet_manager.cc:1403] T 9f11a68d10374d06bc4a756a1d15bf09 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.009s
I20260202 02:52:55.314637 1175 raft_consensus.cc:359] T 9f11a68d10374d06bc4a756a1d15bf09 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.314888 1175 raft_consensus.cc:385] T 9f11a68d10374d06bc4a756a1d15bf09 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:55.314989 1175 raft_consensus.cc:740] T 9f11a68d10374d06bc4a756a1d15bf09 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:55.315035 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c1e408a75fe4d948a355f0dede46316. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.315287 1175 consensus_queue.cc:260] T 9f11a68d10374d06bc4a756a1d15bf09 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.318125 1175 ts_tablet_manager.cc:1434] T 9f11a68d10374d06bc4a756a1d15bf09 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260202 02:52:55.318317 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecd7fccb38c64f73a08a7af4210e1982. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.318558 1175 tablet_bootstrap.cc:492] T 3338964dfed548f980f58147f0283691 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:55.319512 3080 raft_consensus.cc:493] T e95153ee8f2d40de955e3389d3a58644 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:55.319651 3080 raft_consensus.cc:515] T e95153ee8f2d40de955e3389d3a58644 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.320248 3080 leader_election.cc:290] T e95153ee8f2d40de955e3389d3a58644 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:55.321419 1054 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e95153ee8f2d40de955e3389d3a58644" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:55.321615 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 862047d463374985ae0562f86ca2089d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:55.322136 859 leader_election.cc:343] T e95153ee8f2d40de955e3389d3a58644 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:55.323524 3080 raft_consensus.cc:493] T 9f11a68d10374d06bc4a756a1d15bf09 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:55.323664 3080 raft_consensus.cc:515] T 9f11a68d10374d06bc4a756a1d15bf09 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.324257 3080 leader_election.cc:290] T 9f11a68d10374d06bc4a756a1d15bf09 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:55.324458 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 165320266c8e4a258372f225d0372daa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.325016 1054 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f11a68d10374d06bc4a756a1d15bf09" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:55.325829 859 leader_election.cc:343] T 9f11a68d10374d06bc4a756a1d15bf09 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:55.328428 3080 raft_consensus.cc:493] T dc4c3f52f4f44a84b0d0e3c9a4a49f09 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:55.328581 3080 raft_consensus.cc:515] T dc4c3f52f4f44a84b0d0e3c9a4a49f09 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.329182 3080 leader_election.cc:290] T dc4c3f52f4f44a84b0d0e3c9a4a49f09 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:55.329900 1054 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dc4c3f52f4f44a84b0d0e3c9a4a49f09" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:55.330638 859 leader_election.cc:343] T dc4c3f52f4f44a84b0d0e3c9a4a49f09 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:55.332162 1175 tablet_bootstrap.cc:654] T 3338964dfed548f980f58147f0283691 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:55.339124 515 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 0883a26fce2540a5bc1c38e67b2f0dda (table default.Table_1b41581f01c94cc497e8af87b594b565 [id=ec4989eaa7f8482d8698a0e20d6e010f], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 0883a26fce2540a5bc1c38e67b2f0dda consensus state has no leader
I20260202 02:52:55.342761 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 28f6161d0d0a4b41af4560f0f3bee0f7
I20260202 02:52:55.343412 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 8d9ffdb5f10e4510888a90c49cd779bd
I20260202 02:52:55.343820 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 5d2fc0a74684434d85b9922cb4e32fdc
I20260202 02:52:55.343807 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 28f6161d0d0a4b41af4560f0f3bee0f7 with delete_type TABLET_DATA_DELETED (Replaced by bcee6b4b0b6f409790f3a5c30bbe85d5 at 2026-02-02 02:52:55 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:55.344183 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet e33f95e4f17a479c9acef5764c6c4c56
I20260202 02:52:55.344233 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 8d9ffdb5f10e4510888a90c49cd779bd with delete_type TABLET_DATA_DELETED (Replaced by 0c55b4e75ae14b44a4955a3a236620a2 at 2026-02-02 02:52:55 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:55.344628 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 5d2fc0a74684434d85b9922cb4e32fdc with delete_type TABLET_DATA_DELETED (Replaced by 67f279f55a2d42dc8cf5d62d392d3b68 at 2026-02-02 02:52:55 UTC) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:55.345749 999 consensus_peers.cc:597] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dc90eeb79ded4739883ad98928476680. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:55.354350 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 3a4534ce9acb4e0ebb202e7695bbf4d5 (DEFAULT_TABLE table=default.Table_3db4721b7f3d4b09aafe0f29ebb2039f [id=9ae42d61d911493caaa622f688e6c9c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.354856 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a4534ce9acb4e0ebb202e7695bbf4d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.359122 1175 tablet_bootstrap.cc:492] T 3338964dfed548f980f58147f0283691 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:55.359212 1036 tablet_service.cc:1505] Processing CreateTablet for tablet d1de8eae9d13447d89e82208ce3494f7 (DEFAULT_TABLE table=default.Table_45815470ea1c49a4bb2c2c240a9d7b4e [id=fc314b0a5f8f47c88de41b851de71fff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.359423 1175 ts_tablet_manager.cc:1403] T 3338964dfed548f980f58147f0283691 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.041s user 0.016s sys 0.008s
I20260202 02:52:55.359630 1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1de8eae9d13447d89e82208ce3494f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.360244 1175 raft_consensus.cc:359] T 3338964dfed548f980f58147f0283691 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.360733 1175 raft_consensus.cc:385] T 3338964dfed548f980f58147f0283691 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:55.360839 1175 raft_consensus.cc:740] T 3338964dfed548f980f58147f0283691 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:55.361155 1175 consensus_queue.cc:260] T 3338964dfed548f980f58147f0283691 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.365124 1175 ts_tablet_manager.cc:1434] T 3338964dfed548f980f58147f0283691 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260202 02:52:55.365206 1044 tablet_service.cc:1505] Processing CreateTablet for tablet 44c10a6f003946f3971a4fcea535bc67 (DEFAULT_TABLE table=default.Table_c04e684df4804dfda34db628838a00c3 [id=7a4a9741f03548e49e4ef91e9073b747]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.365564 1175 tablet_bootstrap.cc:492] T 916f52df8e0a4838b46772ecab4f852c P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:55.365615 1040 tablet_service.cc:1505] Processing CreateTablet for tablet 9d548ecfbeff4c3ba2981d656b4fa92a (DEFAULT_TABLE table=default.Table_6da2058f0c8c44feb77c1106bda2bffe [id=2ae99f38f2464e099ac578796258f825]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.365736 1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44c10a6f003946f3971a4fcea535bc67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.365968 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d548ecfbeff4c3ba2981d656b4fa92a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.368170 3080 raft_consensus.cc:493] T cef87d5f74f14a6d86150958de7b1ace P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:55.368315 3080 raft_consensus.cc:515] T cef87d5f74f14a6d86150958de7b1ace P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.368960 3080 leader_election.cc:290] T cef87d5f74f14a6d86150958de7b1ace P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:55.369306 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet 28f6161d0d0a4b41af4560f0f3bee0f7 with delete_type TABLET_DATA_DELETED (Replaced by bcee6b4b0b6f409790f3a5c30bbe85d5 at 2026-02-02 02:52:55 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:55.369510 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet 8d9ffdb5f10e4510888a90c49cd779bd with delete_type TABLET_DATA_DELETED (Replaced by 0c55b4e75ae14b44a4955a3a236620a2 at 2026-02-02 02:52:55 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:55.369601 1054 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cef87d5f74f14a6d86150958de7b1ace" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:55.369697 1041 tablet_service.cc:1552] Processing DeleteTablet for tablet 5d2fc0a74684434d85b9922cb4e32fdc with delete_type TABLET_DATA_DELETED (Replaced by 67f279f55a2d42dc8cf5d62d392d3b68 at 2026-02-02 02:52:55 UTC) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:55.370127 859 leader_election.cc:343] T cef87d5f74f14a6d86150958de7b1ace P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:55.372274 1175 tablet_bootstrap.cc:654] T 916f52df8e0a4838b46772ecab4f852c P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:55.373787 1034 tablet_service.cc:1505] Processing CreateTablet for tablet 54e596b4df1247b5b6ed270f92f9d389 (DEFAULT_TABLE table=default.Table_75aa5c9f446e4b6898acfc002ee4e4a9 [id=c600409762e44da6b398161b4829e932]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.374249 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54e596b4df1247b5b6ed270f92f9d389. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.380760 1175 tablet_bootstrap.cc:492] T 916f52df8e0a4838b46772ecab4f852c P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:55.381003 1175 ts_tablet_manager.cc:1403] T 916f52df8e0a4838b46772ecab4f852c P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.006s
I20260202 02:52:55.381961 1175 raft_consensus.cc:359] T 916f52df8e0a4838b46772ecab4f852c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.382306 1175 raft_consensus.cc:385] T 916f52df8e0a4838b46772ecab4f852c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:55.382405 1175 raft_consensus.cc:740] T 916f52df8e0a4838b46772ecab4f852c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:55.382732 1175 consensus_queue.cc:260] T 916f52df8e0a4838b46772ecab4f852c P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.264989 897 tablet_service.cc:1505] Processing CreateTablet for tablet 0883a26fce2540a5bc1c38e67b2f0dda (DEFAULT_TABLE table=default.Table_1b41581f01c94cc497e8af87b594b565 [id=ec4989eaa7f8482d8698a0e20d6e010f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.265133 889 tablet_service.cc:1505] Processing CreateTablet for tablet 9d548ecfbeff4c3ba2981d656b4fa92a (DEFAULT_TABLE table=default.Table_6da2058f0c8c44feb77c1106bda2bffe [id=2ae99f38f2464e099ac578796258f825]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.386878 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d548ecfbeff4c3ba2981d656b4fa92a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.264204 886 tablet_service.cc:1505] Processing CreateTablet for tablet d1de8eae9d13447d89e82208ce3494f7 (DEFAULT_TABLE table=default.Table_45815470ea1c49a4bb2c2c240a9d7b4e [id=fc314b0a5f8f47c88de41b851de71fff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.388305 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1de8eae9d13447d89e82208ce3494f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.394515 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0883a26fce2540a5bc1c38e67b2f0dda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.401486 1175 ts_tablet_manager.cc:1434] T 916f52df8e0a4838b46772ecab4f852c P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.020s user 0.000s sys 0.002s
I20260202 02:52:55.404249 1175 tablet_bootstrap.cc:492] T 42321f4e77484027a485fdd584de3ae3 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:55.405784 1175 tablet_bootstrap.cc:654] T 42321f4e77484027a485fdd584de3ae3 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:55.235947 1182 rpcz_store.cc:276] 0202 02:52:52.938533 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.938581 (+ 48us) service_pool.cc:225] Handling call
0202 02:52:55.151069 (+2212488us) ts_tablet_manager.cc:1151] Deleting tablet 25a407001f964da6b51f42375136c79a
0202 02:52:55.151080 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.167673 (+ 16593us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.172885 (+ 5212us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.232478 (+ 59593us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2212288}
I20260202 02:52:55.272437 918 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4a518e8ad2e34ec98a0425c7ed3a470a" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5" is_pre_election: true
I20260202 02:52:55.273804 890 tablet_service.cc:1505] Processing CreateTablet for tablet cd2e710eebfe4a8f9ec20a24c1374356 (DEFAULT_TABLE table=default.Table_9584e635380e470f877ad287cbad051d [id=ed6c57a8715a49ffa4f6d1c99dca078e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.412657 918 raft_consensus.cc:2468] T 4a518e8ad2e34ec98a0425c7ed3a470a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 0.
I20260202 02:52:55.412901 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd2e710eebfe4a8f9ec20a24c1374356. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.413288 999 leader_election.cc:304] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:55.413578 3122 raft_consensus.cc:2804] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:52:55.413645 3122 raft_consensus.cc:493] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:52:55.413703 3122 raft_consensus.cc:3060] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:55.414014 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 4e90f273439d414caf16d63e1c605d3d
I20260202 02:52:55.414531 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 455e7d1197c144eda1cf1ef405aeb262
I20260202 02:52:55.414974 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 7f227a89344b4aa6a62f6b58f679fbef
I20260202 02:52:55.415450 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 470f9ce5e38f4d22b4850e986a4d5c52
I20260202 02:52:55.415891 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet b34dc1cab5874fa6aee55d79c6e09ade
I20260202 02:52:55.416114 3122 raft_consensus.cc:515] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:55.416422 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet bb145df67e1a44569a3dbc91c024b584
I20260202 02:52:55.416945 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 353bbad7f24f40e3b6316f619c749017
I20260202 02:52:55.417331 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 4e90f273439d414caf16d63e1c605d3d with delete_type TABLET_DATA_DELETED (Replaced by abd4d697cafa4e1a99fd37eb25f757dd at 2026-02-02 02:52:55 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:55.417521 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 5ffdf97dff3e4dca89507f488c5b0f8e
I20260202 02:52:55.417572 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 7f227a89344b4aa6a62f6b58f679fbef with delete_type TABLET_DATA_DELETED (Replaced by 4e2fc5b9113447c193243ea028ce4956 at 2026-02-02 02:52:55 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:55.417711 902 tablet_service.cc:1552] Processing DeleteTablet for tablet 470f9ce5e38f4d22b4850e986a4d5c52 with delete_type TABLET_DATA_DELETED (Replaced by 6f03ed259a444605be1bab5125aed773 at 2026-02-02 02:52:55 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:55.417759 900 tablet_service.cc:1552] Processing DeleteTablet for tablet bb145df67e1a44569a3dbc91c024b584 with delete_type TABLET_DATA_DELETED (Replaced by 8115d7ed07e94aa592ccbad4f096ba3b at 2026-02-02 02:52:55 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:55.417886 886 tablet_service.cc:1552] Processing DeleteTablet for tablet b34dc1cab5874fa6aee55d79c6e09ade with delete_type TABLET_DATA_DELETED (Replaced by 6ccb42ededd146a68dd7c9cc131fef58 at 2026-02-02 02:52:55 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:55.418105 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 916f52df8e0a4838b46772ecab4f852c
I20260202 02:52:55.418385 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 353bbad7f24f40e3b6316f619c749017 with delete_type TABLET_DATA_DELETED (Replaced by 737650f0494646ba822549abde78c846 at 2026-02-02 02:52:55 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:55.418398 900 tablet_service.cc:1552] Processing DeleteTablet for tablet 5ffdf97dff3e4dca89507f488c5b0f8e with delete_type TABLET_DATA_DELETED (Replaced by 1a6720763aeb41fbbe2344b627994856 at 2026-02-02 02:52:55 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:55.418936 900 tablet_service.cc:1552] Processing DeleteTablet for tablet 916f52df8e0a4838b46772ecab4f852c with delete_type TABLET_DATA_DELETED (Replaced by e7937da857e14dd6b55c3b6f988dfa4d at 2026-02-02 02:52:55 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:55.419736 3122 leader_election.cc:290] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Requested vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:55.420720 918 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4a518e8ad2e34ec98a0425c7ed3a470a" candidate_uuid: "b1f05820b6034228bdad884cc348716d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1521a6d262b7451bbef8debe970dd6b5"
I20260202 02:52:55.420904 918 raft_consensus.cc:3060] T 4a518e8ad2e34ec98a0425c7ed3a470a P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:52:55.423475 3080 raft_consensus.cc:493] T f6c2bc531d194a4b8f3062c9b40e3da6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:55.423615 3080 raft_consensus.cc:515] T f6c2bc531d194a4b8f3062c9b40e3da6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.424238 3080 leader_election.cc:290] T f6c2bc531d194a4b8f3062c9b40e3da6 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:55.425175 918 raft_consensus.cc:2468] T 4a518e8ad2e34ec98a0425c7ed3a470a P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f05820b6034228bdad884cc348716d in term 1.
I20260202 02:52:55.425737 900 tablet_service.cc:1505] Processing CreateTablet for tablet bcee6b4b0b6f409790f3a5c30bbe85d5 (DEFAULT_TABLE table=default.Table_b5dcfcce852a48a0b1fbd3deed34f4d2 [id=a77fa0c30d27405294adaef676bc76f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.426136 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcee6b4b0b6f409790f3a5c30bbe85d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.269065 892 tablet_service.cc:1505] Processing CreateTablet for tablet 3a4534ce9acb4e0ebb202e7695bbf4d5 (DEFAULT_TABLE table=default.Table_3db4721b7f3d4b09aafe0f29ebb2039f [id=9ae42d61d911493caaa622f688e6c9c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.428431 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a4534ce9acb4e0ebb202e7695bbf4d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.429454 999 leader_election.cc:304] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5, b1f05820b6034228bdad884cc348716d; no voters:
I20260202 02:52:55.429597 890 tablet_service.cc:1505] Processing CreateTablet for tablet 4d37d9e85b7345379e6bbeaa8cabd182 (DEFAULT_TABLE table=default.Table_910b1d34ecd54cd38bff82420fe1a154 [id=2d8587df1c174dfd8a4b2a4d053c7821]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.430040 3122 raft_consensus.cc:2804] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:52:55.430132 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d37d9e85b7345379e6bbeaa8cabd182. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.430531 3122 raft_consensus.cc:697] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Becoming Leader. State: Replica: b1f05820b6034228bdad884cc348716d, State: Running, Role: LEADER
I20260202 02:52:55.430801 3122 consensus_queue.cc:237] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:55.436344 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 455e7d1197c144eda1cf1ef405aeb262 with delete_type TABLET_DATA_DELETED (Replaced by 46e898063fb449dca45e69ab8045c3df at 2026-02-02 02:52:55 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:55.437288 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 25a407001f964da6b51f42375136c79a (table default.Table_48226e7c77da41cdba56e04221eb9689 [id=05e52ce8e94e4975b51849059b904272]) successfully deleted
I20260202 02:52:55.437673 1182 tablet_replica.cc:333] T e2edc6660b9a41e39e9529627fafcb03 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:55.438230 1182 raft_consensus.cc:2243] T e2edc6660b9a41e39e9529627fafcb03 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:55.438498 1182 raft_consensus.cc:2272] T e2edc6660b9a41e39e9529627fafcb03 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:55.439121 1182 ts_tablet_manager.cc:1916] T e2edc6660b9a41e39e9529627fafcb03 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:55.443306 1882 ts_tablet_manager.cc:1929] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.7
I20260202 02:52:55.443581 1882 log.cc:1199] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/63a9029bc39c4fbd8ac4bf4bc37d6b4e
I20260202 02:52:55.444031 1882 ts_tablet_manager.cc:1950] T 63a9029bc39c4fbd8ac4bf4bc37d6b4e P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:55.446291 894 tablet_service.cc:1552] Processing DeleteTablet for tablet e33f95e4f17a479c9acef5764c6c4c56 with delete_type TABLET_DATA_DELETED (Replaced by ebdbfbea82494ba1b83daa5d93fa61c3 at 2026-02-02 02:52:55 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:55.447355 1175 tablet_bootstrap.cc:492] T 42321f4e77484027a485fdd584de3ae3 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:55.447816 1175 ts_tablet_manager.cc:1403] T 42321f4e77484027a485fdd584de3ae3 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.044s user 0.000s sys 0.003s
I20260202 02:52:55.448575 1175 raft_consensus.cc:359] T 42321f4e77484027a485fdd584de3ae3 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.448810 1175 raft_consensus.cc:385] T 42321f4e77484027a485fdd584de3ae3 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:55.448896 1175 raft_consensus.cc:740] T 42321f4e77484027a485fdd584de3ae3 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:55.449152 1175 consensus_queue.cc:260] T 42321f4e77484027a485fdd584de3ae3 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.451525 1175 ts_tablet_manager.cc:1434] T 42321f4e77484027a485fdd584de3ae3 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260202 02:52:55.451867 1175 tablet_bootstrap.cc:492] T 23671408fcbc454cbd918328015cf6e6 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:55.453373 1175 tablet_bootstrap.cc:654] T 23671408fcbc454cbd918328015cf6e6 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:55.457978 1182 ts_tablet_manager.cc:1929] T e2edc6660b9a41e39e9529627fafcb03 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:55.458175 1182 log.cc:1199] T e2edc6660b9a41e39e9529627fafcb03 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/e2edc6660b9a41e39e9529627fafcb03
I20260202 02:52:55.458545 1182 ts_tablet_manager.cc:1950] T e2edc6660b9a41e39e9529627fafcb03 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:55.459322 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 919) took 2520 ms. Trace:
I20260202 02:52:55.459475 1182 rpcz_store.cc:276] 0202 02:52:52.938612 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.938775 (+ 163us) service_pool.cc:225] Handling call
0202 02:52:55.437620 (+2498845us) ts_tablet_manager.cc:1151] Deleting tablet e2edc6660b9a41e39e9529627fafcb03
0202 02:52:55.437631 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.442147 (+ 4516us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.457892 (+ 15745us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.459302 (+ 1410us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2498702}
I20260202 02:52:55.460301 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet e2edc6660b9a41e39e9529627fafcb03 (table default.Table_9d4fbf619567432a88aa9a0c1b19cb3e [id=fff85ebd1bd1415c932f7c9c5a0da42a]) successfully deleted
I20260202 02:52:55.463601 898 tablet_service.cc:1505] Processing CreateTablet for tablet 3cb014efbe9e46eab97e71fff0ff2887 (DEFAULT_TABLE table=default.Table_e5f4ac82d1ef4b85999695997eb9df2a [id=c97e407e519d4a538860f66931f6aefd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.464236 1054 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f6c2bc531d194a4b8f3062c9b40e3da6" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
W20260202 02:52:55.468977 859 leader_election.cc:343] T f6c2bc531d194a4b8f3062c9b40e3da6 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): Illegal state: must be running to vote when last-logged opid is not known
I20260202 02:52:55.479024 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cb014efbe9e46eab97e71fff0ff2887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.483358 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 801) took 5539 ms. Trace:
I20260202 02:52:55.483575 1882 rpcz_store.cc:276] 0202 02:52:49.944361 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:49.944470 (+ 109us) service_pool.cc:225] Handling call
0202 02:52:55.031450 (+5086980us) ts_tablet_manager.cc:1151] Deleting tablet 63a9029bc39c4fbd8ac4bf4bc37d6b4e
0202 02:52:55.031462 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.440510 (+409048us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.443207 (+ 2697us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.483333 (+ 40126us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5086753}
I20260202 02:52:55.485003 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 802) took 5454 ms. Trace:
I20260202 02:52:55.485149 1882 rpcz_store.cc:276] 0202 02:52:50.030712 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:50.030826 (+ 114us) service_pool.cc:225] Handling call
0202 02:52:55.484865 (+5454039us) ts_tablet_manager.cc:1151] Deleting tablet a81e1d10ca49459989260fbd6a2b8645
0202 02:52:55.484875 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.484987 (+ 112us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5453805}
I20260202 02:52:55.484668 1173 tablet_bootstrap.cc:492] T 3dcf4f794ec14b2f9d181d015fc437fb P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:55.485594 1173 ts_tablet_manager.cc:1403] T 3dcf4f794ec14b2f9d181d015fc437fb P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.380s user 0.003s sys 0.000s
I20260202 02:52:55.486421 1173 raft_consensus.cc:359] T 3dcf4f794ec14b2f9d181d015fc437fb P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.486701 1173 raft_consensus.cc:385] T 3dcf4f794ec14b2f9d181d015fc437fb P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:55.486805 1173 raft_consensus.cc:740] T 3dcf4f794ec14b2f9d181d015fc437fb P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:55.487038 1173 consensus_queue.cc:260] T 3dcf4f794ec14b2f9d181d015fc437fb P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.487588 1173 ts_tablet_manager.cc:1434] T 3dcf4f794ec14b2f9d181d015fc437fb P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:55.487687 897 tablet_service.cc:1552] Processing DeleteTablet for tablet 42321f4e77484027a485fdd584de3ae3 with delete_type TABLET_DATA_DELETED (replaced by tablet fea0fcb8854c4fc6acda80c8ffb8d341) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:55.488878 894 tablet_service.cc:1552] Processing DeleteTablet for tablet 3338964dfed548f980f58147f0283691 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:48 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:55.489113 895 tablet_service.cc:1552] Processing DeleteTablet for tablet e95153ee8f2d40de955e3389d3a58644 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:55.489537 894 tablet_service.cc:1505] Processing CreateTablet for tablet 67f279f55a2d42dc8cf5d62d392d3b68 (DEFAULT_TABLE table=default.Table_ac6e92f3b1f540a7af72ae905e0481a7 [id=63269345f7e842dd9d4d250191e12c6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.489851 897 tablet_service.cc:1505] Processing CreateTablet for tablet 0c55b4e75ae14b44a4955a3a236620a2 (DEFAULT_TABLE table=default.Table_346e800aad794a65acc77ee0bfb0b82c [id=0dcc50bca61b416ca6d7f7b2d3da8582]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.490201 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67f279f55a2d42dc8cf5d62d392d3b68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.490310 895 tablet_service.cc:1505] Processing CreateTablet for tablet ebdbfbea82494ba1b83daa5d93fa61c3 (DEFAULT_TABLE table=default.Table_1455877b4adc4c53a941d7fc145014b7 [id=5c2778a5ed864321bb59e445b4db671a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.490844 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebdbfbea82494ba1b83daa5d93fa61c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.491422 1173 tablet_bootstrap.cc:492] T b9fc33e8efc04734a951f7fcf9048142 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:55.492379 885 tablet_service.cc:1505] Processing CreateTablet for tablet abd4d697cafa4e1a99fd37eb25f757dd (DEFAULT_TABLE table=default.Table_e11021f60bfe480ab07b2242bb36e94f [id=ae5d5815dcce405a897ffbce9d745591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.492780 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd4d697cafa4e1a99fd37eb25f757dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.492879 1173 tablet_bootstrap.cc:654] T b9fc33e8efc04734a951f7fcf9048142 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:55.496824 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet e33f95e4f17a479c9acef5764c6c4c56 with delete_type TABLET_DATA_DELETED (Replaced by ebdbfbea82494ba1b83daa5d93fa61c3 at 2026-02-02 02:52:55 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:55.496948 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 63a9029bc39c4fbd8ac4bf4bc37d6b4e (table default.Table_4062ed5efe774bc2b380c3ae7fa53abe [id=fa95e04db2d443b5972de98c7c0e6457]) successfully deleted
I20260202 02:52:55.497179 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet 4e90f273439d414caf16d63e1c605d3d with delete_type TABLET_DATA_DELETED (Replaced by abd4d697cafa4e1a99fd37eb25f757dd at 2026-02-02 02:52:55 UTC) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:55.497210 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet a81e1d10ca49459989260fbd6a2b8645 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a81e1d10ca49459989260fbd6a2b8645 already in progress: creating tablet
I20260202 02:52:55.497583 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c55b4e75ae14b44a4955a3a236620a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.497735 1882 tablet_replica.cc:333] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
W20260202 02:52:55.498085 999 consensus_peers.cc:597] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 59ad016e15ec4a488921eb564029b689. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:55.498112 1882 raft_consensus.cc:2243] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [term 3 LEADER]: Raft consensus shutting down.
I20260202 02:52:55.498440 1882 raft_consensus.cc:2272] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:55.499118 1882 ts_tablet_manager.cc:1916] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:55.504806 418 external_mini_cluster.cc:1715] Unable to check leaks on /tmp/dist-test-taskYXxklX/build/asan/bin/kudu pid 696: Timed out: CheckLeaks RPC to 127.0.104.129:39403 timed out after 1.000s (SENT)
I20260202 02:52:55.505004 418 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskYXxklX/build/asan/bin/kudu with pid 696
I20260202 02:52:55.508533 1173 tablet_bootstrap.cc:492] T b9fc33e8efc04734a951f7fcf9048142 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:55.508770 1173 ts_tablet_manager.cc:1403] T b9fc33e8efc04734a951f7fcf9048142 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.000s
I20260202 02:52:55.509488 1173 raft_consensus.cc:359] T b9fc33e8efc04734a951f7fcf9048142 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:55.510762 1173 raft_consensus.cc:385] T b9fc33e8efc04734a951f7fcf9048142 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:55.511215 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet 455e7d1197c144eda1cf1ef405aeb262 with delete_type TABLET_DATA_DELETED (Replaced by 46e898063fb449dca45e69ab8045c3df at 2026-02-02 02:52:55 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:55.511792 1173 raft_consensus.cc:740] T b9fc33e8efc04734a951f7fcf9048142 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:55.511812 885 tablet_service.cc:1505] Processing CreateTablet for tablet 46e898063fb449dca45e69ab8045c3df (DEFAULT_TABLE table=default.Table_f4ed15aa08c44cb29cf820bbbac9c6df [id=d3fcc9cc54ba4938858eb27aa840f5d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.512238 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46e898063fb449dca45e69ab8045c3df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.513149 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet 7f227a89344b4aa6a62f6b58f679fbef with delete_type TABLET_DATA_DELETED (Replaced by 4e2fc5b9113447c193243ea028ce4956 at 2026-02-02 02:52:55 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:55.513680 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet 470f9ce5e38f4d22b4850e986a4d5c52 with delete_type TABLET_DATA_DELETED (Replaced by 6f03ed259a444605be1bab5125aed773 at 2026-02-02 02:52:55 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:55.514029 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet b34dc1cab5874fa6aee55d79c6e09ade with delete_type TABLET_DATA_DELETED (Replaced by 6ccb42ededd146a68dd7c9cc131fef58 at 2026-02-02 02:52:55 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:55.516629 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet bb145df67e1a44569a3dbc91c024b584 with delete_type TABLET_DATA_DELETED (Replaced by 8115d7ed07e94aa592ccbad4f096ba3b at 2026-02-02 02:52:55 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:55.521346 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet 353bbad7f24f40e3b6316f619c749017 with delete_type TABLET_DATA_DELETED (Replaced by 737650f0494646ba822549abde78c846 at 2026-02-02 02:52:55 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:55.522384 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet 5ffdf97dff3e4dca89507f488c5b0f8e with delete_type TABLET_DATA_DELETED (Replaced by 1a6720763aeb41fbbe2344b627994856 at 2026-02-02 02:52:55 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:55.522810 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet 916f52df8e0a4838b46772ecab4f852c with delete_type TABLET_DATA_DELETED (Replaced by e7937da857e14dd6b55c3b6f988dfa4d at 2026-02-02 02:52:55 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:55.523538 1036 tablet_service.cc:1505] Processing CreateTablet for tablet bcee6b4b0b6f409790f3a5c30bbe85d5 (DEFAULT_TABLE table=default.Table_b5dcfcce852a48a0b1fbd3deed34f4d2 [id=a77fa0c30d27405294adaef676bc76f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.523993 1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcee6b4b0b6f409790f3a5c30bbe85d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.528663 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 915) took 2590 ms. Trace:
I20260202 02:52:55.528864 1182 rpcz_store.cc:276] 0202 02:52:52.938242 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.938838 (+ 596us) service_pool.cc:225] Handling call
0202 02:52:55.460793 (+2521955us) ts_tablet_manager.cc:1151] Deleting tablet 9ee7687df15d494eaf4150db580de7e3
0202 02:52:55.460804 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.528636 (+ 67832us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2521178}
W20260202 02:52:55.529587 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 9ee7687df15d494eaf4150db580de7e3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9ee7687df15d494eaf4150db580de7e3 already in progress: creating tablet
I20260202 02:52:55.529578 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 4d37d9e85b7345379e6bbeaa8cabd182 (DEFAULT_TABLE table=default.Table_910b1d34ecd54cd38bff82420fe1a154 [id=2d8587df1c174dfd8a4b2a4d053c7821]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.529918 1182 tablet_replica.cc:333] T 51ba0bcd498043c383c493f45ad1fe17 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:55.530011 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d37d9e85b7345379e6bbeaa8cabd182. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.530148 1182 raft_consensus.cc:2243] T 51ba0bcd498043c383c493f45ad1fe17 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:55.530359 1182 raft_consensus.cc:2272] T 51ba0bcd498043c383c493f45ad1fe17 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:55.512470 1173 consensus_queue.cc:260] T b9fc33e8efc04734a951f7fcf9048142 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:55.531388 1173 ts_tablet_manager.cc:1434] T b9fc33e8efc04734a951f7fcf9048142 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.022s user 0.001s sys 0.000s
I20260202 02:52:55.532462 1175 tablet_bootstrap.cc:492] T 23671408fcbc454cbd918328015cf6e6 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:55.533051 1173 tablet_bootstrap.cc:492] T 09504070c9ec4bcfbb59e3fb4c189775 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:55.534565 1173 tablet_bootstrap.cc:654] T 09504070c9ec4bcfbb59e3fb4c189775 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:55.533326 899 tablet_service.cc:1505] Processing CreateTablet for tablet 4e2fc5b9113447c193243ea028ce4956 (DEFAULT_TABLE table=default.Table_efacf68164014a2d8873859a3650a913 [id=baa4d4e86e9b475899c69cc47683f853]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.536705 1175 ts_tablet_manager.cc:1403] T 23671408fcbc454cbd918328015cf6e6 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.085s user 0.002s sys 0.001s
I20260202 02:52:55.536859 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e2fc5b9113447c193243ea028ce4956. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.537499 1175 raft_consensus.cc:359] T 23671408fcbc454cbd918328015cf6e6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.537736 1175 raft_consensus.cc:385] T 23671408fcbc454cbd918328015cf6e6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:55.537822 1175 raft_consensus.cc:740] T 23671408fcbc454cbd918328015cf6e6 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:55.538069 1175 consensus_queue.cc:260] T 23671408fcbc454cbd918328015cf6e6 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.538596 1175 ts_tablet_manager.cc:1434] T 23671408fcbc454cbd918328015cf6e6 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:55.540263 894 tablet_service.cc:1505] Processing CreateTablet for tablet 6f03ed259a444605be1bab5125aed773 (DEFAULT_TABLE table=default.Table_e4bcea76324043f09b2a8d4cdec7f371 [id=4cb26dd223bc4f418094e3ec7c207998]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.540658 1182 ts_tablet_manager.cc:1916] T 51ba0bcd498043c383c493f45ad1fe17 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:55.548053 1175 tablet_bootstrap.cc:492] T a5f864fd8a3d4c4a83e7f68abb8a0a83 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:55.548584 1035 tablet_service.cc:1505] Processing CreateTablet for tablet 3cb014efbe9e46eab97e71fff0ff2887 (DEFAULT_TABLE table=default.Table_e5f4ac82d1ef4b85999695997eb9df2a [id=c97e407e519d4a538860f66931f6aefd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.549189 1040 tablet_service.cc:1505] Processing CreateTablet for tablet 0c55b4e75ae14b44a4955a3a236620a2 (DEFAULT_TABLE table=default.Table_346e800aad794a65acc77ee0bfb0b82c [id=0dcc50bca61b416ca6d7f7b2d3da8582]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.549708 1175 tablet_bootstrap.cc:654] T a5f864fd8a3d4c4a83e7f68abb8a0a83 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:55.551573 895 tablet_service.cc:1505] Processing CreateTablet for tablet 6ccb42ededd146a68dd7c9cc131fef58 (DEFAULT_TABLE table=default.Table_d376bfb6c4d8479b9a306b2f9c124fdd [id=2a2750072b914dc6b6f3adadd645206a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.551606 1034 tablet_service.cc:1505] Processing CreateTablet for tablet 67f279f55a2d42dc8cf5d62d392d3b68 (DEFAULT_TABLE table=default.Table_ac6e92f3b1f540a7af72ae905e0481a7 [id=63269345f7e842dd9d4d250191e12c6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.552301 1025 tablet_service.cc:1505] Processing CreateTablet for tablet ebdbfbea82494ba1b83daa5d93fa61c3 (DEFAULT_TABLE table=default.Table_1455877b4adc4c53a941d7fc145014b7 [id=5c2778a5ed864321bb59e445b4db671a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.552883 1038 tablet_service.cc:1505] Processing CreateTablet for tablet abd4d697cafa4e1a99fd37eb25f757dd (DEFAULT_TABLE table=default.Table_e11021f60bfe480ab07b2242bb36e94f [id=ae5d5815dcce405a897ffbce9d745591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.553526 897 tablet_service.cc:1505] Processing CreateTablet for tablet 8115d7ed07e94aa592ccbad4f096ba3b (DEFAULT_TABLE table=default.Table_8c1b93c2a1234d058590e574f89b770b [id=bf2ff444381f4d62a22258ffbfe1c040]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.553992 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cb014efbe9e46eab97e71fff0ff2887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.554661 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c55b4e75ae14b44a4955a3a236620a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.559301 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67f279f55a2d42dc8cf5d62d392d3b68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.564603 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebdbfbea82494ba1b83daa5d93fa61c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.568838 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd4d697cafa4e1a99fd37eb25f757dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.570551 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f03ed259a444605be1bab5125aed773. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.573071 890 tablet_service.cc:1505] Processing CreateTablet for tablet 1a6720763aeb41fbbe2344b627994856 (DEFAULT_TABLE table=default.Table_18fec323d6b04e10a687de6361b643b2 [id=1ffcc11561524fe8a7b09a7ddc92d5b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.575953 1175 tablet_bootstrap.cc:492] T a5f864fd8a3d4c4a83e7f68abb8a0a83 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:55.577064 1175 ts_tablet_manager.cc:1403] T a5f864fd8a3d4c4a83e7f68abb8a0a83 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.029s user 0.006s sys 0.000s
I20260202 02:52:55.578774 889 tablet_service.cc:1505] Processing CreateTablet for tablet e7937da857e14dd6b55c3b6f988dfa4d (DEFAULT_TABLE table=default.Table_c70528658e2f49bcb97d3c9c815379ec [id=976c3ac7dde84abab2a5666eb46576e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.579281 3122 raft_consensus.cc:493] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:55.579416 3122 raft_consensus.cc:515] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.579447 1175 raft_consensus.cc:359] T a5f864fd8a3d4c4a83e7f68abb8a0a83 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.579524 892 tablet_service.cc:1505] Processing CreateTablet for tablet 737650f0494646ba822549abde78c846 (DEFAULT_TABLE table=default.Table_44e1b09767d14dc899a9d956b5db9b02 [id=bb2dfdb752ee45aea4f8658346231390]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.579614 1175 raft_consensus.cc:385] T a5f864fd8a3d4c4a83e7f68abb8a0a83 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:55.579658 1175 raft_consensus.cc:740] T a5f864fd8a3d4c4a83e7f68abb8a0a83 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:55.579965 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ccb42ededd146a68dd7c9cc131fef58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.580047 3122 leader_election.cc:290] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:55.580142 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a6720763aeb41fbbe2344b627994856. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.580500 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7937da857e14dd6b55c3b6f988dfa4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.585129 900 tablet_service.cc:1552] Processing DeleteTablet for tablet 23671408fcbc454cbd918328015cf6e6 with delete_type TABLET_DATA_DELETED (replaced by tablet 32fb916ae82145f195cc8b951069e5da) from {username='slave'} at 127.0.0.1:41736
W20260202 02:52:55.585579 999 leader_election.cc:343] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Not found: Tablet not found: 79badde1001b432b95f58b5161ace15c
I20260202 02:52:55.588449 1175 consensus_queue.cc:260] T a5f864fd8a3d4c4a83e7f68abb8a0a83 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.588492 1182 ts_tablet_manager.cc:1929] T 51ba0bcd498043c383c493f45ad1fe17 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:55.588861 1182 log.cc:1199] T 51ba0bcd498043c383c493f45ad1fe17 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/51ba0bcd498043c383c493f45ad1fe17
I20260202 02:52:55.589249 1182 ts_tablet_manager.cc:1950] T 51ba0bcd498043c383c493f45ad1fe17 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:55.589648 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 737650f0494646ba822549abde78c846. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.593762 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 916) took 2655 ms. Trace:
I20260202 02:52:55.593914 1182 rpcz_store.cc:276] 0202 02:52:52.938353 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.938942 (+ 589us) service_pool.cc:225] Handling call
0202 02:52:55.529867 (+2590925us) ts_tablet_manager.cc:1151] Deleting tablet 51ba0bcd498043c383c493f45ad1fe17
0202 02:52:55.529879 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.581532 (+ 51653us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.588387 (+ 6855us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.593725 (+ 5338us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2590372}
I20260202 02:52:55.594601 514 catalog_manager.cc:4985] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): tablet 51ba0bcd498043c383c493f45ad1fe17 (table default.Table_619d2fcf726a46bfb5e939bbe6f55612 [id=038692c44a0043bab69ab29f77aa13be]) successfully deleted
I20260202 02:52:55.594862 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 921) took 2640 ms. Trace:
I20260202 02:52:55.594961 1182 rpcz_store.cc:276] 0202 02:52:52.954211 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.954285 (+ 74us) service_pool.cc:225] Handling call
0202 02:52:55.594225 (+2639940us) ts_tablet_manager.cc:1151] Deleting tablet 08fbd094d4df46cf82a694b4e1f30c6d
0202 02:52:55.594398 (+ 173us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.594842 (+ 444us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2639671}
W20260202 02:52:55.595352 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 08fbd094d4df46cf82a694b4e1f30c6d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 08fbd094d4df46cf82a694b4e1f30c6d already in progress: creating tablet
W20260202 02:52:55.595657 998 connection.cc:537] client connection to 127.0.104.129:39403 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260202 02:52:55.595777 998 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107) [suppressed 1 similar messages]
W20260202 02:52:55.595837 857 connection.cc:537] server connection from 127.0.104.129:50505 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260202 02:52:55.596166 1000 connection.cc:537] server connection from 127.0.104.129:38747 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260202 02:52:55.596166 857 connection.cc:537] client connection to 127.0.104.129:39403 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260202 02:52:55.596290 857 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260202 02:52:55.599115 1175 ts_tablet_manager.cc:1434] T a5f864fd8a3d4c4a83e7f68abb8a0a83 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.020s user 0.001s sys 0.001s
I20260202 02:52:55.599267 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8115d7ed07e94aa592ccbad4f096ba3b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:55.599329 515 connection.cc:537] server connection from 127.0.104.129:48435 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260202 02:52:55.599364 513 connection.cc:537] client connection to 127.0.104.129:39403 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260202 02:52:55.599478 513 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107) [suppressed 4 similar messages]
I20260202 02:52:55.603623 1037 tablet_service.cc:1505] Processing CreateTablet for tablet 46e898063fb449dca45e69ab8045c3df (DEFAULT_TABLE table=default.Table_f4ed15aa08c44cb29cf820bbbac9c6df [id=d3fcc9cc54ba4938858eb27aa840f5d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.604051 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46e898063fb449dca45e69ab8045c3df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.604880 1175 tablet_bootstrap.cc:492] T 1c8317adc47a4c399f2db650ed3d2e53 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:55.605559 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 925) took 2625 ms. Trace:
I20260202 02:52:55.605654 1182 rpcz_store.cc:276] 0202 02:52:52.980410 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.984378 (+ 3968us) service_pool.cc:225] Handling call
0202 02:52:55.595564 (+2611186us) ts_tablet_manager.cc:1151] Deleting tablet 81abd5ef66cc4f0bb37844c6ff5c256d
0202 02:52:55.605405 (+ 9841us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.605534 (+ 129us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2610664}
I20260202 02:52:55.605976 1182 tablet_replica.cc:333] T 5e20842de8cb4355b732685605c88834 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:55.606150 1182 raft_consensus.cc:2243] T 5e20842de8cb4355b732685605c88834 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:55.606295 1182 raft_consensus.cc:2272] T 5e20842de8cb4355b732685605c88834 P 1521a6d262b7451bbef8debe970dd6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:55.606747 1175 tablet_bootstrap.cc:654] T 1c8317adc47a4c399f2db650ed3d2e53 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:55.606812 1182 ts_tablet_manager.cc:1916] T 5e20842de8cb4355b732685605c88834 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:55.608458 1173 tablet_bootstrap.cc:492] T 09504070c9ec4bcfbb59e3fb4c189775 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:55.608662 1173 ts_tablet_manager.cc:1403] T 09504070c9ec4bcfbb59e3fb4c189775 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.076s user 0.005s sys 0.001s
I20260202 02:52:55.609460 1173 raft_consensus.cc:359] T 09504070c9ec4bcfbb59e3fb4c189775 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.609699 1173 raft_consensus.cc:385] T 09504070c9ec4bcfbb59e3fb4c189775 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:55.609795 1173 raft_consensus.cc:740] T 09504070c9ec4bcfbb59e3fb4c189775 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:55.609911 418 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskYXxklX/build/asan/bin/kudu pid 834
I20260202 02:52:55.610018 1173 consensus_queue.cc:260] T 09504070c9ec4bcfbb59e3fb4c189775 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.612828 1034 tablet_service.cc:1552] Processing DeleteTablet for tablet 3dcf4f794ec14b2f9d181d015fc437fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:30 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:55.613716 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 6ccb42ededd146a68dd7c9cc131fef58 (DEFAULT_TABLE table=default.Table_d376bfb6c4d8479b9a306b2f9c124fdd [id=2a2750072b914dc6b6f3adadd645206a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.614096 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ccb42ededd146a68dd7c9cc131fef58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.616827 1038 tablet_service.cc:1505] Processing CreateTablet for tablet 8115d7ed07e94aa592ccbad4f096ba3b (DEFAULT_TABLE table=default.Table_8c1b93c2a1234d058590e574f89b770b [id=bf2ff444381f4d62a22258ffbfe1c040]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.617228 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8115d7ed07e94aa592ccbad4f096ba3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.618146 1173 ts_tablet_manager.cc:1434] T 09504070c9ec4bcfbb59e3fb4c189775 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.009s user 0.001s sys 0.000s
I20260202 02:52:55.618432 1173 tablet_bootstrap.cc:492] T c977c146bac34638874b7a990604a0d5 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:55.620924 1040 tablet_service.cc:1505] Processing CreateTablet for tablet 737650f0494646ba822549abde78c846 (DEFAULT_TABLE table=default.Table_44e1b09767d14dc899a9d956b5db9b02 [id=bb2dfdb752ee45aea4f8658346231390]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.621307 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 737650f0494646ba822549abde78c846. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:55.622720 514 catalog_manager.cc:4965] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): delete failed for tablet 81abd5ef66cc4f0bb37844c6ff5c256d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 81abd5ef66cc4f0bb37844c6ff5c256d already in progress: creating tablet
I20260202 02:52:55.627645 1030 tablet_service.cc:1505] Processing CreateTablet for tablet e7937da857e14dd6b55c3b6f988dfa4d (DEFAULT_TABLE table=default.Table_c70528658e2f49bcb97d3c9c815379ec [id=976c3ac7dde84abab2a5666eb46576e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.628032 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7937da857e14dd6b55c3b6f988dfa4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.632787 1173 tablet_bootstrap.cc:654] T c977c146bac34638874b7a990604a0d5 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:55.614966 963 generic_service.cc:196] Checking for leaks (request via RPC)
I20260202 02:52:55.634274 1040 tablet_service.cc:1505] Processing CreateTablet for tablet 1a6720763aeb41fbbe2344b627994856 (DEFAULT_TABLE table=default.Table_18fec323d6b04e10a687de6361b643b2 [id=1ffcc11561524fe8a7b09a7ddc92d5b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.617609 3080 raft_consensus.cc:493] T bb1005103c25423783736ace1866c328 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:55.634688 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a6720763aeb41fbbe2344b627994856. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.634598 3080 raft_consensus.cc:515] T bb1005103c25423783736ace1866c328 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
W20260202 02:52:55.626910 857 leader_election.cc:336] T c9ef08b5830147a3a55049285a40535b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111)
I20260202 02:52:55.627120 892 tablet_service.cc:1552] Processing DeleteTablet for tablet a5f864fd8a3d4c4a83e7f68abb8a0a83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:41736
I20260202 02:52:55.644305 1037 tablet_service.cc:1552] Processing DeleteTablet for tablet b9fc33e8efc04734a951f7fcf9048142 with delete_type TABLET_DATA_DELETED (replaced by tablet 8a3ebe597da64da48db881cef46cfe4e) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:55.644717 1034 tablet_service.cc:1505] Processing CreateTablet for tablet 4e2fc5b9113447c193243ea028ce4956 (DEFAULT_TABLE table=default.Table_efacf68164014a2d8873859a3650a913 [id=baa4d4e86e9b475899c69cc47683f853]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.646775 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e2fc5b9113447c193243ea028ce4956. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:55.655596 1882 ts_tablet_manager.cc:1929] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 3.3
I20260202 02:52:55.655848 1882 log.cc:1199] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/25765fb687294e9db1289f76530dd877
I20260202 02:52:55.656279 1882 ts_tablet_manager.cc:1950] T 25765fb687294e9db1289f76530dd877 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:55.657203 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 803) took 5439 ms. Trace:
I20260202 02:52:55.657382 1882 rpcz_store.cc:276] 0202 02:52:50.217753 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:50.217817 (+ 64us) service_pool.cc:225] Handling call
0202 02:52:55.493800 (+5275983us) ts_tablet_manager.cc:1151] Deleting tablet 25765fb687294e9db1289f76530dd877
0202 02:52:55.497680 (+ 3880us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.502148 (+ 4468us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.655454 (+153306us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.657175 (+ 1721us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5275755}
I20260202 02:52:55.657872 1882 tablet_replica.cc:333] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:55.658126 1882 raft_consensus.cc:2243] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:55.658474 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 25765fb687294e9db1289f76530dd877 (table default.Table_03cb720f804b44e2b29f8cc47336b2db [id=ea0c52a72da8422ba14345fbfc242d27]) successfully deleted
W20260202 02:52:55.661921 513 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 62437.1us
I20260202 02:52:55.664776 1882 raft_consensus.cc:2272] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:55.665263 1882 ts_tablet_manager.cc:1916] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:55.660782 1036 tablet_service.cc:1505] Processing CreateTablet for tablet 6f03ed259a444605be1bab5125aed773 (DEFAULT_TABLE table=default.Table_e4bcea76324043f09b2a8d4cdec7f371 [id=4cb26dd223bc4f418094e3ec7c207998]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:55.667528 1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f03ed259a444605be1bab5125aed773. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:55.662804 998 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.129:39403, user_credentials={real_user=slave}} blocked reactor thread for 67026.4us
I20260202 02:52:55.663692 1173 tablet_bootstrap.cc:492] T c977c146bac34638874b7a990604a0d5 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:55.669780 1173 ts_tablet_manager.cc:1403] T c977c146bac34638874b7a990604a0d5 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.051s user 0.006s sys 0.000s
I20260202 02:52:55.670576 1173 raft_consensus.cc:359] T c977c146bac34638874b7a990604a0d5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:55.671465 1173 raft_consensus.cc:385] T c977c146bac34638874b7a990604a0d5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:55.671576 1173 raft_consensus.cc:740] T c977c146bac34638874b7a990604a0d5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:55.671864 1173 consensus_queue.cc:260] T c977c146bac34638874b7a990604a0d5 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:55.678445 1882 ts_tablet_manager.cc:1929] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:55.680248 1882 log.cc:1199] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/29c996781cb947f3a1aee5dd0a9c4828
I20260202 02:52:55.680738 1882 ts_tablet_manager.cc:1950] T 29c996781cb947f3a1aee5dd0a9c4828 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:55.681598 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet c977c146bac34638874b7a990604a0d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:55.681864 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 804) took 5434 ms. Trace:
I20260202 02:52:55.681982 1882 rpcz_store.cc:276] 0202 02:52:50.247376 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:50.247436 (+ 60us) service_pool.cc:225] Handling call
0202 02:52:55.657816 (+5410380us) ts_tablet_manager.cc:1151] Deleting tablet 29c996781cb947f3a1aee5dd0a9c4828
0202 02:52:55.657829 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.675364 (+ 17535us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.678329 (+ 2965us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.681849 (+ 3520us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5410147}
I20260202 02:52:55.682436 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 29c996781cb947f3a1aee5dd0a9c4828 (table default.Table_458f570486154241ad8c7f10a8135ace [id=d2eb691b1287473990f0a66d02f71554]) successfully deleted
I20260202 02:52:55.682792 1882 tablet_replica.cc:333] T a0b5d82e5fb94b60b4aa324b978a5378 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:55.682835 1173 ts_tablet_manager.cc:1434] T c977c146bac34638874b7a990604a0d5 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.013s user 0.001s sys 0.001s
I20260202 02:52:55.683084 1882 raft_consensus.cc:2243] T a0b5d82e5fb94b60b4aa324b978a5378 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:55.683290 1882 raft_consensus.cc:2272] T a0b5d82e5fb94b60b4aa324b978a5378 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:55.683981 1882 ts_tablet_manager.cc:1916] T a0b5d82e5fb94b60b4aa324b978a5378 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:55.689940 1882 ts_tablet_manager.cc:1929] T a0b5d82e5fb94b60b4aa324b978a5378 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:55.690153 1882 log.cc:1199] T a0b5d82e5fb94b60b4aa324b978a5378 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/a0b5d82e5fb94b60b4aa324b978a5378
I20260202 02:52:55.690596 1882 ts_tablet_manager.cc:1950] T a0b5d82e5fb94b60b4aa324b978a5378 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:55.691449 1173 tablet_bootstrap.cc:492] T a81e1d10ca49459989260fbd6a2b8645 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:55.691424 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 806) took 5416 ms. Trace:
I20260202 02:52:55.692272 1882 rpcz_store.cc:276] 0202 02:52:50.274590 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:50.274648 (+ 58us) service_pool.cc:225] Handling call
0202 02:52:55.682744 (+5408096us) ts_tablet_manager.cc:1151] Deleting tablet a0b5d82e5fb94b60b4aa324b978a5378
0202 02:52:55.682755 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.687030 (+ 4275us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.689819 (+ 2789us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.691405 (+ 1586us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5407915}
I20260202 02:52:55.692857 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 816) took 5368 ms. Trace:
I20260202 02:52:55.692919 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet a0b5d82e5fb94b60b4aa324b978a5378 (table default.Table_cf37f87385ee44cea1ff80cc14050ff2 [id=697b6fe62ee845d895af350881a03556]) successfully deleted
I20260202 02:52:55.694375 1882 rpcz_store.cc:276] 0202 02:52:50.324861 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:50.324961 (+ 100us) service_pool.cc:225] Handling call
0202 02:52:55.692714 (+5367753us) ts_tablet_manager.cc:1151] Deleting tablet 39eaa2848cf4450dbe563ddd3b11df06
0202 02:52:55.692726 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.692843 (+ 117us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5356310}
I20260202 02:52:55.694763 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 839) took 5082 ms. Trace:
I20260202 02:52:55.694835 1882 rpcz_store.cc:276] 0202 02:52:50.612280 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:50.612763 (+ 483us) service_pool.cc:225] Handling call
0202 02:52:55.694626 (+5081863us) ts_tablet_manager.cc:1151] Deleting tablet 1139db8ea6744e168e7b223c23665c58
0202 02:52:55.694637 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.694746 (+ 109us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5050436}
W20260202 02:52:55.694896 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 39eaa2848cf4450dbe563ddd3b11df06 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 39eaa2848cf4450dbe563ddd3b11df06 already in progress: creating tablet
I20260202 02:52:55.695071 1882 tablet_replica.cc:333] T 587cfd0ae62f4cfd937faf6890e78f4a P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:55.695247 1882 raft_consensus.cc:2243] T 587cfd0ae62f4cfd937faf6890e78f4a P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:55.695408 1882 raft_consensus.cc:2272] T 587cfd0ae62f4cfd937faf6890e78f4a P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:55.695456 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 1139db8ea6744e168e7b223c23665c58 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1139db8ea6744e168e7b223c23665c58 already in progress: creating tablet
I20260202 02:52:55.695905 3122 raft_consensus.cc:493] T b9fc33e8efc04734a951f7fcf9048142 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:55.695968 1882 ts_tablet_manager.cc:1916] T 587cfd0ae62f4cfd937faf6890e78f4a P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:55.696018 3122 raft_consensus.cc:515] T b9fc33e8efc04734a951f7fcf9048142 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:55.694314 1173 tablet_bootstrap.cc:654] T a81e1d10ca49459989260fbd6a2b8645 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:55.696636 3122 leader_election.cc:290] T b9fc33e8efc04734a951f7fcf9048142 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:55.697360 998 consensus_peers.cc:597] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:55.697568 998 consensus_peers.cc:597] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:55.697705 998 consensus_peers.cc:597] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:55.697820 998 consensus_peers.cc:597] T af5aa68e47bb420eac300754d274fa73 P b1f05820b6034228bdad884cc348716d -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:55.698030 998 consensus_peers.cc:597] T b427c7e897c64977b924e7d586420e69 P b1f05820b6034228bdad884cc348716d -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:55.698148 998 leader_election.cc:336] T ee04563d3b8e4cb2b989bb8d6ee9b20e P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111)
I20260202 02:52:55.698279 998 leader_election.cc:304] T ee04563d3b8e4cb2b989bb8d6ee9b20e P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:55.698565 3122 raft_consensus.cc:2749] T ee04563d3b8e4cb2b989bb8d6ee9b20e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:55.698952 998 leader_election.cc:336] T eea6bb04ddf24276b46270d009ed78b4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111)
I20260202 02:52:55.699126 998 leader_election.cc:304] T eea6bb04ddf24276b46270d009ed78b4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:55.699353 3122 raft_consensus.cc:2749] T eea6bb04ddf24276b46270d009ed78b4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:55.699623 998 consensus_peers.cc:597] T 939fbcaf75c7439d9208bbbf83a72717 P b1f05820b6034228bdad884cc348716d -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:55.699764 998 leader_election.cc:336] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111)
W20260202 02:52:55.699987 998 leader_election.cc:336] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111)
I20260202 02:52:55.700150 998 leader_election.cc:304] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
W20260202 02:52:55.700104 513 catalog_manager.cc:4712] TS 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): CreateTablet RPC failed for tablet cd2e710eebfe4a8f9ec20a24c1374356: Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111)
I20260202 02:52:55.700414 3122 raft_consensus.cc:2749] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260202 02:52:55.700881 998 consensus_peers.cc:597] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:55.701071 998 consensus_peers.cc:597] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:55.701197 998 leader_election.cc:336] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111)
W20260202 02:52:55.701411 998 consensus_peers.cc:597] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:55.703464 998 leader_election.cc:336] T b9fc33e8efc04734a951f7fcf9048142 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111)
I20260202 02:52:55.704787 1173 tablet_bootstrap.cc:492] T a81e1d10ca49459989260fbd6a2b8645 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:55.704988 1173 ts_tablet_manager.cc:1403] T a81e1d10ca49459989260fbd6a2b8645 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.002s
I20260202 02:52:55.705967 1173 raft_consensus.cc:359] T a81e1d10ca49459989260fbd6a2b8645 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.706240 1173 raft_consensus.cc:385] T a81e1d10ca49459989260fbd6a2b8645 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:55.706331 1173 raft_consensus.cc:740] T a81e1d10ca49459989260fbd6a2b8645 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:55.706743 1882 ts_tablet_manager.cc:1929] T 587cfd0ae62f4cfd937faf6890e78f4a P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:55.706720 1173 consensus_queue.cc:260] T a81e1d10ca49459989260fbd6a2b8645 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.706966 1882 log.cc:1199] T 587cfd0ae62f4cfd937faf6890e78f4a P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/587cfd0ae62f4cfd937faf6890e78f4a
I20260202 02:52:55.716571 1882 ts_tablet_manager.cc:1950] T 587cfd0ae62f4cfd937faf6890e78f4a P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:55.712877 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet a81e1d10ca49459989260fbd6a2b8645 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:49 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:55.717448 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 841) took 5095 ms. Trace:
I20260202 02:52:55.717549 1882 rpcz_store.cc:276] 0202 02:52:50.622186 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:50.622452 (+ 266us) service_pool.cc:225] Handling call
0202 02:52:55.695029 (+5072577us) ts_tablet_manager.cc:1151] Deleting tablet 587cfd0ae62f4cfd937faf6890e78f4a
0202 02:52:55.695038 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.699922 (+ 4884us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.706635 (+ 6713us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.717425 (+ 10790us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5050808}
I20260202 02:52:55.717871 1882 tablet_replica.cc:333] T 65bd1c1a382e42e4a68077264563c55b P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:55.718039 1882 raft_consensus.cc:2243] T 65bd1c1a382e42e4a68077264563c55b P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:55.718175 1882 raft_consensus.cc:2272] T 65bd1c1a382e42e4a68077264563c55b P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:55.719201 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 587cfd0ae62f4cfd937faf6890e78f4a (table default.Table_698a93b0be2e4203836e0522ac44acc3 [id=9f2441734085417fb56dc05a42df54db]) successfully deleted
I20260202 02:52:55.707338 1173 ts_tablet_manager.cc:1434] T a81e1d10ca49459989260fbd6a2b8645 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:55.719825 1882 ts_tablet_manager.cc:1916] T 65bd1c1a382e42e4a68077264563c55b P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:55.722653 1173 tablet_bootstrap.cc:492] T 7921daa000f54c8dbed04473e21cf0a5 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:55.724470 1173 tablet_bootstrap.cc:654] T 7921daa000f54c8dbed04473e21cf0a5 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:55.729455 1882 ts_tablet_manager.cc:1929] T 65bd1c1a382e42e4a68077264563c55b P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:55.729612 1882 log.cc:1199] T 65bd1c1a382e42e4a68077264563c55b P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/65bd1c1a382e42e4a68077264563c55b
I20260202 02:52:55.730006 1882 ts_tablet_manager.cc:1950] T 65bd1c1a382e42e4a68077264563c55b P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:55.730859 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 842) took 5108 ms. Trace:
I20260202 02:52:55.730970 1882 rpcz_store.cc:276] 0202 02:52:50.622317 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:50.622575 (+ 258us) service_pool.cc:225] Handling call
0202 02:52:55.717829 (+5095254us) ts_tablet_manager.cc:1151] Deleting tablet 65bd1c1a382e42e4a68077264563c55b
0202 02:52:55.717839 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.725855 (+ 8016us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.729332 (+ 3477us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.730833 (+ 1501us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5073429}
I20260202 02:52:55.731295 1882 tablet_replica.cc:333] T 1f379a29e2454f689d68f5951837512b P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:55.731503 1882 raft_consensus.cc:2243] T 1f379a29e2454f689d68f5951837512b P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:55.731570 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 65bd1c1a382e42e4a68077264563c55b (table default.Table_1cc0209a40ad471fb1c41012a476df79 [id=b941bc2edc03444394d1ba8a596a0d27]) successfully deleted
I20260202 02:52:55.731676 1882 raft_consensus.cc:2272] T 1f379a29e2454f689d68f5951837512b P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:55.738217 1882 ts_tablet_manager.cc:1916] T 1f379a29e2454f689d68f5951837512b P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:55.744401 1882 ts_tablet_manager.cc:1929] T 1f379a29e2454f689d68f5951837512b P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:55.744598 1882 log.cc:1199] T 1f379a29e2454f689d68f5951837512b P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/1f379a29e2454f689d68f5951837512b
I20260202 02:52:55.744959 1882 ts_tablet_manager.cc:1950] T 1f379a29e2454f689d68f5951837512b P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:55.745769 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 843) took 5123 ms. Trace:
I20260202 02:52:55.745898 1882 rpcz_store.cc:276] 0202 02:52:50.622414 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:50.622621 (+ 207us) service_pool.cc:225] Handling call
0202 02:52:55.731245 (+5108624us) ts_tablet_manager.cc:1151] Deleting tablet 1f379a29e2454f689d68f5951837512b
0202 02:52:55.731255 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.741264 (+ 10009us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.744254 (+ 2990us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.745746 (+ 1492us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5086719}
I20260202 02:52:55.746346 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 838) took 5146 ms. Trace:
I20260202 02:52:55.746439 1882 rpcz_store.cc:276] 0202 02:52:50.599693 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:50.600005 (+ 312us) service_pool.cc:225] Handling call
0202 02:52:55.746211 (+5146206us) ts_tablet_manager.cc:1151] Deleting tablet 87ff5fbc95b84171a94fe1a83a67a7ba
0202 02:52:55.746221 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.746331 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5101581}
I20260202 02:52:55.747045 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 840) took 5130 ms. Trace:
I20260202 02:52:55.747287 1882 rpcz_store.cc:276] 0202 02:52:50.616234 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:50.616310 (+ 76us) service_pool.cc:225] Handling call
0202 02:52:55.746696 (+5130386us) ts_tablet_manager.cc:1151] Deleting tablet e04960479c92479b8bb864544dd0c8ac
0202 02:52:55.746818 (+ 122us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.747026 (+ 208us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5101954}
I20260202 02:52:55.747597 1882 tablet_replica.cc:333] T ac2ab12f5040488aa306cb69b107b485 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:55.747553 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 1f379a29e2454f689d68f5951837512b (table default.Table_67c55c53ca254b03b0c326672a41b559 [id=7e80e30271434635a17d0034580aeca8]) successfully deleted
I20260202 02:52:55.747833 1882 raft_consensus.cc:2243] T ac2ab12f5040488aa306cb69b107b485 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
W20260202 02:52:55.747828 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 87ff5fbc95b84171a94fe1a83a67a7ba because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 87ff5fbc95b84171a94fe1a83a67a7ba already in progress: creating tablet
I20260202 02:52:55.748009 1882 pending_rounds.cc:70] T ac2ab12f5040488aa306cb69b107b485 P b1f05820b6034228bdad884cc348716d: Trying to abort 1 pending ops.
I20260202 02:52:55.748034 1173 tablet_bootstrap.cc:492] T 7921daa000f54c8dbed04473e21cf0a5 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:55.748063 1882 pending_rounds.cc:77] T ac2ab12f5040488aa306cb69b107b485 P b1f05820b6034228bdad884cc348716d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923170694041600 op_type: NO_OP noop_request { }
I20260202 02:52:55.748174 1173 ts_tablet_manager.cc:1403] T 7921daa000f54c8dbed04473e21cf0a5 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.026s user 0.007s sys 0.002s
W20260202 02:52:55.748524 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet e04960479c92479b8bb864544dd0c8ac because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e04960479c92479b8bb864544dd0c8ac already in progress: creating tablet
I20260202 02:52:55.749035 1173 raft_consensus.cc:359] T 7921daa000f54c8dbed04473e21cf0a5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.749204 1173 raft_consensus.cc:385] T 7921daa000f54c8dbed04473e21cf0a5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:55.749253 1173 raft_consensus.cc:740] T 7921daa000f54c8dbed04473e21cf0a5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:55.749454 1173 consensus_queue.cc:260] T 7921daa000f54c8dbed04473e21cf0a5 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.749965 1173 ts_tablet_manager.cc:1434] T 7921daa000f54c8dbed04473e21cf0a5 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260202 02:52:55.748188 1882 raft_consensus.cc:2889] T ac2ab12f5040488aa306cb69b107b485 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:55.750206 1882 raft_consensus.cc:2272] T ac2ab12f5040488aa306cb69b107b485 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:55.750608 1173 tablet_bootstrap.cc:492] T cd1b6662da254889ae4bbe8d0b1ef1bb P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:55.751911 1882 ts_tablet_manager.cc:1916] T ac2ab12f5040488aa306cb69b107b485 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:55.758687 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet 7921daa000f54c8dbed04473e21cf0a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:25 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:55.758973 1882 ts_tablet_manager.cc:1929] T ac2ab12f5040488aa306cb69b107b485 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:55.759099 1882 log.cc:1199] T ac2ab12f5040488aa306cb69b107b485 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/ac2ab12f5040488aa306cb69b107b485
I20260202 02:52:55.759702 1882 ts_tablet_manager.cc:1950] T ac2ab12f5040488aa306cb69b107b485 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:55.761480 1173 tablet_bootstrap.cc:654] T cd1b6662da254889ae4bbe8d0b1ef1bb P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:55.769148 1173 tablet_bootstrap.cc:492] T cd1b6662da254889ae4bbe8d0b1ef1bb P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:55.769326 1173 ts_tablet_manager.cc:1403] T cd1b6662da254889ae4bbe8d0b1ef1bb P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.000s
I20260202 02:52:55.770121 1173 raft_consensus.cc:359] T cd1b6662da254889ae4bbe8d0b1ef1bb P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.770306 1173 raft_consensus.cc:385] T cd1b6662da254889ae4bbe8d0b1ef1bb P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:55.770365 1173 raft_consensus.cc:740] T cd1b6662da254889ae4bbe8d0b1ef1bb P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:55.769172 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 844) took 5146 ms. Trace:
I20260202 02:52:55.770599 1882 rpcz_store.cc:276] 0202 02:52:50.622550 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:50.622604 (+ 54us) service_pool.cc:225] Handling call
0202 02:52:55.747548 (+5124944us) ts_tablet_manager.cc:1151] Deleting tablet ac2ab12f5040488aa306cb69b107b485
0202 02:52:55.747557 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.755645 (+ 8088us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.758851 (+ 3206us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.769146 (+ 10295us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5102758}
I20260202 02:52:55.771389 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 845) took 4851 ms. Trace:
I20260202 02:52:55.771487 1882 rpcz_store.cc:276] 0202 02:52:50.920302 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:50.920381 (+ 79us) service_pool.cc:225] Handling call
0202 02:52:55.771238 (+4850857us) ts_tablet_manager.cc:1151] Deleting tablet 82680e577c504a729122cd4f6b4916a6
0202 02:52:55.771250 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.771369 (+ 119us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4818585}
W20260202 02:52:55.771955 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 82680e577c504a729122cd4f6b4916a6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 82680e577c504a729122cd4f6b4916a6 already in progress: creating tablet
I20260202 02:52:55.772418 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 852) took 4611 ms. Trace:
I20260202 02:52:55.772518 1882 rpcz_store.cc:276] 0202 02:52:51.160675 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:51.160741 (+ 66us) service_pool.cc:225] Handling call
0202 02:52:55.772258 (+4611517us) ts_tablet_manager.cc:1151] Deleting tablet 23dfa02219d748919e3cf24beb2095be
0202 02:52:55.772270 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.772400 (+ 130us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4611312}
W20260202 02:52:55.772979 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 23dfa02219d748919e3cf24beb2095be because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 23dfa02219d748919e3cf24beb2095be already in progress: creating tablet
I20260202 02:52:55.773286 1173 consensus_queue.cc:260] T cd1b6662da254889ae4bbe8d0b1ef1bb P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.773878 1173 ts_tablet_manager.cc:1434] T cd1b6662da254889ae4bbe8d0b1ef1bb P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260202 02:52:55.775664 1173 tablet_bootstrap.cc:492] T 604b3b863c9540d3b55381dbfe1afdc3 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:55.777606 1173 tablet_bootstrap.cc:654] T 604b3b863c9540d3b55381dbfe1afdc3 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:55.782086 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet cd1b6662da254889ae4bbe8d0b1ef1bb with delete_type TABLET_DATA_DELETED (replaced by tablet 83f30c7f03504cdbb379fc0469324306) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:55.784123 998 consensus_peers.cc:597] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:55.788633 998 consensus_peers.cc:597] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:55.792586 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 853) took 4582 ms. Trace:
I20260202 02:52:55.792716 1882 rpcz_store.cc:276] 0202 02:52:51.210219 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:51.210332 (+ 113us) service_pool.cc:225] Handling call
0202 02:52:55.774052 (+4563720us) ts_tablet_manager.cc:1151] Deleting tablet 0da286b7f598457ca4d0254a7073cb74
0202 02:52:55.792358 (+ 18306us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.792555 (+ 197us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4563449}
I20260202 02:52:55.793105 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 855) took 4573 ms. Trace:
I20260202 02:52:55.793181 1882 rpcz_store.cc:276] 0202 02:52:51.220092 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:51.221220 (+ 1128us) service_pool.cc:225] Handling call
0202 02:52:55.792988 (+4571768us) ts_tablet_manager.cc:1151] Deleting tablet 0aa023e74e74480498b721bbb5d15052
0202 02:52:55.793000 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.793088 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4570555}
W20260202 02:52:55.793227 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 0da286b7f598457ca4d0254a7073cb74 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0da286b7f598457ca4d0254a7073cb74 already in progress: creating tablet
I20260202 02:52:55.793493 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 858) took 4493 ms. Trace:
I20260202 02:52:55.793578 1882 rpcz_store.cc:276] 0202 02:52:51.299914 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:51.299987 (+ 73us) service_pool.cc:225] Handling call
0202 02:52:55.793395 (+4493408us) ts_tablet_manager.cc:1151] Deleting tablet ad7c98afaa3447d1aac8a243c6a98ed6
0202 02:52:55.793404 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.793476 (+ 72us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4493211}
W20260202 02:52:55.793658 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 0aa023e74e74480498b721bbb5d15052 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0aa023e74e74480498b721bbb5d15052 already in progress: creating tablet
I20260202 02:52:55.793886 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 859) took 4487 ms. Trace:
W20260202 02:52:55.793992 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet ad7c98afaa3447d1aac8a243c6a98ed6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ad7c98afaa3447d1aac8a243c6a98ed6 already in progress: creating tablet
I20260202 02:52:55.793952 1882 rpcz_store.cc:276] 0202 02:52:51.306843 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:51.344306 (+ 37463us) service_pool.cc:225] Handling call
0202 02:52:55.793790 (+4449484us) ts_tablet_manager.cc:1151] Deleting tablet fea0fcb8854c4fc6acda80c8ffb8d341
0202 02:52:55.793799 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.793867 (+ 68us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4449074}
I20260202 02:52:55.794188 1882 tablet_replica.cc:333] T 2b352d9a815e4bccafb7e2251500c5bc P b1f05820b6034228bdad884cc348716d: stopping tablet replica
W20260202 02:52:55.794293 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet fea0fcb8854c4fc6acda80c8ffb8d341 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fea0fcb8854c4fc6acda80c8ffb8d341 already in progress: creating tablet
I20260202 02:52:55.794368 1882 raft_consensus.cc:2243] T 2b352d9a815e4bccafb7e2251500c5bc P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:55.794540 1882 raft_consensus.cc:2272] T 2b352d9a815e4bccafb7e2251500c5bc P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:55.795516 1882 ts_tablet_manager.cc:1916] T 2b352d9a815e4bccafb7e2251500c5bc P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:55.795562 998 consensus_peers.cc:597] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:55.797122 998 consensus_peers.cc:597] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:55.802999 1882 ts_tablet_manager.cc:1929] T 2b352d9a815e4bccafb7e2251500c5bc P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:55.803149 1882 log.cc:1199] T 2b352d9a815e4bccafb7e2251500c5bc P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/2b352d9a815e4bccafb7e2251500c5bc
I20260202 02:52:55.803467 1882 ts_tablet_manager.cc:1950] T 2b352d9a815e4bccafb7e2251500c5bc P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:55.804639 1173 tablet_bootstrap.cc:492] T 604b3b863c9540d3b55381dbfe1afdc3 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:55.804797 1173 ts_tablet_manager.cc:1403] T 604b3b863c9540d3b55381dbfe1afdc3 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.029s user 0.006s sys 0.001s
I20260202 02:52:55.805600 1173 raft_consensus.cc:359] T 604b3b863c9540d3b55381dbfe1afdc3 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.805773 1173 raft_consensus.cc:385] T 604b3b863c9540d3b55381dbfe1afdc3 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:55.805822 1173 raft_consensus.cc:740] T 604b3b863c9540d3b55381dbfe1afdc3 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:55.806036 1173 consensus_queue.cc:260] T 604b3b863c9540d3b55381dbfe1afdc3 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.806710 1173 ts_tablet_manager.cc:1434] T 604b3b863c9540d3b55381dbfe1afdc3 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:55.809762 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 869) took 4278 ms. Trace:
I20260202 02:52:55.809907 1882 rpcz_store.cc:276] 0202 02:52:51.531360 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:51.531450 (+ 90us) service_pool.cc:225] Handling call
0202 02:52:55.794145 (+4262695us) ts_tablet_manager.cc:1151] Deleting tablet 2b352d9a815e4bccafb7e2251500c5bc
0202 02:52:55.794154 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.799115 (+ 4961us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.802873 (+ 3758us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.809733 (+ 6860us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4262410}
I20260202 02:52:55.810372 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 2b352d9a815e4bccafb7e2251500c5bc (table default.Table_408b8f21ff294ce9880d0b9a4d681f93 [id=c0fa167c7d1f47ea994fcf7dfdd4100f]) successfully deleted
I20260202 02:52:55.811688 1173 tablet_bootstrap.cc:492] T 7f323175ee254df789bcb04e65047dff P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:55.813601 1173 tablet_bootstrap.cc:654] T 7f323175ee254df789bcb04e65047dff P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:55.814064 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 870) took 4282 ms. Trace:
I20260202 02:52:55.821074 1882 rpcz_store.cc:276] 0202 02:52:51.531859 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:51.531911 (+ 52us) service_pool.cc:225] Handling call
0202 02:52:55.810900 (+4278989us) ts_tablet_manager.cc:1151] Deleting tablet b7880cb051c24bc998003f0effed3cf8
0202 02:52:55.813947 (+ 3047us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.814043 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4278812}
W20260202 02:52:55.823681 515 catalog_manager.cc:4952] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet b7880cb051c24bc998003f0effed3cf8 because the tablet was not found. No further retry: Not found: Tablet not found: b7880cb051c24bc998003f0effed3cf8
I20260202 02:52:55.824290 1882 tablet_replica.cc:333] T 7ea7a5d8a92b4ac0903483627258ae66 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:55.824589 1882 raft_consensus.cc:2243] T 7ea7a5d8a92b4ac0903483627258ae66 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:55.825031 1882 raft_consensus.cc:2272] T 7ea7a5d8a92b4ac0903483627258ae66 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:55.828234 1882 ts_tablet_manager.cc:1916] T 7ea7a5d8a92b4ac0903483627258ae66 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:55.829061 1173 tablet_bootstrap.cc:492] T 7f323175ee254df789bcb04e65047dff P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:55.829206 1173 ts_tablet_manager.cc:1403] T 7f323175ee254df789bcb04e65047dff P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.018s user 0.012s sys 0.003s
I20260202 02:52:55.830058 1173 raft_consensus.cc:359] T 7f323175ee254df789bcb04e65047dff P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:55.830240 1173 raft_consensus.cc:385] T 7f323175ee254df789bcb04e65047dff P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:55.830284 1173 raft_consensus.cc:740] T 7f323175ee254df789bcb04e65047dff P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:55.830485 1173 consensus_queue.cc:260] T 7f323175ee254df789bcb04e65047dff P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:55.831091 1173 ts_tablet_manager.cc:1434] T 7f323175ee254df789bcb04e65047dff P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:55.840768 1173 tablet_bootstrap.cc:492] T 7f9dfac19c04475aadb45e95a6474eb4 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:55.844412 1173 tablet_bootstrap.cc:654] T 7f9dfac19c04475aadb45e95a6474eb4 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:55.848018 1882 ts_tablet_manager.cc:1929] T 7ea7a5d8a92b4ac0903483627258ae66 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:55.848158 1882 log.cc:1199] T 7ea7a5d8a92b4ac0903483627258ae66 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/7ea7a5d8a92b4ac0903483627258ae66
I20260202 02:52:55.848533 1882 ts_tablet_manager.cc:1950] T 7ea7a5d8a92b4ac0903483627258ae66 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
W20260202 02:52:55.849593 998 consensus_peers.cc:597] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:55.855448 3181 raft_consensus.cc:493] T c977c146bac34638874b7a990604a0d5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:55.860146 3181 raft_consensus.cc:515] T c977c146bac34638874b7a990604a0d5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:55.860860 3181 leader_election.cc:290] T c977c146bac34638874b7a990604a0d5 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:55.862426 998 leader_election.cc:336] T c977c146bac34638874b7a990604a0d5 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111)
I20260202 02:52:55.863999 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 871) took 4331 ms. Trace:
I20260202 02:52:55.864106 1173 tablet_bootstrap.cc:492] T 7f9dfac19c04475aadb45e95a6474eb4 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:55.864224 1173 ts_tablet_manager.cc:1403] T 7f9dfac19c04475aadb45e95a6474eb4 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.024s user 0.000s sys 0.002s
I20260202 02:52:55.864130 1882 rpcz_store.cc:276] 0202 02:52:51.532658 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:51.532708 (+ 50us) service_pool.cc:225] Handling call
0202 02:52:55.824220 (+4291512us) ts_tablet_manager.cc:1151] Deleting tablet 7ea7a5d8a92b4ac0903483627258ae66
0202 02:52:55.824234 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.837773 (+ 13539us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.847887 (+ 10114us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.863978 (+ 16091us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4291316}
I20260202 02:52:55.864576 1882 tablet_replica.cc:333] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:55.864696 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 7ea7a5d8a92b4ac0903483627258ae66 (table default.Table_d040708acb92480496bac8b012890006 [id=1c26d457c06345b7abeb1941bcee6c31]) successfully deleted
I20260202 02:52:55.865007 1173 raft_consensus.cc:359] T 7f9dfac19c04475aadb45e95a6474eb4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:55.865183 1173 raft_consensus.cc:385] T 7f9dfac19c04475aadb45e95a6474eb4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:55.865235 1173 raft_consensus.cc:740] T 7f9dfac19c04475aadb45e95a6474eb4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:55.865422 1173 consensus_queue.cc:260] T 7f9dfac19c04475aadb45e95a6474eb4 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:55.865849 1173 ts_tablet_manager.cc:1434] T 7f9dfac19c04475aadb45e95a6474eb4 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:55.880777 1173 tablet_bootstrap.cc:492] T e95153ee8f2d40de955e3389d3a58644 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:55.882434 1173 tablet_bootstrap.cc:654] T e95153ee8f2d40de955e3389d3a58644 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:55.885241 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet 7f9dfac19c04475aadb45e95a6474eb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:55.890801 1882 raft_consensus.cc:2243] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:55.891300 1882 raft_consensus.cc:2272] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:55.891968 1882 ts_tablet_manager.cc:1916] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:55.898763 1882 ts_tablet_manager.cc:1929] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:55.899159 1882 log.cc:1199] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/d0ce2121d65b4a228596e0ca9018883d
I20260202 02:52:55.899627 1882 ts_tablet_manager.cc:1950] T d0ce2121d65b4a228596e0ca9018883d P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:55.899650 1173 tablet_bootstrap.cc:492] T e95153ee8f2d40de955e3389d3a58644 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:55.899866 1173 ts_tablet_manager.cc:1403] T e95153ee8f2d40de955e3389d3a58644 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.004s
I20260202 02:52:55.900568 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 868) took 4370 ms. Trace:
I20260202 02:52:55.900631 1173 raft_consensus.cc:359] T e95153ee8f2d40de955e3389d3a58644 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.900816 1173 raft_consensus.cc:385] T e95153ee8f2d40de955e3389d3a58644 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:55.900871 1173 raft_consensus.cc:740] T e95153ee8f2d40de955e3389d3a58644 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:55.900714 1882 rpcz_store.cc:276] 0202 02:52:51.530510 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:51.535436 (+ 4926us) service_pool.cc:225] Handling call
0202 02:52:55.864510 (+4329074us) ts_tablet_manager.cc:1151] Deleting tablet d0ce2121d65b4a228596e0ca9018883d
0202 02:52:55.864519 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.895211 (+ 30692us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.898646 (+ 3435us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.900544 (+ 1898us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":13030,"tablet-delete.queue_time_us":4328742}
I20260202 02:52:55.901064 1173 consensus_queue.cc:260] T e95153ee8f2d40de955e3389d3a58644 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.901425 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet d0ce2121d65b4a228596e0ca9018883d (table default.Table_bde65e061a274734a7df7a428bd8ce53 [id=ca632e6d14ca41e2ab9444dc362a4265]) successfully deleted
I20260202 02:52:55.901539 1173 ts_tablet_manager.cc:1434] T e95153ee8f2d40de955e3389d3a58644 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260202 02:52:55.901806 1173 tablet_bootstrap.cc:492] T 1959438c70c544e69422f7354082cf89 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:55.903864 1173 tablet_bootstrap.cc:654] T 1959438c70c544e69422f7354082cf89 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:55.914459 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet e95153ee8f2d40de955e3389d3a58644 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:23 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:55.915968 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 872) took 4167 ms. Trace:
I20260202 02:52:55.916137 1882 rpcz_store.cc:276] 0202 02:52:51.748262 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:51.754490 (+ 6228us) service_pool.cc:225] Handling call
0202 02:52:55.901910 (+4147420us) ts_tablet_manager.cc:1151] Deleting tablet 21fe651f45bf43609cf972bef5eca149
0202 02:52:55.902232 (+ 322us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.915941 (+ 13709us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4147121}
W20260202 02:52:55.916692 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 21fe651f45bf43609cf972bef5eca149 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 21fe651f45bf43609cf972bef5eca149 already in progress: creating tablet
I20260202 02:52:55.917140 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 873) took 4101 ms. Trace:
I20260202 02:52:55.917250 1882 rpcz_store.cc:276] 0202 02:52:51.815575 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:51.820196 (+ 4621us) service_pool.cc:225] Handling call
0202 02:52:55.917025 (+4096829us) ts_tablet_manager.cc:1151] Deleting tablet 00a8c927e0344033a774a4db2cb97d5b
0202 02:52:55.917037 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.917123 (+ 86us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4096278}
I20260202 02:52:55.918069 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 874) took 4054 ms. Trace:
I20260202 02:52:55.918179 1882 rpcz_store.cc:276] 0202 02:52:51.863513 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:51.864773 (+ 1260us) service_pool.cc:225] Handling call
0202 02:52:55.917959 (+4053186us) ts_tablet_manager.cc:1151] Deleting tablet 0291c3e873bb4266ab040356fe35f4e9
0202 02:52:55.917970 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.918052 (+ 82us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4052952}
W20260202 02:52:55.918586 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 0291c3e873bb4266ab040356fe35f4e9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0291c3e873bb4266ab040356fe35f4e9 already in progress: creating tablet
I20260202 02:52:55.918896 1882 tablet_replica.cc:333] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:55.919114 1882 raft_consensus.cc:2243] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:55.919431 1882 raft_consensus.cc:2272] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:55.919510 998 consensus_peers.cc:597] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:55.920197 1882 ts_tablet_manager.cc:1916] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:55.921452 1173 tablet_bootstrap.cc:492] T 1959438c70c544e69422f7354082cf89 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:55.921613 1173 ts_tablet_manager.cc:1403] T 1959438c70c544e69422f7354082cf89 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.004s
I20260202 02:52:55.922786 1173 raft_consensus.cc:359] T 1959438c70c544e69422f7354082cf89 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.923007 1173 raft_consensus.cc:385] T 1959438c70c544e69422f7354082cf89 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:55.923089 1173 raft_consensus.cc:740] T 1959438c70c544e69422f7354082cf89 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:55.923331 1173 consensus_queue.cc:260] T 1959438c70c544e69422f7354082cf89 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.923978 1173 ts_tablet_manager.cc:1434] T 1959438c70c544e69422f7354082cf89 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:55.927552 1173 tablet_bootstrap.cc:492] T c9ef08b5830147a3a55049285a40535b P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:55.929121 1173 tablet_bootstrap.cc:654] T c9ef08b5830147a3a55049285a40535b P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:55.946889 998 consensus_peers.cc:597] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:55.947228 1173 tablet_bootstrap.cc:492] T c9ef08b5830147a3a55049285a40535b P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:55.947374 1173 ts_tablet_manager.cc:1403] T c9ef08b5830147a3a55049285a40535b P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.004s
I20260202 02:52:55.948042 1173 raft_consensus.cc:359] T c9ef08b5830147a3a55049285a40535b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.948213 1173 raft_consensus.cc:385] T c9ef08b5830147a3a55049285a40535b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:55.948239 2979 raft_consensus.cc:493] T c17c85759e964f0ba3abcaf5fe938fba P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:55.948277 1173 raft_consensus.cc:740] T c9ef08b5830147a3a55049285a40535b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:55.948375 2979 raft_consensus.cc:515] T c17c85759e964f0ba3abcaf5fe938fba P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:55.948513 1173 consensus_queue.cc:260] T c9ef08b5830147a3a55049285a40535b P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:55.948937 2979 leader_election.cc:290] T c17c85759e964f0ba3abcaf5fe938fba P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:55.950805 998 leader_election.cc:336] T c17c85759e964f0ba3abcaf5fe938fba P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111)
I20260202 02:52:55.951355 1173 ts_tablet_manager.cc:1434] T c9ef08b5830147a3a55049285a40535b P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260202 02:52:55.951650 1173 tablet_bootstrap.cc:492] T bbcc723a8079443db455ae20fb0d585d P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:55.953118 1173 tablet_bootstrap.cc:654] T bbcc723a8079443db455ae20fb0d585d P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:55.965480 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet c9ef08b5830147a3a55049285a40535b with delete_type TABLET_DATA_DELETED (replaced by tablet 5d2fc0a74684434d85b9922cb4e32fdc) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:55.970777 1882 ts_tablet_manager.cc:1929] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:55.971036 1882 log.cc:1199] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/2f5aae089fcc492f9ce55f84674a452c
I20260202 02:52:55.972047 1882 ts_tablet_manager.cc:1950] T 2f5aae089fcc492f9ce55f84674a452c P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:55.972956 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 875) took 3903 ms. Trace:
I20260202 02:52:55.973340 1882 rpcz_store.cc:276] 0202 02:52:52.069726 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.069969 (+ 243us) service_pool.cc:225] Handling call
0202 02:52:55.918845 (+3848876us) ts_tablet_manager.cc:1151] Deleting tablet 2f5aae089fcc492f9ce55f84674a452c
0202 02:52:55.918855 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:55.967498 (+ 48643us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.970657 (+ 3159us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:55.972936 (+ 2279us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3848669}
I20260202 02:52:55.973932 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 2f5aae089fcc492f9ce55f84674a452c (table default.Table_84d6693620cc468e82d5cb1ec6cc69ea [id=f6625b21b6064828b11acdfdb694cd71]) successfully deleted
I20260202 02:52:55.974354 1882 tablet_replica.cc:333] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:55.974819 1882 raft_consensus.cc:2243] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:55.975041 1882 raft_consensus.cc:2272] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:55.975204 1173 tablet_bootstrap.cc:492] T bbcc723a8079443db455ae20fb0d585d P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:55.975323 1173 ts_tablet_manager.cc:1403] T bbcc723a8079443db455ae20fb0d585d P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.004s
I20260202 02:52:55.975670 1882 ts_tablet_manager.cc:1916] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:55.975991 1173 raft_consensus.cc:359] T bbcc723a8079443db455ae20fb0d585d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:55.976152 1173 raft_consensus.cc:385] T bbcc723a8079443db455ae20fb0d585d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:55.976203 1173 raft_consensus.cc:740] T bbcc723a8079443db455ae20fb0d585d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:55.976450 1173 consensus_queue.cc:260] T bbcc723a8079443db455ae20fb0d585d P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:55.977123 1173 ts_tablet_manager.cc:1434] T bbcc723a8079443db455ae20fb0d585d P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:55.989703 1173 tablet_bootstrap.cc:492] T f6c2bc531d194a4b8f3062c9b40e3da6 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:55.991248 1173 tablet_bootstrap.cc:654] T f6c2bc531d194a4b8f3062c9b40e3da6 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:56.001195 1173 tablet_bootstrap.cc:492] T f6c2bc531d194a4b8f3062c9b40e3da6 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:56.001348 1173 ts_tablet_manager.cc:1403] T f6c2bc531d194a4b8f3062c9b40e3da6 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.002s
I20260202 02:52:56.002084 1173 raft_consensus.cc:359] T f6c2bc531d194a4b8f3062c9b40e3da6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.002254 1173 raft_consensus.cc:385] T f6c2bc531d194a4b8f3062c9b40e3da6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:56.002303 1173 raft_consensus.cc:740] T f6c2bc531d194a4b8f3062c9b40e3da6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:56.002521 1173 consensus_queue.cc:260] T f6c2bc531d194a4b8f3062c9b40e3da6 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.009415 1173 ts_tablet_manager.cc:1434] T f6c2bc531d194a4b8f3062c9b40e3da6 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260202 02:52:56.014250 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet f6c2bc531d194a4b8f3062c9b40e3da6 with delete_type TABLET_DATA_DELETED (replaced by tablet 29cde4226db6428987d4a39cde804a5c) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:56.015327 1173 tablet_bootstrap.cc:492] T dc4c3f52f4f44a84b0d0e3c9a4a49f09 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:56.016824 1173 tablet_bootstrap.cc:654] T dc4c3f52f4f44a84b0d0e3c9a4a49f09 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:56.020006 1173 tablet_bootstrap.cc:492] T dc4c3f52f4f44a84b0d0e3c9a4a49f09 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:56.020160 1173 ts_tablet_manager.cc:1403] T dc4c3f52f4f44a84b0d0e3c9a4a49f09 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260202 02:52:56.020913 1173 raft_consensus.cc:359] T dc4c3f52f4f44a84b0d0e3c9a4a49f09 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.021083 1173 raft_consensus.cc:385] T dc4c3f52f4f44a84b0d0e3c9a4a49f09 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:56.021133 1173 raft_consensus.cc:740] T dc4c3f52f4f44a84b0d0e3c9a4a49f09 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:56.021363 1173 consensus_queue.cc:260] T dc4c3f52f4f44a84b0d0e3c9a4a49f09 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
W20260202 02:52:56.029150 998 consensus_peers.cc:597] T 44c8d5b74c8e4c499f7f68e8185efc17 P b1f05820b6034228bdad884cc348716d -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:56.030138 1173 ts_tablet_manager.cc:1434] T dc4c3f52f4f44a84b0d0e3c9a4a49f09 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20260202 02:52:56.030730 1882 ts_tablet_manager.cc:1929] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:56.043015 1882 log.cc:1199] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/79badde1001b432b95f58b5161ace15c
I20260202 02:52:56.043382 1882 ts_tablet_manager.cc:1950] T 79badde1001b432b95f58b5161ace15c P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.036016 1173 tablet_bootstrap.cc:492] T 842197c689064660b791b40e521d346b P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:56.039921 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet dc4c3f52f4f44a84b0d0e3c9a4a49f09 with delete_type TABLET_DATA_DELETED (replaced by tablet 3191a80374724effa8389adc842e3448) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:56.045275 1173 tablet_bootstrap.cc:654] T 842197c689064660b791b40e521d346b P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:56.051385 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 876) took 3981 ms. Trace:
I20260202 02:52:56.052734 1882 rpcz_store.cc:276] 0202 02:52:52.069898 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.070213 (+ 315us) service_pool.cc:225] Handling call
0202 02:52:55.974300 (+3904087us) ts_tablet_manager.cc:1151] Deleting tablet 79badde1001b432b95f58b5161ace15c
0202 02:52:55.974311 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.027777 (+ 53466us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.030617 (+ 2840us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.051353 (+ 20736us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3903965}
I20260202 02:52:56.053378 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 79badde1001b432b95f58b5161ace15c (table default.Table_80912cc76f934f139da7f866546ae97a [id=2f3bf7e0cf4c41ac8a2554360b57214a]) successfully deleted
I20260202 02:52:56.053896 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 877) took 3900 ms. Trace:
I20260202 02:52:56.054370 1882 rpcz_store.cc:276] 0202 02:52:52.153366 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.153449 (+ 83us) service_pool.cc:225] Handling call
0202 02:52:56.053734 (+3900285us) ts_tablet_manager.cc:1151] Deleting tablet 104b2c537c4f4e3da6711364cb5d2a2e
0202 02:52:56.053748 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.053876 (+ 128us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3899964}
W20260202 02:52:56.055204 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 104b2c537c4f4e3da6711364cb5d2a2e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 104b2c537c4f4e3da6711364cb5d2a2e already in progress: creating tablet
I20260202 02:52:56.055634 1882 tablet_replica.cc:333] T ee04563d3b8e4cb2b989bb8d6ee9b20e P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.055892 1882 raft_consensus.cc:2243] T ee04563d3b8e4cb2b989bb8d6ee9b20e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:56.056108 1882 raft_consensus.cc:2272] T ee04563d3b8e4cb2b989bb8d6ee9b20e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.056805 1882 ts_tablet_manager.cc:1916] T ee04563d3b8e4cb2b989bb8d6ee9b20e P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.061398 1173 tablet_bootstrap.cc:492] T 842197c689064660b791b40e521d346b P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:56.061565 1173 ts_tablet_manager.cc:1403] T 842197c689064660b791b40e521d346b P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.026s user 0.015s sys 0.002s
I20260202 02:52:56.062335 1173 raft_consensus.cc:359] T 842197c689064660b791b40e521d346b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.062506 1173 raft_consensus.cc:385] T 842197c689064660b791b40e521d346b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:56.062554 1173 raft_consensus.cc:740] T 842197c689064660b791b40e521d346b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:56.062757 1173 consensus_queue.cc:260] T 842197c689064660b791b40e521d346b P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.068995 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet 842197c689064660b791b40e521d346b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:28 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:56.069461 1173 ts_tablet_manager.cc:1434] T 842197c689064660b791b40e521d346b P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260202 02:52:56.075436 1173 tablet_bootstrap.cc:492] T 9a1d7d173c41462c8e32a95b494bb708 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:56.077641 1173 tablet_bootstrap.cc:654] T 9a1d7d173c41462c8e32a95b494bb708 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:56.080616 1882 ts_tablet_manager.cc:1929] T ee04563d3b8e4cb2b989bb8d6ee9b20e P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:56.080837 1882 log.cc:1199] T ee04563d3b8e4cb2b989bb8d6ee9b20e P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/ee04563d3b8e4cb2b989bb8d6ee9b20e
I20260202 02:52:56.082029 1882 ts_tablet_manager.cc:1950] T ee04563d3b8e4cb2b989bb8d6ee9b20e P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.082811 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 878) took 3884 ms. Trace:
I20260202 02:52:56.082959 1882 rpcz_store.cc:276] 0202 02:52:52.198396 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.198551 (+ 155us) service_pool.cc:225] Handling call
0202 02:52:56.055573 (+3857022us) ts_tablet_manager.cc:1151] Deleting tablet ee04563d3b8e4cb2b989bb8d6ee9b20e
0202 02:52:56.055585 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.060294 (+ 4709us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.080488 (+ 20194us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.082792 (+ 2304us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3856800}
I20260202 02:52:56.083561 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet ee04563d3b8e4cb2b989bb8d6ee9b20e (table default.Table_62b7624afec14fb6a860d51a1a09913d [id=78feba5e47064271ac41a928580d1f02]) successfully deleted
I20260202 02:52:56.083986 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 879) took 3587 ms. Trace:
I20260202 02:52:56.084105 1882 rpcz_store.cc:276] 0202 02:52:52.496937 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.497240 (+ 303us) service_pool.cc:225] Handling call
0202 02:52:56.083857 (+3586617us) ts_tablet_manager.cc:1151] Deleting tablet 27158aa96e2742d5b0b2c3fa9531178d
0202 02:52:56.083868 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.083971 (+ 103us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3571166}
W20260202 02:52:56.084635 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 27158aa96e2742d5b0b2c3fa9531178d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 27158aa96e2742d5b0b2c3fa9531178d already in progress: creating tablet
I20260202 02:52:56.085035 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 880) took 3489 ms. Trace:
I20260202 02:52:56.085139 1882 rpcz_store.cc:276] 0202 02:52:52.596009 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.596062 (+ 53us) service_pool.cc:225] Handling call
0202 02:52:56.084927 (+3488865us) ts_tablet_manager.cc:1151] Deleting tablet 41a4143247ed4eed98defbdfe2895372
0202 02:52:56.084937 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.085020 (+ 83us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3482569}
W20260202 02:52:56.085577 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 41a4143247ed4eed98defbdfe2895372 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 41a4143247ed4eed98defbdfe2895372 already in progress: creating tablet
I20260202 02:52:56.085896 1882 tablet_replica.cc:333] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.086112 1882 raft_consensus.cc:2243] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:56.086418 1882 raft_consensus.cc:2272] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.087028 1882 ts_tablet_manager.cc:1916] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:56.090809 998 consensus_peers.cc:597] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:56.093318 1882 ts_tablet_manager.cc:1929] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:56.093812 1882 log.cc:1199] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/da5435f969074364bc8f8a6ddc5e4f8a
I20260202 02:52:56.094205 1882 ts_tablet_manager.cc:1950] T da5435f969074364bc8f8a6ddc5e4f8a P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.095328 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 881) took 3476 ms. Trace:
I20260202 02:52:56.095615 2979 raft_consensus.cc:493] T e95153ee8f2d40de955e3389d3a58644 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:56.095731 2979 raft_consensus.cc:515] T e95153ee8f2d40de955e3389d3a58644 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.096312 2979 leader_election.cc:290] T e95153ee8f2d40de955e3389d3a58644 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:56.098258 998 leader_election.cc:336] T e95153ee8f2d40de955e3389d3a58644 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111)
I20260202 02:52:56.095464 1882 rpcz_store.cc:276] 0202 02:52:52.618744 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.618816 (+ 72us) service_pool.cc:225] Handling call
0202 02:52:56.085852 (+3467036us) ts_tablet_manager.cc:1151] Deleting tablet da5435f969074364bc8f8a6ddc5e4f8a
0202 02:52:56.085860 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.090062 (+ 4202us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.093207 (+ 3145us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.095228 (+ 2021us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3466845}
I20260202 02:52:56.099005 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet da5435f969074364bc8f8a6ddc5e4f8a (table default.Table_51648e2a42614c3d9c5820901d153932 [id=ad256e6219784dff9aa916cf07f67fbe]) successfully deleted
I20260202 02:52:56.099629 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 882) took 3479 ms. Trace:
I20260202 02:52:56.099746 1882 rpcz_store.cc:276] 0202 02:52:52.619721 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.619783 (+ 62us) service_pool.cc:225] Handling call
0202 02:52:56.099517 (+3479734us) ts_tablet_manager.cc:1151] Deleting tablet 26b890ed87c3471b9fc0b172c90e0a11
0202 02:52:56.099531 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.099611 (+ 80us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3479563}
W20260202 02:52:56.100211 515 catalog_manager.cc:4952] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 26b890ed87c3471b9fc0b172c90e0a11 because the tablet was not found. No further retry: Not found: Tablet not found: 26b890ed87c3471b9fc0b172c90e0a11
I20260202 02:52:56.100603 1882 tablet_replica.cc:333] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.101128 1882 raft_consensus.cc:2243] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:56.101563 2979 raft_consensus.cc:493] T f6c2bc531d194a4b8f3062c9b40e3da6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:56.101689 2979 raft_consensus.cc:515] T f6c2bc531d194a4b8f3062c9b40e3da6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.102257 2979 leader_election.cc:290] T f6c2bc531d194a4b8f3062c9b40e3da6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:56.103874 998 leader_election.cc:336] T f6c2bc531d194a4b8f3062c9b40e3da6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111)
I20260202 02:52:56.104185 1882 raft_consensus.cc:2272] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.104818 1882 ts_tablet_manager.cc:1916] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.113134 1882 ts_tablet_manager.cc:1929] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:56.113442 1882 log.cc:1199] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/8153510d5d154c7895b1ee794585612e
I20260202 02:52:56.113866 1882 ts_tablet_manager.cc:1950] T 8153510d5d154c7895b1ee794585612e P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.114626 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 883) took 3494 ms. Trace:
I20260202 02:52:56.114773 1882 rpcz_store.cc:276] 0202 02:52:52.620291 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.620357 (+ 66us) service_pool.cc:225] Handling call
0202 02:52:56.100554 (+3480197us) ts_tablet_manager.cc:1151] Deleting tablet 8153510d5d154c7895b1ee794585612e
0202 02:52:56.100564 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.107936 (+ 7372us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.113016 (+ 5080us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.114608 (+ 1592us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3480027}
I20260202 02:52:56.115283 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 8153510d5d154c7895b1ee794585612e (table default.Table_0e7ceba4956f408c8a9fc0f61fa67c90 [id=c0c6499675894350ac008b534d030c93]) successfully deleted
I20260202 02:52:56.115710 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 884) took 3459 ms. Trace:
I20260202 02:52:56.116691 1882 rpcz_store.cc:276] 0202 02:52:52.656249 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.656312 (+ 63us) service_pool.cc:225] Handling call
0202 02:52:56.115583 (+3459271us) ts_tablet_manager.cc:1151] Deleting tablet 64e2fcb966fc4a06afb65506be1711a2
0202 02:52:56.115594 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.115696 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3459037}
W20260202 02:52:56.117193 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 64e2fcb966fc4a06afb65506be1711a2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 64e2fcb966fc4a06afb65506be1711a2 already in progress: creating tablet
I20260202 02:52:56.117528 1882 tablet_replica.cc:333] T 99e0b5e9a9874aec886155e0d614dec2 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.117743 1882 raft_consensus.cc:2243] T 99e0b5e9a9874aec886155e0d614dec2 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:56.117930 1882 raft_consensus.cc:2272] T 99e0b5e9a9874aec886155e0d614dec2 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.118492 1882 ts_tablet_manager.cc:1916] T 99e0b5e9a9874aec886155e0d614dec2 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.124413 1882 ts_tablet_manager.cc:1929] T 99e0b5e9a9874aec886155e0d614dec2 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:56.116631 1173 tablet_bootstrap.cc:492] T 9a1d7d173c41462c8e32a95b494bb708 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:56.124707 1173 ts_tablet_manager.cc:1403] T 9a1d7d173c41462c8e32a95b494bb708 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.049s user 0.003s sys 0.007s
I20260202 02:52:56.125417 1173 raft_consensus.cc:359] T 9a1d7d173c41462c8e32a95b494bb708 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:56.125581 1173 raft_consensus.cc:385] T 9a1d7d173c41462c8e32a95b494bb708 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:56.125625 1173 raft_consensus.cc:740] T 9a1d7d173c41462c8e32a95b494bb708 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:56.125942 1882 log.cc:1199] T 99e0b5e9a9874aec886155e0d614dec2 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/99e0b5e9a9874aec886155e0d614dec2
I20260202 02:52:56.125818 1173 consensus_queue.cc:260] T 9a1d7d173c41462c8e32a95b494bb708 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:56.126655 1173 ts_tablet_manager.cc:1434] T 9a1d7d173c41462c8e32a95b494bb708 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:56.128966 1173 tablet_bootstrap.cc:492] T 67a8c9a1998b47b3b214a06ce62381e9 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:56.130661 1173 tablet_bootstrap.cc:654] T 67a8c9a1998b47b3b214a06ce62381e9 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:56.138924 2979 raft_consensus.cc:493] T 9a1d7d173c41462c8e32a95b494bb708 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:56.139088 2979 raft_consensus.cc:515] T 9a1d7d173c41462c8e32a95b494bb708 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:56.139729 2979 leader_election.cc:290] T 9a1d7d173c41462c8e32a95b494bb708 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:56.142992 998 leader_election.cc:336] T 9a1d7d173c41462c8e32a95b494bb708 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111)
I20260202 02:52:56.144251 1882 ts_tablet_manager.cc:1950] T 99e0b5e9a9874aec886155e0d614dec2 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.145118 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 886) took 3483 ms. Trace:
I20260202 02:52:56.145208 1882 rpcz_store.cc:276] 0202 02:52:52.661330 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.661412 (+ 82us) service_pool.cc:225] Handling call
0202 02:52:56.117466 (+3456054us) ts_tablet_manager.cc:1151] Deleting tablet 99e0b5e9a9874aec886155e0d614dec2
0202 02:52:56.117475 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.121475 (+ 4000us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.124253 (+ 2778us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.145096 (+ 20843us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3455878}
I20260202 02:52:56.145444 1173 tablet_bootstrap.cc:492] T 67a8c9a1998b47b3b214a06ce62381e9 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:56.145536 1882 tablet_replica.cc:333] T f3eebefa74584c1791d52377f5ba1290 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.145594 1173 ts_tablet_manager.cc:1403] T 67a8c9a1998b47b3b214a06ce62381e9 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.004s
I20260202 02:52:56.145902 1882 raft_consensus.cc:2243] T f3eebefa74584c1791d52377f5ba1290 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:56.146323 1173 raft_consensus.cc:359] T 67a8c9a1998b47b3b214a06ce62381e9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.146509 1173 raft_consensus.cc:385] T 67a8c9a1998b47b3b214a06ce62381e9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:56.146569 1173 raft_consensus.cc:740] T 67a8c9a1998b47b3b214a06ce62381e9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:56.146740 1882 raft_consensus.cc:2272] T f3eebefa74584c1791d52377f5ba1290 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.146766 1173 consensus_queue.cc:260] T 67a8c9a1998b47b3b214a06ce62381e9 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.147250 1173 ts_tablet_manager.cc:1434] T 67a8c9a1998b47b3b214a06ce62381e9 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260202 02:52:56.147411 1882 ts_tablet_manager.cc:1916] T f3eebefa74584c1791d52377f5ba1290 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.147547 1173 tablet_bootstrap.cc:492] T 2f664f6119ba4d28b5ef1c212e611866 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:56.149272 1173 tablet_bootstrap.cc:654] T 2f664f6119ba4d28b5ef1c212e611866 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:56.150343 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 99e0b5e9a9874aec886155e0d614dec2 (table default.Table_a8c4f67a58b74bf8bb3716174ed5654f [id=e7975854ed2f4810a2d438b590267584]) successfully deleted
I20260202 02:52:56.153671 1882 ts_tablet_manager.cc:1929] T f3eebefa74584c1791d52377f5ba1290 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:56.162413 1882 log.cc:1199] T f3eebefa74584c1791d52377f5ba1290 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/f3eebefa74584c1791d52377f5ba1290
I20260202 02:52:56.162919 1882 ts_tablet_manager.cc:1950] T f3eebefa74584c1791d52377f5ba1290 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.163833 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 887) took 3501 ms. Trace:
I20260202 02:52:56.163985 1882 rpcz_store.cc:276] 0202 02:52:52.662665 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.662718 (+ 53us) service_pool.cc:225] Handling call
0202 02:52:56.145478 (+3482760us) ts_tablet_manager.cc:1151] Deleting tablet f3eebefa74584c1791d52377f5ba1290
0202 02:52:56.145487 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.150618 (+ 5131us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.153565 (+ 2947us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.163811 (+ 10246us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":552,"tablet-delete.queue_time_us":3482567}
I20260202 02:52:56.164580 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 888) took 3486 ms. Trace:
I20260202 02:52:56.164594 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet f3eebefa74584c1791d52377f5ba1290 (table default.Table_37b8a9e1666a4ddbaf5a13d828e9702c [id=e0cea4989f66443faf074eacf434658c]) successfully deleted
I20260202 02:52:56.164739 1882 rpcz_store.cc:276] 0202 02:52:52.677851 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.677916 (+ 65us) service_pool.cc:225] Handling call
0202 02:52:56.164410 (+3486494us) ts_tablet_manager.cc:1151] Deleting tablet 898778f8f0b548ec89cb1417d6f07926
0202 02:52:56.164420 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.164561 (+ 141us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3485315}
W20260202 02:52:56.165127 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 898778f8f0b548ec89cb1417d6f07926 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 898778f8f0b548ec89cb1417d6f07926 already in progress: creating tablet
I20260202 02:52:56.165582 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 889) took 3485 ms. Trace:
I20260202 02:52:56.165695 1882 rpcz_store.cc:276] 0202 02:52:52.679984 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.680037 (+ 53us) service_pool.cc:225] Handling call
0202 02:52:56.165439 (+3485402us) ts_tablet_manager.cc:1151] Deleting tablet af29c7a2ad9d4797b08052606151a7d5
0202 02:52:56.165447 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.165566 (+ 119us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3485236}
W20260202 02:52:56.166123 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet af29c7a2ad9d4797b08052606151a7d5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet af29c7a2ad9d4797b08052606151a7d5 already in progress: creating tablet
I20260202 02:52:56.168097 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 890) took 3487 ms. Trace:
I20260202 02:52:56.168241 1882 rpcz_store.cc:276] 0202 02:52:52.681074 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.681135 (+ 61us) service_pool.cc:225] Handling call
0202 02:52:56.166393 (+3485258us) ts_tablet_manager.cc:1151] Deleting tablet b00b4a0b85374bfe83928162c4da6479
0202 02:52:56.167972 (+ 1579us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.168074 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3485090}
W20260202 02:52:56.168761 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet b00b4a0b85374bfe83928162c4da6479 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b00b4a0b85374bfe83928162c4da6479 already in progress: creating tablet
I20260202 02:52:56.169164 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 891) took 3434 ms. Trace:
I20260202 02:52:56.169278 1882 rpcz_store.cc:276] 0202 02:52:52.734694 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.734751 (+ 57us) service_pool.cc:225] Handling call
0202 02:52:56.169078 (+3434327us) ts_tablet_manager.cc:1151] Deleting tablet 8153510d5d154c7895b1ee794585612e
0202 02:52:56.169088 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.169148 (+ 60us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3434140}
W20260202 02:52:56.169763 515 catalog_manager.cc:4952] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 8153510d5d154c7895b1ee794585612e because the tablet was not found. No further retry: Not found: Tablet not found: 8153510d5d154c7895b1ee794585612e
I20260202 02:52:56.170148 1882 tablet_replica.cc:333] T c8a273ac8cd344719a48fd81945bed0a P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.170387 1882 raft_consensus.cc:2243] T c8a273ac8cd344719a48fd81945bed0a P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:56.170609 1882 raft_consensus.cc:2272] T c8a273ac8cd344719a48fd81945bed0a P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.170885 2979 raft_consensus.cc:493] T 7921daa000f54c8dbed04473e21cf0a5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:56.170997 2979 raft_consensus.cc:515] T 7921daa000f54c8dbed04473e21cf0a5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.171226 1882 ts_tablet_manager.cc:1916] T c8a273ac8cd344719a48fd81945bed0a P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.171594 2979 leader_election.cc:290] T 7921daa000f54c8dbed04473e21cf0a5 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
W20260202 02:52:56.173504 998 leader_election.cc:336] T 7921daa000f54c8dbed04473e21cf0a5 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111)
I20260202 02:52:56.177397 1882 ts_tablet_manager.cc:1929] T c8a273ac8cd344719a48fd81945bed0a P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:56.177534 1882 log.cc:1199] T c8a273ac8cd344719a48fd81945bed0a P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/c8a273ac8cd344719a48fd81945bed0a
I20260202 02:52:56.177906 1882 ts_tablet_manager.cc:1950] T c8a273ac8cd344719a48fd81945bed0a P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.178673 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 892) took 3302 ms. Trace:
I20260202 02:52:56.178761 1882 rpcz_store.cc:276] 0202 02:52:52.875796 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.875855 (+ 59us) service_pool.cc:225] Handling call
0202 02:52:56.170098 (+3294243us) ts_tablet_manager.cc:1151] Deleting tablet c8a273ac8cd344719a48fd81945bed0a
0202 02:52:56.170108 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.174309 (+ 4201us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.177284 (+ 2975us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.178654 (+ 1370us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3294040}
I20260202 02:52:56.179162 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet c8a273ac8cd344719a48fd81945bed0a (table default.Table_5a703dc84eb34674bdc5793ac7295098 [id=62c3c39fb73249349cee49b86d535612]) successfully deleted
I20260202 02:52:56.196581 1882 tablet_replica.cc:333] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.196906 1882 raft_consensus.cc:2243] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:56.197261 1882 pending_rounds.cc:70] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d: Trying to abort 1 pending ops.
I20260202 02:52:56.197368 1882 pending_rounds.cc:77] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923174296227840 op_type: NO_OP noop_request { }
I20260202 02:52:56.197537 1882 raft_consensus.cc:2889] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:56.197672 1882 raft_consensus.cc:2272] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.198443 1882 ts_tablet_manager.cc:1916] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.199574 3252 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92a8191cf1e14e4d99f1c378c2d3c85b)
I20260202 02:52:56.200120 3252 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } }
I20260202 02:52:56.201622 538 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7dab2abef14445d688ce1335b1f9b2f8" candidate_term: 3 candidate_status { last_received { term: 2 index: 1176 } } ignore_live_leader: false dest_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" is_pre_election: true
I20260202 02:52:56.203084 3252 leader_election.cc:290] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 92a8191cf1e14e4d99f1c378c2d3c85b (127.0.104.189:46271), a9c3e3612bb94fb7b94c308041acfaeb (127.0.104.188:39077)
I20260202 02:52:56.203090 1447 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7dab2abef14445d688ce1335b1f9b2f8" candidate_term: 3 candidate_status { last_received { term: 2 index: 1176 } } ignore_live_leader: false dest_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" is_pre_election: true
I20260202 02:52:56.203704 2256 leader_election.cc:304] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7dab2abef14445d688ce1335b1f9b2f8; no voters: 92a8191cf1e14e4d99f1c378c2d3c85b, a9c3e3612bb94fb7b94c308041acfaeb
I20260202 02:52:56.204190 3252 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 7dab2abef14445d688ce1335b1f9b2f8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260202 02:52:56.206013 1173 tablet_bootstrap.cc:492] T 2f664f6119ba4d28b5ef1c212e611866 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:56.206584 1173 ts_tablet_manager.cc:1403] T 2f664f6119ba4d28b5ef1c212e611866 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.059s user 0.009s sys 0.021s
I20260202 02:52:56.206940 1882 ts_tablet_manager.cc:1929] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:56.207049 1882 log.cc:1199] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/f5727f9b9bbb4e77abfef065c51606a1
I20260202 02:52:56.207391 1882 ts_tablet_manager.cc:1950] T f5727f9b9bbb4e77abfef065c51606a1 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.207742 2979 raft_consensus.cc:493] T 09504070c9ec4bcfbb59e3fb4c189775 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:56.207835 1173 raft_consensus.cc:359] T 2f664f6119ba4d28b5ef1c212e611866 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.207983 1173 raft_consensus.cc:385] T 2f664f6119ba4d28b5ef1c212e611866 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:56.207868 2979 raft_consensus.cc:515] T 09504070c9ec4bcfbb59e3fb4c189775 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.208035 1173 raft_consensus.cc:740] T 2f664f6119ba4d28b5ef1c212e611866 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:56.208165 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 893) took 3331 ms. Trace:
I20260202 02:52:56.208271 1173 consensus_queue.cc:260] T 2f664f6119ba4d28b5ef1c212e611866 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.208705 2979 leader_election.cc:290] T 09504070c9ec4bcfbb59e3fb4c189775 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:56.208823 1173 ts_tablet_manager.cc:1434] T 2f664f6119ba4d28b5ef1c212e611866 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:56.209059 1173 tablet_bootstrap.cc:492] T bb1005103c25423783736ace1866c328 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:56.210757 1173 tablet_bootstrap.cc:654] T bb1005103c25423783736ace1866c328 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:56.208241 1882 rpcz_store.cc:276] 0202 02:52:52.876445 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.876493 (+ 48us) service_pool.cc:225] Handling call
0202 02:52:56.196509 (+3320016us) ts_tablet_manager.cc:1151] Deleting tablet f5727f9b9bbb4e77abfef065c51606a1
0202 02:52:56.196524 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.201946 (+ 5422us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.206827 (+ 4881us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.208146 (+ 1319us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3319832}
I20260202 02:52:56.212056 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet f5727f9b9bbb4e77abfef065c51606a1 (table default.Table_4e6c3886f58b49698a00d45b957f4c9a [id=b5cc03b77f414304b29c5fb4bb7dc3ee]) successfully deleted
I20260202 02:52:56.217407 1882 tablet_replica.cc:333] T eea6bb04ddf24276b46270d009ed78b4 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.217650 1882 raft_consensus.cc:2243] T eea6bb04ddf24276b46270d009ed78b4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:56.217816 1882 raft_consensus.cc:2272] T eea6bb04ddf24276b46270d009ed78b4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.218374 1882 ts_tablet_manager.cc:1916] T eea6bb04ddf24276b46270d009ed78b4 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.219715 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet 2f664f6119ba4d28b5ef1c212e611866 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:56.217660 998 leader_election.cc:336] T 09504070c9ec4bcfbb59e3fb4c189775 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111)
I20260202 02:52:56.225584 1882 ts_tablet_manager.cc:1929] T eea6bb04ddf24276b46270d009ed78b4 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:56.225729 1882 log.cc:1199] T eea6bb04ddf24276b46270d009ed78b4 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/eea6bb04ddf24276b46270d009ed78b4
I20260202 02:52:56.226034 1882 ts_tablet_manager.cc:1950] T eea6bb04ddf24276b46270d009ed78b4 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.227010 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 894) took 3350 ms. Trace:
I20260202 02:52:56.227105 1882 rpcz_store.cc:276] 0202 02:52:52.876970 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.877014 (+ 44us) service_pool.cc:225] Handling call
0202 02:52:56.212358 (+3335344us) ts_tablet_manager.cc:1151] Deleting tablet eea6bb04ddf24276b46270d009ed78b4
0202 02:52:56.212417 (+ 59us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.221471 (+ 9054us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.225450 (+ 3979us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.226987 (+ 1537us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3335178}
I20260202 02:52:56.227407 1882 tablet_replica.cc:333] T b427c7e897c64977b924e7d586420e69 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.227587 1882 raft_consensus.cc:2243] T b427c7e897c64977b924e7d586420e69 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:56.227588 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet eea6bb04ddf24276b46270d009ed78b4 (table default.Table_1366b68811c14d28b1785266d6f65a3a [id=b7139f5062824d66af2dbd62469a9c03]) successfully deleted
I20260202 02:52:56.227859 1882 raft_consensus.cc:2272] T b427c7e897c64977b924e7d586420e69 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.228349 1882 ts_tablet_manager.cc:1916] T b427c7e897c64977b924e7d586420e69 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.229292 1173 tablet_bootstrap.cc:492] T bb1005103c25423783736ace1866c328 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:56.229471 1173 ts_tablet_manager.cc:1403] T bb1005103c25423783736ace1866c328 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.001s
W20260202 02:52:56.232260 998 consensus_peers.cc:597] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:56.232570 1173 raft_consensus.cc:359] T bb1005103c25423783736ace1866c328 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.232828 1173 raft_consensus.cc:385] T bb1005103c25423783736ace1866c328 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:56.232915 1173 raft_consensus.cc:740] T bb1005103c25423783736ace1866c328 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:56.233186 1173 consensus_queue.cc:260] T bb1005103c25423783736ace1866c328 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.240316 1882 ts_tablet_manager.cc:1929] T b427c7e897c64977b924e7d586420e69 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:56.240509 1882 log.cc:1199] T b427c7e897c64977b924e7d586420e69 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/b427c7e897c64977b924e7d586420e69
I20260202 02:52:56.240881 1882 ts_tablet_manager.cc:1950] T b427c7e897c64977b924e7d586420e69 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.241662 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 895) took 3363 ms. Trace:
I20260202 02:52:56.241746 1882 rpcz_store.cc:276] 0202 02:52:52.877967 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.878020 (+ 53us) service_pool.cc:225] Handling call
0202 02:52:56.227359 (+3349339us) ts_tablet_manager.cc:1151] Deleting tablet b427c7e897c64977b924e7d586420e69
0202 02:52:56.227370 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.231250 (+ 3880us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.240193 (+ 8943us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.241638 (+ 1445us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3349189}
I20260202 02:52:56.242141 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet b427c7e897c64977b924e7d586420e69 (table default.Table_616a8ade4b5e481b8d4906b7a2a026d0 [id=98ee1a7044cd46d8b39dfda755ed8a7d]) successfully deleted
I20260202 02:52:56.242517 1882 tablet_replica.cc:333] T 8b0368e03f644f1db28dbb57c95c48c1 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.242740 1882 raft_consensus.cc:2243] T 8b0368e03f644f1db28dbb57c95c48c1 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:56.242959 1882 raft_consensus.cc:2272] T 8b0368e03f644f1db28dbb57c95c48c1 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.243182 1173 ts_tablet_manager.cc:1434] T bb1005103c25423783736ace1866c328 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.014s user 0.001s sys 0.001s
I20260202 02:52:56.249246 1882 ts_tablet_manager.cc:1916] T 8b0368e03f644f1db28dbb57c95c48c1 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.252713 1173 tablet_bootstrap.cc:492] T 9f11a68d10374d06bc4a756a1d15bf09 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:56.255096 1173 tablet_bootstrap.cc:654] T 9f11a68d10374d06bc4a756a1d15bf09 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:56.255903 1882 ts_tablet_manager.cc:1929] T 8b0368e03f644f1db28dbb57c95c48c1 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:56.256081 1882 log.cc:1199] T 8b0368e03f644f1db28dbb57c95c48c1 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/8b0368e03f644f1db28dbb57c95c48c1
I20260202 02:52:56.262728 1882 ts_tablet_manager.cc:1950] T 8b0368e03f644f1db28dbb57c95c48c1 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.263573 3254 raft_consensus.cc:493] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92a8191cf1e14e4d99f1c378c2d3c85b)
I20260202 02:52:56.264006 3254 raft_consensus.cc:515] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dab2abef14445d688ce1335b1f9b2f8" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 33981 } } peers { permanent_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 46271 } } peers { permanent_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 39077 } }
I20260202 02:52:56.265465 3254 leader_election.cc:290] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7dab2abef14445d688ce1335b1f9b2f8 (127.0.104.190:33981), 92a8191cf1e14e4d99f1c378c2d3c85b (127.0.104.189:46271)
I20260202 02:52:56.265528 2331 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 1176 } } ignore_live_leader: false dest_uuid: "7dab2abef14445d688ce1335b1f9b2f8" is_pre_election: true
I20260202 02:52:56.267745 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 896) took 3387 ms. Trace:
I20260202 02:52:56.267917 1882 rpcz_store.cc:276] 0202 02:52:52.879946 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.879996 (+ 50us) service_pool.cc:225] Handling call
0202 02:52:56.242437 (+3362441us) ts_tablet_manager.cc:1151] Deleting tablet 8b0368e03f644f1db28dbb57c95c48c1
0202 02:52:56.242447 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.252584 (+ 10137us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.255795 (+ 3211us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.267722 (+ 11927us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3362238}
I20260202 02:52:56.268515 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 8b0368e03f644f1db28dbb57c95c48c1 (table default.Table_f8d8615b34774b738faeb83039f9601a [id=8882e1fbac2949838c99adb1290f273e]) successfully deleted
I20260202 02:52:56.269042 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 897) took 3388 ms. Trace:
I20260202 02:52:56.269170 1882 rpcz_store.cc:276] 0202 02:52:52.880572 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.880664 (+ 92us) service_pool.cc:225] Handling call
0202 02:52:56.268904 (+3388240us) ts_tablet_manager.cc:1151] Deleting tablet fa57a267b0484262ac31039862f40a76
0202 02:52:56.268914 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.269021 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3388086}
W20260202 02:52:56.269708 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet fa57a267b0484262ac31039862f40a76 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fa57a267b0484262ac31039862f40a76 already in progress: creating tablet
I20260202 02:52:56.270097 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 898) took 3387 ms. Trace:
I20260202 02:52:56.270205 1882 rpcz_store.cc:276] 0202 02:52:52.882341 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.882385 (+ 44us) service_pool.cc:225] Handling call
0202 02:52:56.269992 (+3387607us) ts_tablet_manager.cc:1151] Deleting tablet efe78242e0654a1c9bdbaf9cfa88b034
0202 02:52:56.270001 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.270082 (+ 81us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3387459}
W20260202 02:52:56.270663 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet efe78242e0654a1c9bdbaf9cfa88b034 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet efe78242e0654a1c9bdbaf9cfa88b034 already in progress: creating tablet
I20260202 02:52:56.271034 1882 tablet_replica.cc:333] T 7709ea34429e4e91ad4922a3d73ebd14 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.271286 1882 raft_consensus.cc:2243] T 7709ea34429e4e91ad4922a3d73ebd14 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:56.271517 1882 raft_consensus.cc:2272] T 7709ea34429e4e91ad4922a3d73ebd14 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.272056 538 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9c3e3612bb94fb7b94c308041acfaeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 1176 } } ignore_live_leader: false dest_uuid: "92a8191cf1e14e4d99f1c378c2d3c85b" is_pre_election: true
I20260202 02:52:56.272292 1882 ts_tablet_manager.cc:1916] T 7709ea34429e4e91ad4922a3d73ebd14 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.272732 1385 leader_election.cc:304] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9c3e3612bb94fb7b94c308041acfaeb; no voters: 7dab2abef14445d688ce1335b1f9b2f8, 92a8191cf1e14e4d99f1c378c2d3c85b
I20260202 02:52:56.276800 3254 raft_consensus.cc:2749] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260202 02:52:56.281306 1882 ts_tablet_manager.cc:1929] T 7709ea34429e4e91ad4922a3d73ebd14 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:56.281570 1882 log.cc:1199] T 7709ea34429e4e91ad4922a3d73ebd14 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/7709ea34429e4e91ad4922a3d73ebd14
I20260202 02:52:56.282049 1882 ts_tablet_manager.cc:1950] T 7709ea34429e4e91ad4922a3d73ebd14 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.282996 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 899) took 3399 ms. Trace:
I20260202 02:52:56.283190 1882 rpcz_store.cc:276] 0202 02:52:52.883710 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.883755 (+ 45us) service_pool.cc:225] Handling call
0202 02:52:56.270980 (+3387225us) ts_tablet_manager.cc:1151] Deleting tablet 7709ea34429e4e91ad4922a3d73ebd14
0202 02:52:56.270990 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.277670 (+ 6680us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.281167 (+ 3497us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.282969 (+ 1802us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3387071}
I20260202 02:52:56.283895 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 7709ea34429e4e91ad4922a3d73ebd14 (table default.Table_a9d14eea240d4bca96de4ea77b5b31f2 [id=c6e5418bf48e43e2adb6d1b60445f0b9]) successfully deleted
I20260202 02:52:56.284392 1882 tablet_replica.cc:333] T 528d4269f91a44679acfad16e9fbbe32 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.284624 1882 raft_consensus.cc:2243] T 528d4269f91a44679acfad16e9fbbe32 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:56.284777 1882 raft_consensus.cc:2272] T 528d4269f91a44679acfad16e9fbbe32 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.285313 1882 ts_tablet_manager.cc:1916] T 528d4269f91a44679acfad16e9fbbe32 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.285463 1173 tablet_bootstrap.cc:492] T 9f11a68d10374d06bc4a756a1d15bf09 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:56.285609 1173 ts_tablet_manager.cc:1403] T 9f11a68d10374d06bc4a756a1d15bf09 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.033s user 0.004s sys 0.012s
I20260202 02:52:56.286399 1173 raft_consensus.cc:359] T 9f11a68d10374d06bc4a756a1d15bf09 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.286625 1173 raft_consensus.cc:385] T 9f11a68d10374d06bc4a756a1d15bf09 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:56.286713 1173 raft_consensus.cc:740] T 9f11a68d10374d06bc4a756a1d15bf09 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:56.286969 1173 consensus_queue.cc:260] T 9f11a68d10374d06bc4a756a1d15bf09 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.294697 1173 ts_tablet_manager.cc:1434] T 9f11a68d10374d06bc4a756a1d15bf09 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260202 02:52:56.296407 3181 raft_consensus.cc:493] T 7f9dfac19c04475aadb45e95a6474eb4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:56.296558 3181 raft_consensus.cc:515] T 7f9dfac19c04475aadb45e95a6474eb4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } }
I20260202 02:52:56.297129 3181 leader_election.cc:290] T 7f9dfac19c04475aadb45e95a6474eb4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:56.299862 998 leader_election.cc:336] T 7f9dfac19c04475aadb45e95a6474eb4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111)
I20260202 02:52:56.307328 1173 tablet_bootstrap.cc:492] T cef87d5f74f14a6d86150958de7b1ace P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:56.307906 1882 ts_tablet_manager.cc:1929] T 528d4269f91a44679acfad16e9fbbe32 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:56.308024 1882 log.cc:1199] T 528d4269f91a44679acfad16e9fbbe32 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/528d4269f91a44679acfad16e9fbbe32
I20260202 02:52:56.308454 1882 ts_tablet_manager.cc:1950] T 528d4269f91a44679acfad16e9fbbe32 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.309187 1173 tablet_bootstrap.cc:654] T cef87d5f74f14a6d86150958de7b1ace P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:56.309237 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 900) took 3425 ms. Trace:
I20260202 02:52:56.309332 1882 rpcz_store.cc:276] 0202 02:52:52.884226 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.884272 (+ 46us) service_pool.cc:225] Handling call
0202 02:52:56.284284 (+3400012us) ts_tablet_manager.cc:1151] Deleting tablet 528d4269f91a44679acfad16e9fbbe32
0202 02:52:56.284298 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.304389 (+ 20091us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.307804 (+ 3415us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.309217 (+ 1413us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3399810}
I20260202 02:52:56.310024 1882 tablet_replica.cc:333] T 1d8143f529b74e6ab779dd695920413c P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.310209 1882 raft_consensus.cc:2243] T 1d8143f529b74e6ab779dd695920413c P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:56.310377 1882 raft_consensus.cc:2272] T 1d8143f529b74e6ab779dd695920413c P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.310914 1882 ts_tablet_manager.cc:1916] T 1d8143f529b74e6ab779dd695920413c P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.317344 1882 ts_tablet_manager.cc:1929] T 1d8143f529b74e6ab779dd695920413c P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:56.317513 1882 log.cc:1199] T 1d8143f529b74e6ab779dd695920413c P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/1d8143f529b74e6ab779dd695920413c
I20260202 02:52:56.317845 1882 ts_tablet_manager.cc:1950] T 1d8143f529b74e6ab779dd695920413c P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.318709 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 901) took 3433 ms. Trace:
I20260202 02:52:56.318815 1882 rpcz_store.cc:276] 0202 02:52:52.884778 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.884823 (+ 45us) service_pool.cc:225] Handling call
0202 02:52:56.309975 (+3425152us) ts_tablet_manager.cc:1151] Deleting tablet 1d8143f529b74e6ab779dd695920413c
0202 02:52:56.309983 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.314048 (+ 4065us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.317214 (+ 3166us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.318681 (+ 1467us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3425007}
I20260202 02:52:56.319376 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 1d8143f529b74e6ab779dd695920413c (table default.Table_33fc32ec03094e3c8fe78de4f6a26777 [id=85340c1716bd4235bdad7042b53ce58c]) successfully deleted
I20260202 02:52:56.319993 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 904) took 3402 ms. Trace:
I20260202 02:52:56.320125 1882 rpcz_store.cc:276] 0202 02:52:52.917419 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.917483 (+ 64us) service_pool.cc:225] Handling call
0202 02:52:56.319831 (+3402348us) ts_tablet_manager.cc:1151] Deleting tablet 3857680ccbdc4dd4ad8247da96254434
0202 02:52:56.319845 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.319973 (+ 128us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3340050}
W20260202 02:52:56.320773 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 3857680ccbdc4dd4ad8247da96254434 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3857680ccbdc4dd4ad8247da96254434 already in progress: creating tablet
I20260202 02:52:56.321415 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 903) took 3404 ms. Trace:
I20260202 02:52:56.321552 1882 rpcz_store.cc:276] 0202 02:52:52.917272 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.917999 (+ 727us) service_pool.cc:225] Handling call
0202 02:52:56.321293 (+3403294us) ts_tablet_manager.cc:1151] Deleting tablet 6f2739c054ce4e6b87634eced940d548
0202 02:52:56.321302 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.321398 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3341489}
W20260202 02:52:56.321969 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 6f2739c054ce4e6b87634eced940d548 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6f2739c054ce4e6b87634eced940d548 already in progress: creating tablet
I20260202 02:52:56.322100 1173 tablet_bootstrap.cc:492] T cef87d5f74f14a6d86150958de7b1ace P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:56.322233 1173 ts_tablet_manager.cc:1403] T cef87d5f74f14a6d86150958de7b1ace P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.003s
I20260202 02:52:56.322261 1882 tablet_replica.cc:333] T 51ba0bcd498043c383c493f45ad1fe17 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.322430 1882 raft_consensus.cc:2243] T 51ba0bcd498043c383c493f45ad1fe17 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:56.322590 1882 raft_consensus.cc:2272] T 51ba0bcd498043c383c493f45ad1fe17 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.323094 1882 ts_tablet_manager.cc:1916] T 51ba0bcd498043c383c493f45ad1fe17 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.323343 1173 raft_consensus.cc:359] T cef87d5f74f14a6d86150958de7b1ace P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.323503 1173 raft_consensus.cc:385] T cef87d5f74f14a6d86150958de7b1ace P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:56.323559 1173 raft_consensus.cc:740] T cef87d5f74f14a6d86150958de7b1ace P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:56.323762 1173 consensus_queue.cc:260] T cef87d5f74f14a6d86150958de7b1ace P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.331403 1173 ts_tablet_manager.cc:1434] T cef87d5f74f14a6d86150958de7b1ace P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260202 02:52:56.340075 1882 ts_tablet_manager.cc:1929] T 51ba0bcd498043c383c493f45ad1fe17 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:56.340215 1882 log.cc:1199] T 51ba0bcd498043c383c493f45ad1fe17 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/51ba0bcd498043c383c493f45ad1fe17
I20260202 02:52:56.340548 1882 ts_tablet_manager.cc:1950] T 51ba0bcd498043c383c493f45ad1fe17 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.341296 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 905) took 3400 ms. Trace:
I20260202 02:52:56.341393 1882 rpcz_store.cc:276] 0202 02:52:52.940511 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.941750 (+ 1239us) service_pool.cc:225] Handling call
0202 02:52:56.322214 (+3380464us) ts_tablet_manager.cc:1151] Deleting tablet 51ba0bcd498043c383c493f45ad1fe17
0202 02:52:56.322224 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.331700 (+ 9476us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.339955 (+ 8255us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.341276 (+ 1321us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3342351}
I20260202 02:52:56.341825 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 906) took 3401 ms. Trace:
I20260202 02:52:56.341907 1882 rpcz_store.cc:276] 0202 02:52:52.940635 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.941963 (+ 1328us) service_pool.cc:225] Handling call
0202 02:52:56.341693 (+3399730us) ts_tablet_manager.cc:1151] Deleting tablet 9ee7687df15d494eaf4150db580de7e3
0202 02:52:56.341703 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.341810 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3361777}
I20260202 02:52:56.342159 1882 tablet_replica.cc:333] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.342346 1882 raft_consensus.cc:2243] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:56.342619 1882 pending_rounds.cc:70] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d: Trying to abort 1 pending ops.
I20260202 02:52:56.342579 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 51ba0bcd498043c383c493f45ad1fe17 (table default.Table_619d2fcf726a46bfb5e939bbe6f55612 [id=038692c44a0043bab69ab29f77aa13be]) successfully deleted
I20260202 02:52:56.342674 1882 pending_rounds.cc:77] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923170313838592 op_type: NO_OP noop_request { }
I20260202 02:52:56.342772 1882 raft_consensus.cc:2889] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:56.342854 1882 raft_consensus.cc:2272] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:56.343262 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 9ee7687df15d494eaf4150db580de7e3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9ee7687df15d494eaf4150db580de7e3 already in progress: creating tablet
I20260202 02:52:56.343411 1882 ts_tablet_manager.cc:1916] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.344058 1173 tablet_bootstrap.cc:492] T 23671408fcbc454cbd918328015cf6e6 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:56.345274 1173 tablet_bootstrap.cc:654] T 23671408fcbc454cbd918328015cf6e6 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:56.349120 1173 tablet_bootstrap.cc:492] T 23671408fcbc454cbd918328015cf6e6 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:56.349256 1173 ts_tablet_manager.cc:1403] T 23671408fcbc454cbd918328015cf6e6 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260202 02:52:56.349821 1173 raft_consensus.cc:359] T 23671408fcbc454cbd918328015cf6e6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.349957 1173 raft_consensus.cc:385] T 23671408fcbc454cbd918328015cf6e6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:56.350006 1173 raft_consensus.cc:740] T 23671408fcbc454cbd918328015cf6e6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:56.350185 1173 consensus_queue.cc:260] T 23671408fcbc454cbd918328015cf6e6 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.350636 1173 ts_tablet_manager.cc:1434] T 23671408fcbc454cbd918328015cf6e6 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260202 02:52:56.352909 1882 ts_tablet_manager.cc:1929] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:56.353017 1882 log.cc:1199] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/7a44585a217b48f39f247bae3ca2e255
I20260202 02:52:56.353309 1882 ts_tablet_manager.cc:1950] T 7a44585a217b48f39f247bae3ca2e255 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.361348 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet 23671408fcbc454cbd918328015cf6e6 with delete_type TABLET_DATA_DELETED (replaced by tablet 32fb916ae82145f195cc8b951069e5da) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:56.363206 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 907) took 3422 ms. Trace:
I20260202 02:52:56.363314 1882 rpcz_store.cc:276] 0202 02:52:52.940731 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.942053 (+ 1322us) service_pool.cc:225] Handling call
0202 02:52:56.342116 (+3400063us) ts_tablet_manager.cc:1151] Deleting tablet 7a44585a217b48f39f247bae3ca2e255
0202 02:52:56.342124 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.350746 (+ 8622us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.352805 (+ 2059us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.363182 (+ 10377us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3362163}
I20260202 02:52:56.363947 1882 tablet_replica.cc:333] T 25a407001f964da6b51f42375136c79a P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.364145 1882 raft_consensus.cc:2243] T 25a407001f964da6b51f42375136c79a P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:56.364295 1882 raft_consensus.cc:2272] T 25a407001f964da6b51f42375136c79a P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.364679 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 7a44585a217b48f39f247bae3ca2e255 (table default.Table_2ee0b536f96e44e4a3a4a9439c7bccc9 [id=d95486b11c494863be7380146ebcf0c8]) successfully deleted
I20260202 02:52:56.364871 1882 ts_tablet_manager.cc:1916] T 25a407001f964da6b51f42375136c79a P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.375255 1173 tablet_bootstrap.cc:492] T a5f864fd8a3d4c4a83e7f68abb8a0a83 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:56.377566 1173 tablet_bootstrap.cc:654] T a5f864fd8a3d4c4a83e7f68abb8a0a83 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:56.383246 1882 ts_tablet_manager.cc:1929] T 25a407001f964da6b51f42375136c79a P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:56.383396 1882 log.cc:1199] T 25a407001f964da6b51f42375136c79a P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/25a407001f964da6b51f42375136c79a
I20260202 02:52:56.384142 1882 ts_tablet_manager.cc:1950] T 25a407001f964da6b51f42375136c79a P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.385262 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 908) took 3443 ms. Trace:
I20260202 02:52:56.385454 1173 tablet_bootstrap.cc:492] T a5f864fd8a3d4c4a83e7f68abb8a0a83 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:56.385352 1882 rpcz_store.cc:276] 0202 02:52:52.941407 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.943208 (+ 1801us) service_pool.cc:225] Handling call
0202 02:52:56.363578 (+3420370us) ts_tablet_manager.cc:1151] Deleting tablet 25a407001f964da6b51f42375136c79a
0202 02:52:56.363588 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.379495 (+ 15907us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.383101 (+ 3606us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.385236 (+ 2135us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3379474}
I20260202 02:52:56.385699 1882 tablet_replica.cc:333] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.385824 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 25a407001f964da6b51f42375136c79a (table default.Table_48226e7c77da41cdba56e04221eb9689 [id=05e52ce8e94e4975b51849059b904272]) successfully deleted
I20260202 02:52:56.385875 1882 raft_consensus.cc:2243] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:56.386015 1173 ts_tablet_manager.cc:1403] T a5f864fd8a3d4c4a83e7f68abb8a0a83 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260202 02:52:56.386129 1882 raft_consensus.cc:2272] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.386680 1882 ts_tablet_manager.cc:1916] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.386677 1173 raft_consensus.cc:359] T a5f864fd8a3d4c4a83e7f68abb8a0a83 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.386883 1173 raft_consensus.cc:385] T a5f864fd8a3d4c4a83e7f68abb8a0a83 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:56.386929 1173 raft_consensus.cc:740] T a5f864fd8a3d4c4a83e7f68abb8a0a83 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:56.387140 1173 consensus_queue.cc:260] T a5f864fd8a3d4c4a83e7f68abb8a0a83 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.396137 1882 ts_tablet_manager.cc:1929] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:56.396281 1882 log.cc:1199] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/e2edc6660b9a41e39e9529627fafcb03
I20260202 02:52:56.396672 1882 ts_tablet_manager.cc:1950] T e2edc6660b9a41e39e9529627fafcb03 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.397411 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet a5f864fd8a3d4c4a83e7f68abb8a0a83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:56.397468 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 909) took 3455 ms. Trace:
I20260202 02:52:56.397526 1882 rpcz_store.cc:276] 0202 02:52:52.941507 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.944522 (+ 3015us) service_pool.cc:225] Handling call
0202 02:52:56.385625 (+3441103us) ts_tablet_manager.cc:1151] Deleting tablet e2edc6660b9a41e39e9529627fafcb03
0202 02:52:56.385635 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.389821 (+ 4186us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.396007 (+ 6186us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.397443 (+ 1436us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3396643}
I20260202 02:52:56.397823 1882 tablet_replica.cc:333] T 75a24a1bf54d4c799938a5723ac6fa99 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.398029 1882 raft_consensus.cc:2243] T 75a24a1bf54d4c799938a5723ac6fa99 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:56.398177 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet e2edc6660b9a41e39e9529627fafcb03 (table default.Table_9d4fbf619567432a88aa9a0c1b19cb3e [id=fff85ebd1bd1415c932f7c9c5a0da42a]) successfully deleted
I20260202 02:52:56.398243 1882 raft_consensus.cc:2272] T 75a24a1bf54d4c799938a5723ac6fa99 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.398456 1173 ts_tablet_manager.cc:1434] T a5f864fd8a3d4c4a83e7f68abb8a0a83 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.012s user 0.001s sys 0.001s
I20260202 02:52:56.398770 1882 ts_tablet_manager.cc:1916] T 75a24a1bf54d4c799938a5723ac6fa99 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.400935 3181 raft_consensus.cc:493] T 3dcf4f794ec14b2f9d181d015fc437fb P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:56.401082 3181 raft_consensus.cc:515] T 3dcf4f794ec14b2f9d181d015fc437fb P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.401631 3181 leader_election.cc:290] T 3dcf4f794ec14b2f9d181d015fc437fb P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:56.403748 998 leader_election.cc:336] T 3dcf4f794ec14b2f9d181d015fc437fb P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111)
I20260202 02:52:56.404875 1882 ts_tablet_manager.cc:1929] T 75a24a1bf54d4c799938a5723ac6fa99 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:56.404978 1882 log.cc:1199] T 75a24a1bf54d4c799938a5723ac6fa99 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/75a24a1bf54d4c799938a5723ac6fa99
I20260202 02:52:56.405274 1882 ts_tablet_manager.cc:1950] T 75a24a1bf54d4c799938a5723ac6fa99 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.405958 1173 tablet_bootstrap.cc:492] T 42321f4e77484027a485fdd584de3ae3 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:56.407575 1173 tablet_bootstrap.cc:654] T 42321f4e77484027a485fdd584de3ae3 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:56.411212 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 911) took 3469 ms. Trace:
I20260202 02:52:56.411325 1882 rpcz_store.cc:276] 0202 02:52:52.941691 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.944665 (+ 2974us) service_pool.cc:225] Handling call
0202 02:52:56.397779 (+3453114us) ts_tablet_manager.cc:1151] Deleting tablet 75a24a1bf54d4c799938a5723ac6fa99
0202 02:52:56.397789 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.401940 (+ 4151us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.404767 (+ 2827us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.411185 (+ 6418us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":34,"tablet-delete.queue_time_us":3408725}
I20260202 02:52:56.411765 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 912) took 3454 ms. Trace:
I20260202 02:52:56.411805 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 75a24a1bf54d4c799938a5723ac6fa99 (table default.Table_52a2f389faba40fea93e2bca972e81e1 [id=70ac373200604907b00132eb4727ea5e]) successfully deleted
I20260202 02:52:56.411842 1882 rpcz_store.cc:276] 0202 02:52:52.957622 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:52.957679 (+ 57us) service_pool.cc:225] Handling call
0202 02:52:56.411628 (+3453949us) ts_tablet_manager.cc:1151] Deleting tablet 08fbd094d4df46cf82a694b4e1f30c6d
0202 02:52:56.411638 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.411749 (+ 111us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3422436}
I20260202 02:52:56.412142 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 916) took 3371 ms. Trace:
I20260202 02:52:56.412220 1882 rpcz_store.cc:276] 0202 02:52:53.040487 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:53.040543 (+ 56us) service_pool.cc:225] Handling call
0202 02:52:56.412049 (+3371506us) ts_tablet_manager.cc:1151] Deleting tablet 81abd5ef66cc4f0bb37844c6ff5c256d
0202 02:52:56.412056 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.412128 (+ 72us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3371075}
I20260202 02:52:56.412528 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 918) took 3206 ms. Trace:
I20260202 02:52:56.412595 1882 rpcz_store.cc:276] 0202 02:52:53.206442 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:53.206496 (+ 54us) service_pool.cc:225] Handling call
0202 02:52:56.412430 (+3205934us) ts_tablet_manager.cc:1151] Deleting tablet d3f6a30e08244a07a1f81b5e98bc6965
0202 02:52:56.412438 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.412513 (+ 75us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3205075}
W20260202 02:52:56.412679 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 08fbd094d4df46cf82a694b4e1f30c6d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 08fbd094d4df46cf82a694b4e1f30c6d already in progress: creating tablet
I20260202 02:52:56.412853 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 917) took 3207 ms. Trace:
W20260202 02:52:56.412911 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 81abd5ef66cc4f0bb37844c6ff5c256d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 81abd5ef66cc4f0bb37844c6ff5c256d already in progress: creating tablet
I20260202 02:52:56.412907 1882 rpcz_store.cc:276] 0202 02:52:53.205068 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:53.206115 (+ 1047us) service_pool.cc:225] Handling call
0202 02:52:56.412773 (+3206658us) ts_tablet_manager.cc:1151] Deleting tablet 477c599f8c9b4eb0b435ea50abd5d36b
0202 02:52:56.412780 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.412840 (+ 60us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3188819}
I20260202 02:52:56.413142 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 920) took 3170 ms. Trace:
I20260202 02:52:56.413200 1882 rpcz_store.cc:276] 0202 02:52:53.242625 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:53.242689 (+ 64us) service_pool.cc:225] Handling call
0202 02:52:56.413069 (+3170380us) ts_tablet_manager.cc:1151] Deleting tablet 137dae39f8f54be5af6354e2b6226adc
0202 02:52:56.413075 (+ 6us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.413131 (+ 56us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3169421}
I20260202 02:52:56.413403 1882 tablet_replica.cc:333] T bb1005103c25423783736ace1866c328 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.413604 1882 raft_consensus.cc:2243] T bb1005103c25423783736ace1866c328 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
W20260202 02:52:56.413604 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet d3f6a30e08244a07a1f81b5e98bc6965 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d3f6a30e08244a07a1f81b5e98bc6965 already in progress: creating tablet
W20260202 02:52:56.413798 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 477c599f8c9b4eb0b435ea50abd5d36b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 477c599f8c9b4eb0b435ea50abd5d36b already in progress: creating tablet
I20260202 02:52:56.413887 1882 raft_consensus.cc:2272] T bb1005103c25423783736ace1866c328 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:56.413973 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 137dae39f8f54be5af6354e2b6226adc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 137dae39f8f54be5af6354e2b6226adc already in progress: creating tablet
I20260202 02:52:56.415073 1882 ts_tablet_manager.cc:1916] T bb1005103c25423783736ace1866c328 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.418192 1173 tablet_bootstrap.cc:492] T 42321f4e77484027a485fdd584de3ae3 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:56.418340 1173 ts_tablet_manager.cc:1403] T 42321f4e77484027a485fdd584de3ae3 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.004s
I20260202 02:52:56.419014 1173 raft_consensus.cc:359] T 42321f4e77484027a485fdd584de3ae3 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.419178 1173 raft_consensus.cc:385] T 42321f4e77484027a485fdd584de3ae3 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:56.419225 1173 raft_consensus.cc:740] T 42321f4e77484027a485fdd584de3ae3 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:56.419420 1173 consensus_queue.cc:260] T 42321f4e77484027a485fdd584de3ae3 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.421114 1882 ts_tablet_manager.cc:1929] T bb1005103c25423783736ace1866c328 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:56.421226 1882 log.cc:1199] T bb1005103c25423783736ace1866c328 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/bb1005103c25423783736ace1866c328
I20260202 02:52:56.421530 1882 ts_tablet_manager.cc:1950] T bb1005103c25423783736ace1866c328 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.422222 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 921) took 3095 ms. Trace:
I20260202 02:52:56.422313 1882 rpcz_store.cc:276] 0202 02:52:53.327242 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:53.327488 (+ 246us) service_pool.cc:225] Handling call
0202 02:52:56.413364 (+3085876us) ts_tablet_manager.cc:1151] Deleting tablet bb1005103c25423783736ace1866c328
0202 02:52:56.413371 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.418213 (+ 4842us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.420995 (+ 2782us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.422202 (+ 1207us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3085641}
I20260202 02:52:56.422842 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet bb1005103c25423783736ace1866c328 (table default.Table_dfa22e5a08544023b64919ccf651c07a [id=15a0348866d34a209fb8332e17c91ae0]) successfully deleted
I20260202 02:52:56.423038 1882 tablet_replica.cc:333] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.423226 1882 raft_consensus.cc:2243] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:56.423357 1882 raft_consensus.cc:2272] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.424052 1882 ts_tablet_manager.cc:1916] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.426225 1173 ts_tablet_manager.cc:1434] T 42321f4e77484027a485fdd584de3ae3 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260202 02:52:56.426517 1173 tablet_bootstrap.cc:492] T e33f95e4f17a479c9acef5764c6c4c56 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:56.427856 1173 tablet_bootstrap.cc:654] T e33f95e4f17a479c9acef5764c6c4c56 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:56.431875 3181 raft_consensus.cc:493] T 67a8c9a1998b47b3b214a06ce62381e9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:56.432062 3181 raft_consensus.cc:515] T 67a8c9a1998b47b3b214a06ce62381e9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.432736 3181 leader_election.cc:290] T 67a8c9a1998b47b3b214a06ce62381e9 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:56.434051 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet 42321f4e77484027a485fdd584de3ae3 with delete_type TABLET_DATA_DELETED (replaced by tablet fea0fcb8854c4fc6acda80c8ffb8d341) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:56.437602 998 leader_election.cc:336] T 67a8c9a1998b47b3b214a06ce62381e9 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111)
W20260202 02:52:56.438738 998 consensus_peers.cc:597] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d -> Peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Couldn't send request to peer 1f442ef85029461091de847118c34c0d. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260202 02:52:56.438810 1882 ts_tablet_manager.cc:1929] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:56.438898 1882 log.cc:1199] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/f1c8f7e1dc9644a6b6dd2bc264cce8ef
I20260202 02:52:56.439178 1882 ts_tablet_manager.cc:1950] T f1c8f7e1dc9644a6b6dd2bc264cce8ef P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.439905 3181 raft_consensus.cc:493] T 842197c689064660b791b40e521d346b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:56.439906 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 925) took 3056 ms. Trace:
I20260202 02:52:56.440078 3181 raft_consensus.cc:515] T 842197c689064660b791b40e521d346b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.440045 1882 rpcz_store.cc:276] 0202 02:52:53.383414 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:53.383478 (+ 64us) service_pool.cc:225] Handling call
0202 02:52:56.422591 (+3039113us) ts_tablet_manager.cc:1151] Deleting tablet f1c8f7e1dc9644a6b6dd2bc264cce8ef
0202 02:52:56.422998 (+ 407us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.426928 (+ 3930us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.438700 (+ 11772us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.439886 (+ 1186us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3038842}
I20260202 02:52:56.440469 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 927) took 3056 ms. Trace:
I20260202 02:52:56.440589 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet f1c8f7e1dc9644a6b6dd2bc264cce8ef (table default.Table_66adee49a1b045918360d13af6bca9a2 [id=4096475017214ffdba27d20aca4e8d64]) successfully deleted
I20260202 02:52:56.440548 1882 rpcz_store.cc:276] 0202 02:52:53.383662 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:53.383786 (+ 124us) service_pool.cc:225] Handling call
0202 02:52:56.440347 (+3056561us) ts_tablet_manager.cc:1151] Deleting tablet 80eb3013f5354022a2dd78bb294a8497
0202 02:52:56.440358 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.440452 (+ 94us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3056349}
I20260202 02:52:56.440810 1882 tablet_replica.cc:333] T 07580ad1d049475a9645acfad0b1d219 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.440982 1882 raft_consensus.cc:2243] T 07580ad1d049475a9645acfad0b1d219 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:56.441120 1882 raft_consensus.cc:2272] T 07580ad1d049475a9645acfad0b1d219 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:56.441108 515 catalog_manager.cc:4952] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 80eb3013f5354022a2dd78bb294a8497 because the tablet was not found. No further retry: Not found: Tablet not found: 80eb3013f5354022a2dd78bb294a8497
I20260202 02:52:56.441653 1882 ts_tablet_manager.cc:1916] T 07580ad1d049475a9645acfad0b1d219 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.441708 3181 leader_election.cc:290] T 842197c689064660b791b40e521d346b P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:56.443039 998 leader_election.cc:336] T 842197c689064660b791b40e521d346b P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111)
I20260202 02:52:56.448097 1882 ts_tablet_manager.cc:1929] T 07580ad1d049475a9645acfad0b1d219 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:56.448258 1882 log.cc:1199] T 07580ad1d049475a9645acfad0b1d219 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/07580ad1d049475a9645acfad0b1d219
I20260202 02:52:56.448629 1882 ts_tablet_manager.cc:1950] T 07580ad1d049475a9645acfad0b1d219 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
W20260202 02:52:56.449462 515 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 16b2d80274a54f1193a3f1746c278252 (table default.Table_d01f7dfd99d0405a896f6fbfd581dfd2 [id=163eefc3d38a4b28bce88b948cc5df3c], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet 16b2d80274a54f1193a3f1746c278252 consensus state has no leader
I20260202 02:52:56.449487 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 929) took 3065 ms. Trace:
I20260202 02:52:56.449594 1882 rpcz_store.cc:276] 0202 02:52:53.383895 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:53.383929 (+ 34us) service_pool.cc:225] Handling call
0202 02:52:56.440766 (+3056837us) ts_tablet_manager.cc:1151] Deleting tablet 07580ad1d049475a9645acfad0b1d219
0202 02:52:56.440774 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.444598 (+ 3824us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.447953 (+ 3355us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.449461 (+ 1508us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3056677}
I20260202 02:52:56.449985 1882 tablet_replica.cc:333] T 0138ba7326674741b0729324735f1de7 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.450178 1882 raft_consensus.cc:2243] T 0138ba7326674741b0729324735f1de7 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:56.450204 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 07580ad1d049475a9645acfad0b1d219 (table default.Table_444485558ccc4c6ba771ab686bc9e1c1 [id=c4207a758ebb4784a67c662cfa4333d2]) successfully deleted
I20260202 02:52:56.450345 1882 raft_consensus.cc:2272] T 0138ba7326674741b0729324735f1de7 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.450960 1882 ts_tablet_manager.cc:1916] T 0138ba7326674741b0729324735f1de7 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.452279 3181 raft_consensus.cc:493] T 2f664f6119ba4d28b5ef1c212e611866 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:56.452446 3181 raft_consensus.cc:515] T 2f664f6119ba4d28b5ef1c212e611866 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.453342 1173 tablet_bootstrap.cc:492] T e33f95e4f17a479c9acef5764c6c4c56 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:56.453470 1173 ts_tablet_manager.cc:1403] T e33f95e4f17a479c9acef5764c6c4c56 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.027s user 0.005s sys 0.009s
I20260202 02:52:56.454255 3181 leader_election.cc:290] T 2f664f6119ba4d28b5ef1c212e611866 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:56.454909 998 leader_election.cc:336] T 2f664f6119ba4d28b5ef1c212e611866 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111)
I20260202 02:52:56.455016 1173 raft_consensus.cc:359] T e33f95e4f17a479c9acef5764c6c4c56 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.455176 1173 raft_consensus.cc:385] T e33f95e4f17a479c9acef5764c6c4c56 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:56.455224 1173 raft_consensus.cc:740] T e33f95e4f17a479c9acef5764c6c4c56 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:56.455403 1173 consensus_queue.cc:260] T e33f95e4f17a479c9acef5764c6c4c56 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.455845 1173 ts_tablet_manager.cc:1434] T e33f95e4f17a479c9acef5764c6c4c56 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260202 02:52:56.457194 1882 ts_tablet_manager.cc:1929] T 0138ba7326674741b0729324735f1de7 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:56.457293 1882 log.cc:1199] T 0138ba7326674741b0729324735f1de7 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/0138ba7326674741b0729324735f1de7
I20260202 02:52:56.457582 1882 ts_tablet_manager.cc:1950] T 0138ba7326674741b0729324735f1de7 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.458422 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 928) took 3074 ms. Trace:
I20260202 02:52:56.458520 1882 rpcz_store.cc:276] 0202 02:52:53.383767 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:53.384009 (+ 242us) service_pool.cc:225] Handling call
0202 02:52:56.449926 (+3065917us) ts_tablet_manager.cc:1151] Deleting tablet 0138ba7326674741b0729324735f1de7
0202 02:52:56.449939 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.454577 (+ 4638us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.457107 (+ 2530us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.458398 (+ 1291us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2304,"tablet-delete.queue_time_us":3065741}
I20260202 02:52:56.458920 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 930) took 3025 ms. Trace:
I20260202 02:52:56.458984 1882 rpcz_store.cc:276] 0202 02:52:53.433712 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:53.433791 (+ 79us) service_pool.cc:225] Handling call
0202 02:52:56.458801 (+3025010us) ts_tablet_manager.cc:1151] Deleting tablet 2becb2f65a124c3990ea3a8318ff1d8c
0202 02:52:56.458810 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.458904 (+ 94us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3024776}
I20260202 02:52:56.459205 1882 tablet_replica.cc:333] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.459381 1882 raft_consensus.cc:2243] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:56.459648 1882 pending_rounds.cc:70] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d: Trying to abort 1 pending ops.
I20260202 02:52:56.459688 1882 pending_rounds.cc:77] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923169517125632 op_type: NO_OP noop_request { }
I20260202 02:52:56.459771 1882 raft_consensus.cc:2889] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:56.459846 1882 raft_consensus.cc:2272] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.460454 1882 ts_tablet_manager.cc:1916] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.462570 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 0138ba7326674741b0729324735f1de7 (table default.Table_516d610640654ea4b49f7de3b04cbb0c [id=5c8367fbb78840d18a26a026d75c3d6b]) successfully deleted
W20260202 02:52:56.462848 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 2becb2f65a124c3990ea3a8318ff1d8c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2becb2f65a124c3990ea3a8318ff1d8c already in progress: creating tablet
I20260202 02:52:56.465901 1173 tablet_bootstrap.cc:492] T 916f52df8e0a4838b46772ecab4f852c P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:56.466270 1882 ts_tablet_manager.cc:1929] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:56.466367 1882 log.cc:1199] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/2616e1bbce4049dbb584550ccc150288
I20260202 02:52:56.466728 1882 ts_tablet_manager.cc:1950] T 2616e1bbce4049dbb584550ccc150288 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.467569 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 931) took 3024 ms. Trace:
I20260202 02:52:56.467712 1882 rpcz_store.cc:276] 0202 02:52:53.442845 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:53.442907 (+ 62us) service_pool.cc:225] Handling call
0202 02:52:56.459169 (+3016262us) ts_tablet_manager.cc:1151] Deleting tablet 2616e1bbce4049dbb584550ccc150288
0202 02:52:56.459176 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.463335 (+ 4159us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.466169 (+ 2834us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.467549 (+ 1380us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3016029}
I20260202 02:52:56.468086 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 940) took 2680 ms. Trace:
I20260202 02:52:56.468150 1882 rpcz_store.cc:276] 0202 02:52:53.787381 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:53.788153 (+ 772us) service_pool.cc:225] Handling call
0202 02:52:56.467971 (+2679818us) ts_tablet_manager.cc:1151] Deleting tablet 8cc5e23a86a3467c89794ae8c403230e
0202 02:52:56.467980 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.468071 (+ 91us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2679559}
I20260202 02:52:56.468411 1173 tablet_bootstrap.cc:654] T 916f52df8e0a4838b46772ecab4f852c P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:56.468428 1882 tablet_replica.cc:333] T cef87d5f74f14a6d86150958de7b1ace P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.468606 1882 raft_consensus.cc:2243] T cef87d5f74f14a6d86150958de7b1ace P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:56.468752 1882 raft_consensus.cc:2272] T cef87d5f74f14a6d86150958de7b1ace P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.468748 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 2616e1bbce4049dbb584550ccc150288 (table default.Table_4fb44d5cbe71439ba073bd75711e77e0 [id=8c2d0f27f5d04a98bfad01483c783ecb]) successfully deleted
W20260202 02:52:56.469010 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 8cc5e23a86a3467c89794ae8c403230e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8cc5e23a86a3467c89794ae8c403230e already in progress: creating tablet
I20260202 02:52:56.469599 1882 ts_tablet_manager.cc:1916] T cef87d5f74f14a6d86150958de7b1ace P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.475966 1882 ts_tablet_manager.cc:1929] T cef87d5f74f14a6d86150958de7b1ace P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:56.476112 1882 log.cc:1199] T cef87d5f74f14a6d86150958de7b1ace P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/cef87d5f74f14a6d86150958de7b1ace
I20260202 02:52:56.476447 1882 ts_tablet_manager.cc:1950] T cef87d5f74f14a6d86150958de7b1ace P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.477241 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 957) took 2111 ms. Trace:
I20260202 02:52:56.477332 1882 rpcz_store.cc:276] 0202 02:52:54.365747 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:54.365831 (+ 84us) service_pool.cc:225] Handling call
0202 02:52:56.468388 (+2102557us) ts_tablet_manager.cc:1151] Deleting tablet cef87d5f74f14a6d86150958de7b1ace
0202 02:52:56.468397 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.471890 (+ 3493us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.475840 (+ 3950us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.477219 (+ 1379us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2102281}
I20260202 02:52:56.477701 1882 tablet_replica.cc:333] T 1959438c70c544e69422f7354082cf89 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.477887 1882 raft_consensus.cc:2243] T 1959438c70c544e69422f7354082cf89 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:56.478027 1882 raft_consensus.cc:2272] T 1959438c70c544e69422f7354082cf89 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.478530 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet cef87d5f74f14a6d86150958de7b1ace (table default.Table_6831562d21db463986dc58b0100e9792 [id=50e58db611f247ee9dcd5b31776ace05]) successfully deleted
I20260202 02:52:56.478603 1882 ts_tablet_manager.cc:1916] T 1959438c70c544e69422f7354082cf89 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.484018 857 leader_election.cc:304] T c9ef08b5830147a3a55049285a40535b P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1521a6d262b7451bbef8debe970dd6b5; no voters: 1f442ef85029461091de847118c34c0d, b1f05820b6034228bdad884cc348716d
W20260202 02:52:56.484203 963 rpcz_store.cc:267] Call kudu.server.GenericService.CheckLeaks from 127.0.0.1:41734 (request call id 174) took 871 ms (client timeout 1000 ms). Trace:
W20260202 02:52:56.484303 963 rpcz_store.cc:269] 0202 02:52:55.612635 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:55.614758 (+ 2123us) service_pool.cc:225] Handling call
0202 02:52:56.484171 (+869413us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:56.485471 3233 raft_consensus.cc:493] T 3338964dfed548f980f58147f0283691 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:56.485599 3233 raft_consensus.cc:515] T 3338964dfed548f980f58147f0283691 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.486479 3233 leader_election.cc:290] T 3338964dfed548f980f58147f0283691 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:56.486591 3233 raft_consensus.cc:493] T 42321f4e77484027a485fdd584de3ae3 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:56.486711 3233 raft_consensus.cc:515] T 42321f4e77484027a485fdd584de3ae3 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.487373 3233 leader_election.cc:290] T 42321f4e77484027a485fdd584de3ae3 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:56.487586 3233 raft_consensus.cc:2749] T c9ef08b5830147a3a55049285a40535b P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:56.488358 3080 leader_election.cc:290] T bb1005103c25423783736ace1866c328 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
W20260202 02:52:56.488482 856 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.845s user 0.000s sys 0.000s
W20260202 02:52:56.488533 856 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.845s user 0.000s sys 0.000s
I20260202 02:52:56.490029 1882 ts_tablet_manager.cc:1929] T 1959438c70c544e69422f7354082cf89 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:56.490159 1882 log.cc:1199] T 1959438c70c544e69422f7354082cf89 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/1959438c70c544e69422f7354082cf89
I20260202 02:52:56.490512 1882 ts_tablet_manager.cc:1950] T 1959438c70c544e69422f7354082cf89 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.491287 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 958) took 2125 ms. Trace:
I20260202 02:52:56.491385 1882 rpcz_store.cc:276] 0202 02:52:54.365827 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:54.366143 (+ 316us) service_pool.cc:225] Handling call
0202 02:52:56.477649 (+2111506us) ts_tablet_manager.cc:1151] Deleting tablet 1959438c70c544e69422f7354082cf89
0202 02:52:56.477660 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.481903 (+ 4243us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.486413 (+ 4510us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.491266 (+ 4853us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2111360}
I20260202 02:52:56.491789 1882 tablet_replica.cc:333] T abfe7f76b2424c4d9ab07b29b66ec103 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.491921 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 1959438c70c544e69422f7354082cf89 (table default.Table_eeb507a1a49d407196bff72270b0834f [id=a18136bbb87a47d69600f926a474da24]) successfully deleted
I20260202 02:52:56.491966 1882 raft_consensus.cc:2243] T abfe7f76b2424c4d9ab07b29b66ec103 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:56.492113 1882 raft_consensus.cc:2272] T abfe7f76b2424c4d9ab07b29b66ec103 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.492671 1882 ts_tablet_manager.cc:1916] T abfe7f76b2424c4d9ab07b29b66ec103 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.492816 1173 tablet_bootstrap.cc:492] T 916f52df8e0a4838b46772ecab4f852c P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:56.492937 1173 ts_tablet_manager.cc:1403] T 916f52df8e0a4838b46772ecab4f852c P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.027s user 0.000s sys 0.002s
I20260202 02:52:56.493587 1173 raft_consensus.cc:359] T 916f52df8e0a4838b46772ecab4f852c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.493765 1173 raft_consensus.cc:385] T 916f52df8e0a4838b46772ecab4f852c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:56.493819 1173 raft_consensus.cc:740] T 916f52df8e0a4838b46772ecab4f852c P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:56.494009 1173 consensus_queue.cc:260] T 916f52df8e0a4838b46772ecab4f852c P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.494444 1173 ts_tablet_manager.cc:1434] T 916f52df8e0a4838b46772ecab4f852c P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260202 02:52:56.494701 1173 tablet_bootstrap.cc:492] T 4e90f273439d414caf16d63e1c605d3d P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:56.495986 1173 tablet_bootstrap.cc:654] T 4e90f273439d414caf16d63e1c605d3d P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:56.496218 3080 raft_consensus.cc:493] T 916f52df8e0a4838b46772ecab4f852c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:56.496362 3080 raft_consensus.cc:515] T 916f52df8e0a4838b46772ecab4f852c P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.498422 1882 ts_tablet_manager.cc:1929] T abfe7f76b2424c4d9ab07b29b66ec103 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:56.498535 1882 log.cc:1199] T abfe7f76b2424c4d9ab07b29b66ec103 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/abfe7f76b2424c4d9ab07b29b66ec103
I20260202 02:52:56.498818 1882 ts_tablet_manager.cc:1950] T abfe7f76b2424c4d9ab07b29b66ec103 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.499562 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 961) took 2133 ms. Trace:
I20260202 02:52:56.499641 1882 rpcz_store.cc:276] 0202 02:52:54.366030 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:54.366137 (+ 107us) service_pool.cc:225] Handling call
0202 02:52:56.491743 (+2125606us) ts_tablet_manager.cc:1151] Deleting tablet abfe7f76b2424c4d9ab07b29b66ec103
0202 02:52:56.491751 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.495514 (+ 3763us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.498310 (+ 2796us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.499541 (+ 1231us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2125363}
I20260202 02:52:56.499951 1882 tablet_replica.cc:333] T 7f323175ee254df789bcb04e65047dff P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.500114 1882 raft_consensus.cc:2243] T 7f323175ee254df789bcb04e65047dff P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:56.500248 1882 raft_consensus.cc:2272] T 7f323175ee254df789bcb04e65047dff P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.500519 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet abfe7f76b2424c4d9ab07b29b66ec103 (table default.Table_ff54266e8b5a4334ad1fbeda5270f0f2 [id=b6c09db68b4542a3bb11bf20b1d54571]) successfully deleted
I20260202 02:52:56.500730 3080 leader_election.cc:290] T 916f52df8e0a4838b46772ecab4f852c P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:56.500810 1882 ts_tablet_manager.cc:1916] T 7f323175ee254df789bcb04e65047dff P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.500866 3080 raft_consensus.cc:493] T a5f864fd8a3d4c4a83e7f68abb8a0a83 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:56.500962 3080 raft_consensus.cc:515] T a5f864fd8a3d4c4a83e7f68abb8a0a83 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.501895 3080 leader_election.cc:290] T a5f864fd8a3d4c4a83e7f68abb8a0a83 P 1521a6d262b7451bbef8debe970dd6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), b1f05820b6034228bdad884cc348716d (127.0.104.131:35257)
I20260202 02:52:56.506930 1882 ts_tablet_manager.cc:1929] T 7f323175ee254df789bcb04e65047dff P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:56.507063 1882 log.cc:1199] T 7f323175ee254df789bcb04e65047dff P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/7f323175ee254df789bcb04e65047dff
I20260202 02:52:56.507373 1882 ts_tablet_manager.cc:1950] T 7f323175ee254df789bcb04e65047dff P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.508723 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 965) took 2142 ms. Trace:
I20260202 02:52:56.508809 1882 rpcz_store.cc:276] 0202 02:52:54.366301 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:54.366396 (+ 95us) service_pool.cc:225] Handling call
0202 02:52:56.499907 (+2133511us) ts_tablet_manager.cc:1151] Deleting tablet 7f323175ee254df789bcb04e65047dff
0202 02:52:56.499917 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.503822 (+ 3905us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.506814 (+ 2992us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.508700 (+ 1886us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2133392}
I20260202 02:52:56.509150 1882 tablet_replica.cc:333] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.509334 1882 raft_consensus.cc:2243] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:56.509629 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 7f323175ee254df789bcb04e65047dff (table default.Table_c08aceb74c164c06a3a149c454c9c59e [id=0d6674bc4f264a36aad4ea7962a9e0e7]) successfully deleted
I20260202 02:52:56.511222 1173 tablet_bootstrap.cc:492] T 4e90f273439d414caf16d63e1c605d3d P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:56.511359 1173 ts_tablet_manager.cc:1403] T 4e90f273439d414caf16d63e1c605d3d P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.002s
I20260202 02:52:56.512050 1173 raft_consensus.cc:359] T 4e90f273439d414caf16d63e1c605d3d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.512226 1173 raft_consensus.cc:385] T 4e90f273439d414caf16d63e1c605d3d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:56.512271 1173 raft_consensus.cc:740] T 4e90f273439d414caf16d63e1c605d3d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:56.512542 1882 pending_rounds.cc:70] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d: Trying to abort 1 pending ops.
I20260202 02:52:56.512588 1882 pending_rounds.cc:77] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923171469762560 op_type: NO_OP noop_request { }
I20260202 02:52:56.512660 1882 raft_consensus.cc:2889] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:56.512750 1882 raft_consensus.cc:2272] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.512647 1173 consensus_queue.cc:260] T 4e90f273439d414caf16d63e1c605d3d P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.513123 1173 ts_tablet_manager.cc:1434] T 4e90f273439d414caf16d63e1c605d3d P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:52:56.513267 1882 ts_tablet_manager.cc:1916] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.513384 1173 tablet_bootstrap.cc:492] T 3338964dfed548f980f58147f0283691 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:56.514714 1173 tablet_bootstrap.cc:654] T 3338964dfed548f980f58147f0283691 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:56.520639 885 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41736 (request call id 1071) took 1009 ms. Trace:
I20260202 02:52:56.520753 885 rpcz_store.cc:276] 0202 02:52:55.511527 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:55.511577 (+ 50us) service_pool.cc:225] Handling call
0202 02:52:55.512029 (+ 452us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0202 02:52:55.512066 (+ 37us) ts_tablet_manager.cc:682] Creating new metadata...
0202 02:52:55.645770 (+133704us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.520475 (+874705us) tablet_replica.cc:206] Creating consensus instance
0202 02:52:56.520613 (+ 138us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260202 02:52:56.531138 1882 ts_tablet_manager.cc:1929] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:56.531304 1882 log.cc:1199] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/59ad016e15ec4a488921eb564029b689
I20260202 02:52:56.531705 1882 ts_tablet_manager.cc:1950] T 59ad016e15ec4a488921eb564029b689 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.533272 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 967) took 2166 ms. Trace:
I20260202 02:52:56.533398 1882 rpcz_store.cc:276] 0202 02:52:54.366428 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:54.366538 (+ 110us) service_pool.cc:225] Handling call
0202 02:52:56.509103 (+2142565us) ts_tablet_manager.cc:1151] Deleting tablet 59ad016e15ec4a488921eb564029b689
0202 02:52:56.509113 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.516113 (+ 7000us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.530993 (+ 14880us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.533239 (+ 2246us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2142411}
I20260202 02:52:56.533807 1882 tablet_replica.cc:333] T 9f11a68d10374d06bc4a756a1d15bf09 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.534006 1882 raft_consensus.cc:2243] T 9f11a68d10374d06bc4a756a1d15bf09 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:56.533994 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 59ad016e15ec4a488921eb564029b689 (table default.Table_49cb996504bf45e9a24c6f3c01f457ff [id=ec1d1bc3e38643f7ad1c57e98ef66388]) successfully deleted
I20260202 02:52:56.534157 1882 raft_consensus.cc:2272] T 9f11a68d10374d06bc4a756a1d15bf09 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.534938 1882 ts_tablet_manager.cc:1916] T 9f11a68d10374d06bc4a756a1d15bf09 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.536069 1182 ts_tablet_manager.cc:1929] T 5e20842de8cb4355b732685605c88834 P 1521a6d262b7451bbef8debe970dd6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:56.536228 1182 log.cc:1199] T 5e20842de8cb4355b732685605c88834 P 1521a6d262b7451bbef8debe970dd6b5: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/5e20842de8cb4355b732685605c88834
I20260202 02:52:56.536592 1182 ts_tablet_manager.cc:1950] T 5e20842de8cb4355b732685605c88834 P 1521a6d262b7451bbef8debe970dd6b5: Deleting consensus metadata
I20260202 02:52:56.537396 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 926) took 3502 ms. Trace:
I20260202 02:52:56.537604 1182 rpcz_store.cc:276] 0202 02:52:53.035464 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:53.035534 (+ 70us) service_pool.cc:225] Handling call
0202 02:52:55.605929 (+2570395us) ts_tablet_manager.cc:1151] Deleting tablet 5e20842de8cb4355b732685605c88834
0202 02:52:55.605938 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.514016 (+908078us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.535949 (+ 21933us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.537373 (+ 1424us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2570178}
I20260202 02:52:56.538175 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 928) took 3429 ms. Trace:
I20260202 02:52:56.538525 1182 rpcz_store.cc:276] 0202 02:52:53.108447 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:53.108517 (+ 70us) service_pool.cc:225] Handling call
0202 02:52:56.538032 (+3429515us) ts_tablet_manager.cc:1151] Deleting tablet 477c599f8c9b4eb0b435ea50abd5d36b
0202 02:52:56.538042 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.538159 (+ 117us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3429293}
I20260202 02:52:56.539326 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 929) took 3384 ms. Trace:
I20260202 02:52:56.540167 1182 rpcz_store.cc:276] 0202 02:52:53.155364 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:53.155435 (+ 71us) service_pool.cc:225] Handling call
0202 02:52:56.539194 (+3383759us) ts_tablet_manager.cc:1151] Deleting tablet d3f6a30e08244a07a1f81b5e98bc6965
0202 02:52:56.539204 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.539310 (+ 106us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3383518}
I20260202 02:52:56.541193 1182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41736 (request call id 930) took 3307 ms. Trace:
I20260202 02:52:56.541883 1882 ts_tablet_manager.cc:1929] T 9f11a68d10374d06bc4a756a1d15bf09 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:56.542009 1882 log.cc:1199] T 9f11a68d10374d06bc4a756a1d15bf09 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/9f11a68d10374d06bc4a756a1d15bf09
I20260202 02:52:56.542313 1182 rpcz_store.cc:276] 0202 02:52:53.233788 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:53.233847 (+ 59us) service_pool.cc:225] Handling call
0202 02:52:56.541065 (+3307218us) ts_tablet_manager.cc:1151] Deleting tablet 137dae39f8f54be5af6354e2b6226adc
0202 02:52:56.541076 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.541176 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3307032}
I20260202 02:52:56.542455 1882 ts_tablet_manager.cc:1950] T 9f11a68d10374d06bc4a756a1d15bf09 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.543238 1182 tablet_replica.cc:333] T b9fc33e8efc04734a951f7fcf9048142 P 1521a6d262b7451bbef8debe970dd6b5: stopping tablet replica
I20260202 02:52:56.543255 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 959) took 2177 ms. Trace:
I20260202 02:52:56.543350 1882 rpcz_store.cc:276] 0202 02:52:54.365893 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:54.366600 (+ 707us) service_pool.cc:225] Handling call
0202 02:52:56.533752 (+2167152us) ts_tablet_manager.cc:1151] Deleting tablet 9f11a68d10374d06bc4a756a1d15bf09
0202 02:52:56.533765 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.538366 (+ 4601us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.541758 (+ 3392us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.543235 (+ 1477us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2166945}
I20260202 02:52:56.543681 1882 tablet_replica.cc:333] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.543871 1882 raft_consensus.cc:2243] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:56.543833 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 9f11a68d10374d06bc4a756a1d15bf09 (table default.Table_509184aa45184b7c8228aba2de1c528a [id=d3627805d70e4e269f073250e883f3e3]) successfully deleted
I20260202 02:52:56.544118 1882 pending_rounds.cc:70] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d: Trying to abort 1 pending ops.
I20260202 02:52:56.544170 1882 pending_rounds.cc:77] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923176167763968 op_type: NO_OP noop_request { }
I20260202 02:52:56.544270 1882 raft_consensus.cc:2889] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:56.544368 1882 raft_consensus.cc:2272] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.544751 1182 raft_consensus.cc:2243] T b9fc33e8efc04734a951f7fcf9048142 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:56.544978 1882 ts_tablet_manager.cc:1916] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.545495 1182 raft_consensus.cc:2272] T b9fc33e8efc04734a951f7fcf9048142 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.546469 1182 ts_tablet_manager.cc:1916] T b9fc33e8efc04734a951f7fcf9048142 P 1521a6d262b7451bbef8debe970dd6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.551571 1882 ts_tablet_manager.cc:1929] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:56.551723 1882 log.cc:1199] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/4a518e8ad2e34ec98a0425c7ed3a470a
I20260202 02:52:56.552099 1882 ts_tablet_manager.cc:1950] T 4a518e8ad2e34ec98a0425c7ed3a470a P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.553033 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 968) took 2186 ms. Trace:
I20260202 02:52:56.553155 1882 rpcz_store.cc:276] 0202 02:52:54.366548 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:54.366718 (+ 170us) service_pool.cc:225] Handling call
0202 02:52:56.543634 (+2176916us) ts_tablet_manager.cc:1151] Deleting tablet 4a518e8ad2e34ec98a0425c7ed3a470a
0202 02:52:56.543644 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.548236 (+ 4592us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.551438 (+ 3202us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.553005 (+ 1567us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2176799}
I20260202 02:52:56.553603 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 962) took 2187 ms. Trace:
I20260202 02:52:56.553687 1882 rpcz_store.cc:276] 0202 02:52:54.366108 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:54.366832 (+ 724us) service_pool.cc:225] Handling call
0202 02:52:56.553484 (+2186652us) ts_tablet_manager.cc:1151] Deleting tablet 8bb14bf551f441dcbaaff7057b51b2be
0202 02:52:56.553496 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.553586 (+ 90us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2186529}
I20260202 02:52:56.553807 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 4a518e8ad2e34ec98a0425c7ed3a470a (table default.Table_3dcd1dea44ed4e54a89f4413e7f17dbe [id=f868f7fd84f247c89a963da591f16c2d]) successfully deleted
I20260202 02:52:56.553965 1882 tablet_replica.cc:333] T 9a1d7d173c41462c8e32a95b494bb708 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.554160 1882 raft_consensus.cc:2243] T 9a1d7d173c41462c8e32a95b494bb708 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
W20260202 02:52:56.554301 515 catalog_manager.cc:4952] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 8bb14bf551f441dcbaaff7057b51b2be because the tablet was not found. No further retry: Not found: Tablet not found: 8bb14bf551f441dcbaaff7057b51b2be
I20260202 02:52:56.554342 1882 raft_consensus.cc:2272] T 9a1d7d173c41462c8e32a95b494bb708 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.555096 1882 ts_tablet_manager.cc:1916] T 9a1d7d173c41462c8e32a95b494bb708 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.556105 1173 tablet_bootstrap.cc:492] T 3338964dfed548f980f58147f0283691 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:56.557093 1173 ts_tablet_manager.cc:1403] T 3338964dfed548f980f58147f0283691 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.044s user 0.013s sys 0.020s
I20260202 02:52:56.557898 1173 raft_consensus.cc:359] T 3338964dfed548f980f58147f0283691 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.558135 1173 raft_consensus.cc:385] T 3338964dfed548f980f58147f0283691 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:56.558224 1173 raft_consensus.cc:740] T 3338964dfed548f980f58147f0283691 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:56.558538 1173 consensus_queue.cc:260] T 3338964dfed548f980f58147f0283691 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.559106 1173 ts_tablet_manager.cc:1434] T 3338964dfed548f980f58147f0283691 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:56.560406 1173 tablet_bootstrap.cc:492] T 5ffdf97dff3e4dca89507f488c5b0f8e P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:56.561908 1173 tablet_bootstrap.cc:654] T 5ffdf97dff3e4dca89507f488c5b0f8e P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:56.563786 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet 3338964dfed548f980f58147f0283691 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:48 UTC) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:56.552139 1175 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.945s user 0.011s sys 0.007s
W20260202 02:52:56.571259 1175 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.964s user 0.011s sys 0.007s
I20260202 02:52:56.572189 1175 tablet_bootstrap.cc:492] T 1c8317adc47a4c399f2db650ed3d2e53 P 1521a6d262b7451bbef8debe970dd6b5: No bootstrap required, opened a new log
I20260202 02:52:56.572451 1175 ts_tablet_manager.cc:1403] T 1c8317adc47a4c399f2db650ed3d2e53 P 1521a6d262b7451bbef8debe970dd6b5: Time spent bootstrapping tablet: real 0.968s user 0.012s sys 0.008s
I20260202 02:52:56.572604 1173 tablet_bootstrap.cc:492] T 5ffdf97dff3e4dca89507f488c5b0f8e P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:56.572750 1173 ts_tablet_manager.cc:1403] T 5ffdf97dff3e4dca89507f488c5b0f8e P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260202 02:52:56.573382 1175 raft_consensus.cc:359] T 1c8317adc47a4c399f2db650ed3d2e53 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.573397 1173 raft_consensus.cc:359] T 5ffdf97dff3e4dca89507f488c5b0f8e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.573568 1173 raft_consensus.cc:385] T 5ffdf97dff3e4dca89507f488c5b0f8e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:56.573616 1882 ts_tablet_manager.cc:1929] T 9a1d7d173c41462c8e32a95b494bb708 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:56.573626 1173 raft_consensus.cc:740] T 5ffdf97dff3e4dca89507f488c5b0f8e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:56.573721 1882 log.cc:1199] T 9a1d7d173c41462c8e32a95b494bb708 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/9a1d7d173c41462c8e32a95b494bb708
I20260202 02:52:56.573659 1175 raft_consensus.cc:385] T 1c8317adc47a4c399f2db650ed3d2e53 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:56.573877 1175 raft_consensus.cc:740] T 1c8317adc47a4c399f2db650ed3d2e53 P 1521a6d262b7451bbef8debe970dd6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1521a6d262b7451bbef8debe970dd6b5, State: Initialized, Role: FOLLOWER
I20260202 02:52:56.573922 1173 consensus_queue.cc:260] T 5ffdf97dff3e4dca89507f488c5b0f8e P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.574229 1175 consensus_queue.cc:260] T 1c8317adc47a4c399f2db650ed3d2e53 P 1521a6d262b7451bbef8debe970dd6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.574865 1175 ts_tablet_manager.cc:1434] T 1c8317adc47a4c399f2db650ed3d2e53 P 1521a6d262b7451bbef8debe970dd6b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260202 02:52:56.575166 1175 tablet_bootstrap.cc:492] T e33f95e4f17a479c9acef5764c6c4c56 P 1521a6d262b7451bbef8debe970dd6b5: Bootstrap starting.
I20260202 02:52:56.576911 1175 tablet_bootstrap.cc:654] T e33f95e4f17a479c9acef5764c6c4c56 P 1521a6d262b7451bbef8debe970dd6b5: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:56.574100 1882 ts_tablet_manager.cc:1950] T 9a1d7d173c41462c8e32a95b494bb708 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.580977 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 963) took 2214 ms. Trace:
I20260202 02:52:56.581080 1882 rpcz_store.cc:276] 0202 02:52:54.366173 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:54.366856 (+ 683us) service_pool.cc:225] Handling call
0202 02:52:56.553917 (+2187061us) ts_tablet_manager.cc:1151] Deleting tablet 9a1d7d173c41462c8e32a95b494bb708
0202 02:52:56.553926 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.558465 (+ 4539us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.573515 (+ 15050us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.580958 (+ 7443us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2186929}
I20260202 02:52:56.581432 1882 tablet_replica.cc:333] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.581571 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 9a1d7d173c41462c8e32a95b494bb708 (table default.Table_28e7d9aa8131413e9a174756c246d164 [id=ec07c59893cf45c9b2a5eece556d10c5]) successfully deleted
I20260202 02:52:56.581607 1882 raft_consensus.cc:2243] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:56.581881 1882 pending_rounds.cc:70] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d: Trying to abort 1 pending ops.
I20260202 02:52:56.581938 1882 pending_rounds.cc:77] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923174290345984 op_type: NO_OP noop_request { }
I20260202 02:52:56.582036 1882 raft_consensus.cc:2889] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:56.582120 1882 raft_consensus.cc:2272] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.582670 1882 ts_tablet_manager.cc:1916] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.587630 1882 ts_tablet_manager.cc:1929] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:56.587756 1882 log.cc:1199] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/93f958924885457ab15f2aac5509e0aa
I20260202 02:52:56.588056 1882 ts_tablet_manager.cc:1950] T 93f958924885457ab15f2aac5509e0aa P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.579418 1173 ts_tablet_manager.cc:1434] T 5ffdf97dff3e4dca89507f488c5b0f8e P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260202 02:52:56.588976 418 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskYXxklX/build/asan/bin/kudu with pid 834
I20260202 02:52:56.589561 1054 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3338964dfed548f980f58147f0283691" candidate_uuid: "1521a6d262b7451bbef8debe970dd6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f05820b6034228bdad884cc348716d" is_pre_election: true
I20260202 02:52:56.589787 1054 raft_consensus.cc:2468] T 3338964dfed548f980f58147f0283691 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1521a6d262b7451bbef8debe970dd6b5 in term 0.
I20260202 02:52:56.592125 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 964) took 2225 ms. Trace:
I20260202 02:52:56.592245 1882 rpcz_store.cc:276] 0202 02:52:54.366237 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:54.366960 (+ 723us) service_pool.cc:225] Handling call
0202 02:52:56.581387 (+2214427us) ts_tablet_manager.cc:1151] Deleting tablet 93f958924885457ab15f2aac5509e0aa
0202 02:52:56.581397 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.585374 (+ 3977us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.587524 (+ 2150us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.592101 (+ 4577us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2214312}
I20260202 02:52:56.592641 1882 tablet_replica.cc:333] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.592829 1882 raft_consensus.cc:2243] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:56.592800 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 93f958924885457ab15f2aac5509e0aa (table default.Table_2e2049e67b8943acab5a0db3deb1535a [id=0db38635f6af4f168f221f4749adf0dd]) successfully deleted
I20260202 02:52:56.593078 1882 pending_rounds.cc:70] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d: Trying to abort 1 pending ops.
I20260202 02:52:56.593135 1882 pending_rounds.cc:77] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923170792681472 op_type: NO_OP noop_request { }
I20260202 02:52:56.593230 1882 raft_consensus.cc:2889] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:56.593317 1882 raft_consensus.cc:2272] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.593899 1882 ts_tablet_manager.cc:1916] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.596661 1173 tablet_bootstrap.cc:492] T 1c8317adc47a4c399f2db650ed3d2e53 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
W20260202 02:52:56.597062 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet de320a4afaa14ffd9d6123f2285e79fa (table default.Table_62e7c5405f2b46248650ac463717478b [id=0e3edffd39bd460b8ca0d02a983b76b5]) was not created within the allowed timeout. Replacing with a new tablet eeb3deca5bae40b7b772a958f1c2b61c
W20260202 02:52:56.597494 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet 0d4efc9e96f54e03bc1a990de90ac0db (table default.Table_3cc05aefd1034220bc3d74f6428e11f7 [id=f4f53cb760134f9198764d93c13a68b8]) was not created within the allowed timeout. Replacing with a new tablet c36babb5f2634da189de3a2562e0a818
W20260202 02:52:56.597931 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet f30b698c177c48ee9a4e577552bf7eee (table default.Table_44f181eda9a64d218b9bac7f62831d07 [id=14bbe505bc8b4a4bbde264a2c0dcdf50]) was not created within the allowed timeout. Replacing with a new tablet 1f21e3b538234b26af036ff512837e65
I20260202 02:52:56.598101 1173 tablet_bootstrap.cc:654] T 1c8317adc47a4c399f2db650ed3d2e53 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
W20260202 02:52:56.598424 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet 0545aad1286948e1ac4aa3958bfada16 (table default.Table_2cf49aa4b72d4448943e3ea50fa21c78 [id=9aa9a92096d2492a84e98022117a91ef]) was not created within the allowed timeout. Replacing with a new tablet 2e02d741772a41e98afe546e67229d25
W20260202 02:52:56.598726 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet 198432a3bb3a4090af38aace27a0b9c8 (table default.Table_8a639c50c6674b2da7771e12fe2d7237 [id=576ab2d635b74195835c1c2ee9464acd]) was not created within the allowed timeout. Replacing with a new tablet cbdd968fb4a54a848175019b2a660f3a
W20260202 02:52:56.599061 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet a806e1bd36564d659ff06cbe2016bb2a (table default.Table_ba53f6b46a524a03832adc957671ae98 [id=71a5a7d475ba498099f2bd7cc7289808]) was not created within the allowed timeout. Replacing with a new tablet fa0c6354eca24e8c8f8bcf79f3a0aca3
W20260202 02:52:56.599337 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet b6e44b2a88e649599b398a7c9c4028c5 (table default.Table_6596d734c9de42ccbf1af1b077742369 [id=0583f0adc4aa46aa8d25f82f5539c669]) was not created within the allowed timeout. Replacing with a new tablet b95be19c12ac4615aca3b7c1b4aeecdf
W20260202 02:52:56.599599 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet 8b4dcfed8ec348e4b48a85788d728ca2 (table default.Table_72656c0c3e7f4157b3c57bcd71d097b7 [id=fb15f49f2a484130aa798949cb3f26e1]) was not created within the allowed timeout. Replacing with a new tablet f6cd9d7fb08e4bfc874ac105b0fafba7
I20260202 02:52:56.599843 1882 ts_tablet_manager.cc:1929] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:56.599956 1882 log.cc:1199] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/dc90eeb79ded4739883ad98928476680
W20260202 02:52:56.600163 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet ff5966cfdcdd49adaaa3dbc994f82e57 (table default.Table_c65ca0837da14b6baeb2f46f36018548 [id=3ce5b518d0b0428f9624bc81c3b5c36b]) was not created within the allowed timeout. Replacing with a new tablet f2eb000e27a84943b86d7a9c2946b046
I20260202 02:52:56.600291 1882 ts_tablet_manager.cc:1950] T dc90eeb79ded4739883ad98928476680 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
W20260202 02:52:56.600540 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet e99a0003b6b74b99bf2b834c3690c1bb (table default.Table_df0389a8470d4dcfa6d9b90366867bae [id=305ae4f445fe4461aaed24b6dba424e1]) was not created within the allowed timeout. Replacing with a new tablet 92e06040c4064e98886b76849f02d0c2
W20260202 02:52:56.600811 682 catalog_manager.cc:6117] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Tablet a472b3b95de1476c99c74dcdddbd022d (table default.Table_bcaaa936a01e4aafbb3dc42f01abff56 [id=f3cca9e684db48a08ed7a40cf8683cce]) was not created within the allowed timeout. Replacing with a new tablet 59a1390e38fd4cc19d650a95c4277169
I20260202 02:52:56.601141 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 966) took 2234 ms. Trace:
I20260202 02:52:56.601235 1882 rpcz_store.cc:276] 0202 02:52:54.366363 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:54.367088 (+ 725us) service_pool.cc:225] Handling call
0202 02:52:56.592593 (+2225505us) ts_tablet_manager.cc:1151] Deleting tablet dc90eeb79ded4739883ad98928476680
0202 02:52:56.592603 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.596909 (+ 4306us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.599739 (+ 2830us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.601121 (+ 1382us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2225394}
I20260202 02:52:56.601567 1882 tablet_replica.cc:333] T bbcc723a8079443db455ae20fb0d585d P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.601661 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet dc90eeb79ded4739883ad98928476680 (table default.Table_a9b816e5b5b84151b840bfad315deb70 [id=b46bd49d8fe14af094e69597521a1a22]) successfully deleted
I20260202 02:52:56.601775 1882 raft_consensus.cc:2243] T bbcc723a8079443db455ae20fb0d585d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:56.602020 1882 raft_consensus.cc:2272] T bbcc723a8079443db455ae20fb0d585d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.602605 1882 ts_tablet_manager.cc:1916] T bbcc723a8079443db455ae20fb0d585d P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.609671 1882 ts_tablet_manager.cc:1929] T bbcc723a8079443db455ae20fb0d585d P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:56.609858 1882 log.cc:1199] T bbcc723a8079443db455ae20fb0d585d P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/bbcc723a8079443db455ae20fb0d585d
I20260202 02:52:56.610235 1882 ts_tablet_manager.cc:1950] T bbcc723a8079443db455ae20fb0d585d P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.611387 1173 tablet_bootstrap.cc:492] T 1c8317adc47a4c399f2db650ed3d2e53 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:56.611495 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 960) took 2245 ms. Trace:
I20260202 02:52:56.611574 1173 ts_tablet_manager.cc:1403] T 1c8317adc47a4c399f2db650ed3d2e53 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.004s
I20260202 02:52:56.611620 1882 rpcz_store.cc:276] 0202 02:52:54.365960 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:54.367465 (+ 1505us) service_pool.cc:225] Handling call
0202 02:52:56.601518 (+2234053us) ts_tablet_manager.cc:1151] Deleting tablet bbcc723a8079443db455ae20fb0d585d
0202 02:52:56.601528 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.606161 (+ 4633us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.609492 (+ 3331us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.611435 (+ 1943us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2232328}
I20260202 02:52:56.611979 1882 tablet_replica.cc:333] T 604b3b863c9540d3b55381dbfe1afdc3 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.612182 1882 raft_consensus.cc:2243] T 604b3b863c9540d3b55381dbfe1afdc3 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:56.612181 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet bbcc723a8079443db455ae20fb0d585d (table default.Table_93c8b5e956214e88abc353bfb46c8757 [id=1cb84b11b2f34287bbade3af6cd4df67]) successfully deleted
I20260202 02:52:56.612380 1882 raft_consensus.cc:2272] T 604b3b863c9540d3b55381dbfe1afdc3 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.612921 1882 ts_tablet_manager.cc:1916] T 604b3b863c9540d3b55381dbfe1afdc3 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.612938 1173 raft_consensus.cc:359] T 1c8317adc47a4c399f2db650ed3d2e53 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.613157 1173 raft_consensus.cc:385] T 1c8317adc47a4c399f2db650ed3d2e53 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:56.613240 1173 raft_consensus.cc:740] T 1c8317adc47a4c399f2db650ed3d2e53 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:56.613492 1173 consensus_queue.cc:260] T 1c8317adc47a4c399f2db650ed3d2e53 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.618992 1882 ts_tablet_manager.cc:1929] T 604b3b863c9540d3b55381dbfe1afdc3 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:56.619145 1882 log.cc:1199] T 604b3b863c9540d3b55381dbfe1afdc3 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/604b3b863c9540d3b55381dbfe1afdc3
I20260202 02:52:56.619452 1882 ts_tablet_manager.cc:1950] T 604b3b863c9540d3b55381dbfe1afdc3 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.620252 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 970) took 2246 ms. Trace:
I20260202 02:52:56.620407 1882 rpcz_store.cc:276] 0202 02:52:54.374022 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:54.374068 (+ 46us) service_pool.cc:225] Handling call
0202 02:52:56.611909 (+2237841us) ts_tablet_manager.cc:1151] Deleting tablet 604b3b863c9540d3b55381dbfe1afdc3
0202 02:52:56.611918 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.615905 (+ 3987us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.618861 (+ 2956us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.620203 (+ 1342us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2237649}
I20260202 02:52:56.620801 1882 tablet_replica.cc:333] T c17c85759e964f0ba3abcaf5fe938fba P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.621003 1882 raft_consensus.cc:2243] T c17c85759e964f0ba3abcaf5fe938fba P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:56.620976 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 604b3b863c9540d3b55381dbfe1afdc3 (table default.Table_756b4deb63c645e79a6fbb8f8f131ee0 [id=ce752170017a4e119a8c42c73fd95302]) successfully deleted
I20260202 02:52:56.621176 1882 raft_consensus.cc:2272] T c17c85759e964f0ba3abcaf5fe938fba P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.621754 1882 ts_tablet_manager.cc:1916] T c17c85759e964f0ba3abcaf5fe938fba P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.622072 1173 ts_tablet_manager.cc:1434] T 1c8317adc47a4c399f2db650ed3d2e53 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20260202 02:52:56.627193 1173 tablet_bootstrap.cc:492] T af29c7a2ad9d4797b08052606151a7d5 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:56.627682 1882 ts_tablet_manager.cc:1929] T c17c85759e964f0ba3abcaf5fe938fba P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:56.627872 1882 log.cc:1199] T c17c85759e964f0ba3abcaf5fe938fba P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/c17c85759e964f0ba3abcaf5fe938fba
I20260202 02:52:56.628268 1882 ts_tablet_manager.cc:1950] T c17c85759e964f0ba3abcaf5fe938fba P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.628773 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 969) took 2254 ms. Trace:
I20260202 02:52:56.629032 1173 tablet_bootstrap.cc:654] T af29c7a2ad9d4797b08052606151a7d5 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:56.628887 1882 rpcz_store.cc:276] 0202 02:52:54.373924 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:54.374266 (+ 342us) service_pool.cc:225] Handling call
0202 02:52:56.620729 (+2246463us) ts_tablet_manager.cc:1151] Deleting tablet c17c85759e964f0ba3abcaf5fe938fba
0202 02:52:56.620739 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.624780 (+ 4041us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.627559 (+ 2779us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.628728 (+ 1169us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2246312}
I20260202 02:52:56.629315 1882 tablet_replica.cc:333] T 67a8c9a1998b47b3b214a06ce62381e9 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.629509 1882 raft_consensus.cc:2243] T 67a8c9a1998b47b3b214a06ce62381e9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:56.629482 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet c17c85759e964f0ba3abcaf5fe938fba (table default.Table_67001aa310ae44c0bb9e2b24bb0b57b8 [id=c17d62a148094547af24de85773a1b8d]) successfully deleted
I20260202 02:52:56.629683 1882 raft_consensus.cc:2272] T 67a8c9a1998b47b3b214a06ce62381e9 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.630565 1882 ts_tablet_manager.cc:1916] T 67a8c9a1998b47b3b214a06ce62381e9 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.633816 3181 raft_consensus.cc:493] T a81e1d10ca49459989260fbd6a2b8645 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:56.633982 3181 raft_consensus.cc:515] T a81e1d10ca49459989260fbd6a2b8645 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.634654 3181 leader_election.cc:290] T a81e1d10ca49459989260fbd6a2b8645 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
W20260202 02:52:56.636402 998 leader_election.cc:336] T a81e1d10ca49459989260fbd6a2b8645 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111)
I20260202 02:52:56.637238 1882 ts_tablet_manager.cc:1929] T 67a8c9a1998b47b3b214a06ce62381e9 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:56.637368 1882 log.cc:1199] T 67a8c9a1998b47b3b214a06ce62381e9 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/67a8c9a1998b47b3b214a06ce62381e9
I20260202 02:52:56.637722 1882 ts_tablet_manager.cc:1950] T 67a8c9a1998b47b3b214a06ce62381e9 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.638216 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 956) took 2272 ms. Trace:
I20260202 02:52:56.638324 1882 rpcz_store.cc:276] 0202 02:52:54.365666 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:54.377842 (+ 12176us) service_pool.cc:225] Handling call
0202 02:52:56.629243 (+2251401us) ts_tablet_manager.cc:1151] Deleting tablet 67a8c9a1998b47b3b214a06ce62381e9
0202 02:52:56.629252 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.634071 (+ 4819us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.637091 (+ 3020us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.638168 (+ 1077us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2251218}
I20260202 02:52:56.638703 1882 tablet_replica.cc:333] T 09504070c9ec4bcfbb59e3fb4c189775 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.638765 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 67a8c9a1998b47b3b214a06ce62381e9 (table default.Table_1fa8930369e844ca8384126bf8f22e66 [id=116247212fca4c0a95ad1bdbbac4787f]) successfully deleted
I20260202 02:52:56.638932 1882 raft_consensus.cc:2243] T 09504070c9ec4bcfbb59e3fb4c189775 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:56.639117 1882 raft_consensus.cc:2272] T 09504070c9ec4bcfbb59e3fb4c189775 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.640569 1882 ts_tablet_manager.cc:1916] T 09504070c9ec4bcfbb59e3fb4c189775 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.642494 1173 tablet_bootstrap.cc:492] T af29c7a2ad9d4797b08052606151a7d5 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:56.642653 1173 ts_tablet_manager.cc:1403] T af29c7a2ad9d4797b08052606151a7d5 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.012s
I20260202 02:52:56.643362 1173 raft_consensus.cc:359] T af29c7a2ad9d4797b08052606151a7d5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.643559 1173 raft_consensus.cc:385] T af29c7a2ad9d4797b08052606151a7d5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:56.643623 1173 raft_consensus.cc:740] T af29c7a2ad9d4797b08052606151a7d5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:56.643854 1173 consensus_queue.cc:260] T af29c7a2ad9d4797b08052606151a7d5 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.645649 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet 1c8317adc47a4c399f2db650ed3d2e53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:26 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:56.647719 1882 ts_tablet_manager.cc:1929] T 09504070c9ec4bcfbb59e3fb4c189775 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:56.647864 1882 log.cc:1199] T 09504070c9ec4bcfbb59e3fb4c189775 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/09504070c9ec4bcfbb59e3fb4c189775
I20260202 02:52:56.648203 1882 ts_tablet_manager.cc:1950] T 09504070c9ec4bcfbb59e3fb4c189775 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.648694 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 971) took 2271 ms. Trace:
I20260202 02:52:56.648839 1882 rpcz_store.cc:276] 0202 02:52:54.377073 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:54.379750 (+ 2677us) service_pool.cc:225] Handling call
0202 02:52:56.638633 (+2258883us) ts_tablet_manager.cc:1151] Deleting tablet 09504070c9ec4bcfbb59e3fb4c189775
0202 02:52:56.638642 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.644399 (+ 5757us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.647562 (+ 3163us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.648650 (+ 1088us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2258711}
I20260202 02:52:56.649310 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 09504070c9ec4bcfbb59e3fb4c189775 (table default.Table_abd89cee70a04e2aa3a0d59b7eb943c0 [id=4c17da59a0554cee93ce4e626838a2c6]) successfully deleted
I20260202 02:52:56.649648 1882 tablet_replica.cc:333] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.649923 1882 raft_consensus.cc:2243] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:56.650184 1882 raft_consensus.cc:2272] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.650858 1882 ts_tablet_manager.cc:1916] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.651618 1173 ts_tablet_manager.cc:1434] T af29c7a2ad9d4797b08052606151a7d5 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260202 02:52:56.651932 1173 tablet_bootstrap.cc:492] T 361c0c3832614ec78695c28720b841e0 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:56.653385 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet af29c7a2ad9d4797b08052606151a7d5 with delete_type TABLET_DATA_DELETED (replaced by tablet c8606fb177db49c7b2e2b16b18d4725c) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:56.653921 1173 tablet_bootstrap.cc:654] T 361c0c3832614ec78695c28720b841e0 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:56.656811 1882 ts_tablet_manager.cc:1929] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.3
I20260202 02:52:56.656934 1882 log.cc:1199] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/cb634d4bae554ef3b39eeaa9958adc6c
I20260202 02:52:56.657276 1882 ts_tablet_manager.cc:1950] T cb634d4bae554ef3b39eeaa9958adc6c P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.658015 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 955) took 2292 ms. Trace:
I20260202 02:52:56.658109 1882 rpcz_store.cc:276] 0202 02:52:54.365569 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:54.384964 (+ 19395us) service_pool.cc:225] Handling call
0202 02:52:56.649124 (+2264160us) ts_tablet_manager.cc:1151] Deleting tablet cb634d4bae554ef3b39eeaa9958adc6c
0202 02:52:56.649583 (+ 459us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.653992 (+ 4409us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.656675 (+ 2683us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.657994 (+ 1319us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2263892}
I20260202 02:52:56.658420 1882 tablet_replica.cc:333] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.658579 1882 raft_consensus.cc:2243] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:56.658648 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet cb634d4bae554ef3b39eeaa9958adc6c (table default.Table_e5f4ac82d1ef4b85999695997eb9df2a [id=c97e407e519d4a538860f66931f6aefd]) successfully deleted
I20260202 02:52:56.658804 1882 pending_rounds.cc:70] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d: Trying to abort 1 pending ops.
I20260202 02:52:56.658841 1882 pending_rounds.cc:77] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923174409355264 op_type: NO_OP noop_request { }
I20260202 02:52:56.658933 1882 raft_consensus.cc:2889] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:56.659008 1882 raft_consensus.cc:2272] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.659521 1882 ts_tablet_manager.cc:1916] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.665505 1882 ts_tablet_manager.cc:1929] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:56.665635 1882 log.cc:1199] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/7ff6d2fa085343a6b11be0898da6233d
I20260202 02:52:56.665982 1882 ts_tablet_manager.cc:1950] T 7ff6d2fa085343a6b11be0898da6233d P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.666678 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 977) took 2235 ms. Trace:
I20260202 02:52:56.666779 1882 rpcz_store.cc:276] 0202 02:52:54.430889 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:54.430957 (+ 68us) service_pool.cc:225] Handling call
0202 02:52:56.658374 (+2227417us) ts_tablet_manager.cc:1151] Deleting tablet 7ff6d2fa085343a6b11be0898da6233d
0202 02:52:56.658384 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.662412 (+ 4028us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.665395 (+ 2983us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.666658 (+ 1263us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2227213}
I20260202 02:52:56.667292 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 7ff6d2fa085343a6b11be0898da6233d (table default.Table_bcf2aa761e104360bb4a1251204a6608 [id=5eaf1a210fa44066b81bb5b01db4d7cf]) successfully deleted
I20260202 02:52:56.667652 1882 tablet_replica.cc:333] T 5e20842de8cb4355b732685605c88834 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.667876 1882 raft_consensus.cc:2243] T 5e20842de8cb4355b732685605c88834 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:56.668097 1882 raft_consensus.cc:2272] T 5e20842de8cb4355b732685605c88834 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.668738 1882 ts_tablet_manager.cc:1916] T 5e20842de8cb4355b732685605c88834 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.674679 1882 ts_tablet_manager.cc:1929] T 5e20842de8cb4355b732685605c88834 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:56.674919 1882 log.cc:1199] T 5e20842de8cb4355b732685605c88834 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/5e20842de8cb4355b732685605c88834
I20260202 02:52:56.675230 1882 ts_tablet_manager.cc:1950] T 5e20842de8cb4355b732685605c88834 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.676002 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 979) took 2244 ms. Trace:
I20260202 02:52:56.676146 1882 rpcz_store.cc:276] 0202 02:52:54.431734 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:54.431869 (+ 135us) service_pool.cc:225] Handling call
0202 02:52:56.667045 (+2235176us) ts_tablet_manager.cc:1151] Deleting tablet 5e20842de8cb4355b732685605c88834
0202 02:52:56.667605 (+ 560us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.671896 (+ 4291us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.674551 (+ 2655us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.675982 (+ 1431us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2235009}
I20260202 02:52:56.676813 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 5e20842de8cb4355b732685605c88834 (table default.Table_bb379cca6ffa4183bea51c1e561862e7 [id=7d8f7ac4c2ee4b72b2ff25aff060010f]) successfully deleted
I20260202 02:52:56.677083 1882 tablet_replica.cc:333] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.677290 1882 raft_consensus.cc:2243] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:56.677551 1882 pending_rounds.cc:70] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d: Trying to abort 1 pending ops.
I20260202 02:52:56.677611 1882 pending_rounds.cc:77] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923174320746496 op_type: NO_OP noop_request { }
I20260202 02:52:56.677726 1882 raft_consensus.cc:2889] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:56.677804 1882 raft_consensus.cc:2272] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.678331 1882 ts_tablet_manager.cc:1916] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:56.683087 997 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260202 02:52:56.684139 1882 ts_tablet_manager.cc:1929] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:56.684252 1882 log.cc:1199] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/b4a4c75a906f40beb896bf6fff705ae6
I20260202 02:52:56.684615 1882 ts_tablet_manager.cc:1950] T b4a4c75a906f40beb896bf6fff705ae6 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.685901 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 980) took 2254 ms. Trace:
I20260202 02:52:56.685990 1882 rpcz_store.cc:276] 0202 02:52:54.431825 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:54.432063 (+ 238us) service_pool.cc:225] Handling call
0202 02:52:56.676615 (+2244552us) ts_tablet_manager.cc:1151] Deleting tablet b4a4c75a906f40beb896bf6fff705ae6
0202 02:52:56.676627 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.681285 (+ 4658us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.684028 (+ 2743us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.685878 (+ 1850us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2244437}
I20260202 02:52:56.686298 1882 tablet_replica.cc:333] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.686468 1882 raft_consensus.cc:2243] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:52:56.686569 1173 tablet_bootstrap.cc:492] T 361c0c3832614ec78695c28720b841e0 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:56.686681 1173 ts_tablet_manager.cc:1403] T 361c0c3832614ec78695c28720b841e0 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.035s user 0.005s sys 0.001s
I20260202 02:52:56.686699 1882 pending_rounds.cc:70] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d: Trying to abort 1 pending ops.
I20260202 02:52:56.686746 1882 pending_rounds.cc:77] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7249923173747179520 op_type: NO_OP noop_request { }
I20260202 02:52:56.686838 1882 raft_consensus.cc:2889] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260202 02:52:56.686913 1882 raft_consensus.cc:2272] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.687388 1882 ts_tablet_manager.cc:1916] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.687312 1173 raft_consensus.cc:359] T 361c0c3832614ec78695c28720b841e0 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.687517 1173 raft_consensus.cc:385] T 361c0c3832614ec78695c28720b841e0 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:56.687573 1173 raft_consensus.cc:740] T 361c0c3832614ec78695c28720b841e0 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:56.687758 1173 consensus_queue.cc:260] T 361c0c3832614ec78695c28720b841e0 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
W20260202 02:52:56.691614 999 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260202 02:52:56.692693 1173 ts_tablet_manager.cc:1434] T 361c0c3832614ec78695c28720b841e0 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260202 02:52:56.693214 1882 ts_tablet_manager.cc:1929] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:52:56.693307 1882 log.cc:1199] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/6a3dac5cd1ce47488a50b4c32eca67c7
I20260202 02:52:56.693609 1882 ts_tablet_manager.cc:1950] T 6a3dac5cd1ce47488a50b4c32eca67c7 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:56.694314 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 978) took 2262 ms. Trace:
I20260202 02:52:56.694392 1882 rpcz_store.cc:276] 0202 02:52:54.431640 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:54.432299 (+ 659us) service_pool.cc:225] Handling call
0202 02:52:56.686255 (+2253956us) ts_tablet_manager.cc:1151] Deleting tablet 6a3dac5cd1ce47488a50b4c32eca67c7
0202 02:52:56.686263 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.690296 (+ 4033us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.693122 (+ 2826us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.694294 (+ 1172us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2253799}
I20260202 02:52:56.694790 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 982) took 2253 ms. Trace:
I20260202 02:52:56.694859 1882 rpcz_store.cc:276] 0202 02:52:54.441738 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:54.441802 (+ 64us) service_pool.cc:225] Handling call
0202 02:52:56.694660 (+2252858us) ts_tablet_manager.cc:1151] Deleting tablet 7c07195a46184d24bc709e4967945472
0202 02:52:56.694670 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.694775 (+ 105us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2252648}
I20260202 02:52:56.695163 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 1041) took 1351 ms. Trace:
I20260202 02:52:56.695230 1882 rpcz_store.cc:276] 0202 02:52:55.344067 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:55.369206 (+ 25139us) service_pool.cc:225] Handling call
0202 02:52:56.695068 (+1325862us) ts_tablet_manager.cc:1151] Deleting tablet 28f6161d0d0a4b41af4560f0f3bee0f7
0202 02:52:56.695076 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.695151 (+ 75us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1325637}
I20260202 02:52:56.695518 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 1042) took 1351 ms. Trace:
I20260202 02:52:56.695575 1882 rpcz_store.cc:276] 0202 02:52:55.344181 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:55.369448 (+ 25267us) service_pool.cc:225] Handling call
0202 02:52:56.695429 (+1325981us) ts_tablet_manager.cc:1151] Deleting tablet 8d9ffdb5f10e4510888a90c49cd779bd
0202 02:52:56.695437 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.695504 (+ 67us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1325814}
I20260202 02:52:56.695852 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 1043) took 1351 ms. Trace:
I20260202 02:52:56.695936 3181 raft_consensus.cc:493] T cd1b6662da254889ae4bbe8d0b1ef1bb P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:56.695914 1882 rpcz_store.cc:276] 0202 02:52:55.344598 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:55.369632 (+ 25034us) service_pool.cc:225] Handling call
0202 02:52:56.695765 (+1326133us) ts_tablet_manager.cc:1151] Deleting tablet 5d2fc0a74684434d85b9922cb4e32fdc
0202 02:52:56.695773 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.695837 (+ 64us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1325963}
I20260202 02:52:56.696081 3181 raft_consensus.cc:515] T cd1b6662da254889ae4bbe8d0b1ef1bb P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.696195 1882 tablet_replica.cc:333] T e33f95e4f17a479c9acef5764c6c4c56 P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.696429 1882 raft_consensus.cc:2243] T e33f95e4f17a479c9acef5764c6c4c56 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:56.696583 1882 raft_consensus.cc:2272] T e33f95e4f17a479c9acef5764c6c4c56 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:56.696717 3181 leader_election.cc:290] T cd1b6662da254889ae4bbe8d0b1ef1bb P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:56.697342 1882 ts_tablet_manager.cc:1916] T e33f95e4f17a479c9acef5764c6c4c56 P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:56.697809 1173 tablet_bootstrap.cc:492] T 1d98c7a46139413b8f0d3697b74b1e52 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
W20260202 02:52:56.698589 998 leader_election.cc:336] T cd1b6662da254889ae4bbe8d0b1ef1bb P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1f442ef85029461091de847118c34c0d (127.0.104.129:39403): Network error: Client connection negotiation failed: client connection to 127.0.104.129:39403: connect: Connection refused (error 111)
I20260202 02:52:56.699415 1173 tablet_bootstrap.cc:654] T 1d98c7a46139413b8f0d3697b74b1e52 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:56.700304 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet b4a4c75a906f40beb896bf6fff705ae6 (table default.Table_6806575803c4468fa7b7ef990004b446 [id=944fbe1acd894017bae3c54f135ee9b0]) successfully deleted
I20260202 02:52:56.700579 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet 6a3dac5cd1ce47488a50b4c32eca67c7 (table default.Table_25f0023dd22a4bb0a7c2863da66d0002 [id=e24ffad12f514d60907b6cadfd09dbe6]) successfully deleted
W20260202 02:52:56.700785 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 7c07195a46184d24bc709e4967945472 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7c07195a46184d24bc709e4967945472 already in progress: creating tablet
W20260202 02:52:56.700963 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 28f6161d0d0a4b41af4560f0f3bee0f7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 28f6161d0d0a4b41af4560f0f3bee0f7 already in progress: creating tablet
W20260202 02:52:56.701122 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 8d9ffdb5f10e4510888a90c49cd779bd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8d9ffdb5f10e4510888a90c49cd779bd already in progress: creating tablet
W20260202 02:52:56.701318 515 catalog_manager.cc:4965] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): delete failed for tablet 5d2fc0a74684434d85b9922cb4e32fdc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5d2fc0a74684434d85b9922cb4e32fdc already in progress: creating tablet
I20260202 02:52:56.707455 1173 tablet_bootstrap.cc:492] T 1d98c7a46139413b8f0d3697b74b1e52 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:56.710204 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet de320a4afaa14ffd9d6123f2285e79fa
I20260202 02:52:56.710686 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 0d4efc9e96f54e03bc1a990de90ac0db
I20260202 02:52:56.711058 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet f30b698c177c48ee9a4e577552bf7eee
I20260202 02:52:56.709648 1882 ts_tablet_manager.cc:1929] T e33f95e4f17a479c9acef5764c6c4c56 P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:56.711377 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 0545aad1286948e1ac4aa3958bfada16
I20260202 02:52:56.710584 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet 361c0c3832614ec78695c28720b841e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:24 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:56.711478 1882 log.cc:1199] T e33f95e4f17a479c9acef5764c6c4c56 P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/e33f95e4f17a479c9acef5764c6c4c56
W20260202 02:52:56.711633 1000 connection.cc:537] server connection from 127.0.104.130:37027 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260202 02:52:56.711681 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 198432a3bb3a4090af38aace27a0b9c8
I20260202 02:52:56.711794 1882 ts_tablet_manager.cc:1950] T e33f95e4f17a479c9acef5764c6c4c56 P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
W20260202 02:52:56.711949 999 connection.cc:537] client connection to 127.0.104.130:38539 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260202 02:52:56.712446 514 connection.cc:700] client connection to 127.0.104.130:38539 send error: Network error: sendmsg error: Broken pipe (error 32)
I20260202 02:52:56.712924 418 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskYXxklX/build/asan/bin/kudu pid 975
W20260202 02:52:56.716631 999 consensus_peers.cc:597] T 939fbcaf75c7439d9208bbbf83a72717 P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.130:38539: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:56.716763 999 consensus_peers.cc:597] T af5aa68e47bb420eac300754d274fa73 P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.130:38539: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:56.716833 999 leader_election.cc:336] T b9fc33e8efc04734a951f7fcf9048142 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Network error: Client connection negotiation failed: client connection to 127.0.104.130:38539: connect: Connection refused (error 111)
I20260202 02:52:56.716917 999 leader_election.cc:304] T b9fc33e8efc04734a951f7fcf9048142 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:56.717404 3181 raft_consensus.cc:2749] T b9fc33e8efc04734a951f7fcf9048142 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:56.717707 514 catalog_manager.cc:4712] TS 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e95153ee8f2d40de955e3389d3a58644: Network error: Client connection negotiation failed: client connection to 127.0.104.130:38539: connect: Connection refused (error 111)
I20260202 02:52:56.717864 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet de320a4afaa14ffd9d6123f2285e79fa with delete_type TABLET_DATA_DELETED (Replaced by eeb3deca5bae40b7b772a958f1c2b61c at 2026-02-02 02:52:56 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:56.718043 1034 tablet_service.cc:1552] Processing DeleteTablet for tablet 0d4efc9e96f54e03bc1a990de90ac0db with delete_type TABLET_DATA_DELETED (Replaced by c36babb5f2634da189de3a2562e0a818 at 2026-02-02 02:52:56 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:56.718166 1036 tablet_service.cc:1552] Processing DeleteTablet for tablet 0545aad1286948e1ac4aa3958bfada16 with delete_type TABLET_DATA_DELETED (Replaced by 2e02d741772a41e98afe546e67229d25 at 2026-02-02 02:52:56 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:56.718182 1040 tablet_service.cc:1552] Processing DeleteTablet for tablet f30b698c177c48ee9a4e577552bf7eee with delete_type TABLET_DATA_DELETED (Replaced by 1f21e3b538234b26af036ff512837e65 at 2026-02-02 02:52:56 UTC) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:56.718642 999 leader_election.cc:336] T c977c146bac34638874b7a990604a0d5 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Network error: Client connection negotiation failed: client connection to 127.0.104.130:38539: connect: Connection refused (error 111)
I20260202 02:52:56.719050 999 leader_election.cc:304] T c977c146bac34638874b7a990604a0d5 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:56.719319 3181 raft_consensus.cc:2749] T c977c146bac34638874b7a990604a0d5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:56.719722 999 consensus_peers.cc:597] T 44c8d5b74c8e4c499f7f68e8185efc17 P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.130:38539: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:56.720005 999 leader_election.cc:336] T c17c85759e964f0ba3abcaf5fe938fba P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Network error: Client connection negotiation failed: client connection to 127.0.104.130:38539: connect: Connection refused (error 111)
I20260202 02:52:56.720139 999 leader_election.cc:304] T c17c85759e964f0ba3abcaf5fe938fba P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
W20260202 02:52:56.720360 999 raft_consensus.cc:2686] T c17c85759e964f0ba3abcaf5fe938fba P b1f05820b6034228bdad884cc348716d: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260202 02:52:56.721524 999 leader_election.cc:336] T 842197c689064660b791b40e521d346b P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Network error: Client connection negotiation failed: client connection to 127.0.104.130:38539: connect: Connection refused (error 111)
I20260202 02:52:56.721745 999 leader_election.cc:304] T 842197c689064660b791b40e521d346b P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:56.712585 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 1044) took 1215 ms. Trace:
I20260202 02:52:56.722005 3181 raft_consensus.cc:2749] T 842197c689064660b791b40e521d346b P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:56.721978 1882 rpcz_store.cc:276] 0202 02:52:55.496688 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:55.496734 (+ 46us) service_pool.cc:225] Handling call
0202 02:52:56.696155 (+1199421us) ts_tablet_manager.cc:1151] Deleting tablet e33f95e4f17a479c9acef5764c6c4c56
0202 02:52:56.696162 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.700226 (+ 4064us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.709528 (+ 9302us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:56.712566 (+ 3038us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1199180}
I20260202 02:52:56.722301 1882 tablet_replica.cc:333] T 4e90f273439d414caf16d63e1c605d3d P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:56.722468 1882 raft_consensus.cc:2243] T 4e90f273439d414caf16d63e1c605d3d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:56.722513 515 catalog_manager.cc:4985] TS b1f05820b6034228bdad884cc348716d (127.0.104.131:35257): tablet e33f95e4f17a479c9acef5764c6c4c56 (table default.Table_1455877b4adc4c53a941d7fc145014b7 [id=5c2778a5ed864321bb59e445b4db671a]) successfully deleted
I20260202 02:52:56.722610 1882 raft_consensus.cc:2272] T 4e90f273439d414caf16d63e1c605d3d P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
W20260202 02:52:56.722914 999 consensus_peers.cc:597] T 16b47a2831f54d809ed0b16deaec7e3d P b1f05820b6034228bdad884cc348716d -> Peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Couldn't send request to peer 1521a6d262b7451bbef8debe970dd6b5. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.130:38539: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260202 02:52:56.723084 999 leader_election.cc:336] T 2f664f6119ba4d28b5ef1c212e611866 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Network error: Client connection negotiation failed: client connection to 127.0.104.130:38539: connect: Connection refused (error 111)
I20260202 02:52:56.723213 999 leader_election.cc:304] T 2f664f6119ba4d28b5ef1c212e611866 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:56.723232 1040 tablet_service.cc:1552] Processing DeleteTablet for tablet 198432a3bb3a4090af38aace27a0b9c8 with delete_type TABLET_DATA_DELETED (Replaced by cbdd968fb4a54a848175019b2a660f3a at 2026-02-02 02:52:56 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:56.723593 3181 raft_consensus.cc:2749] T 2f664f6119ba4d28b5ef1c212e611866 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:56.723986 999 leader_election.cc:336] T cd1b6662da254889ae4bbe8d0b1ef1bb P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Network error: Client connection negotiation failed: client connection to 127.0.104.130:38539: connect: Connection refused (error 111)
I20260202 02:52:56.724133 999 leader_election.cc:304] T cd1b6662da254889ae4bbe8d0b1ef1bb P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:56.724372 3181 raft_consensus.cc:2749] T cd1b6662da254889ae4bbe8d0b1ef1bb P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:56.711794 1173 ts_tablet_manager.cc:1403] T 1d98c7a46139413b8f0d3697b74b1e52 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.005s
I20260202 02:52:56.724681 1882 ts_tablet_manager.cc:1916] T 4e90f273439d414caf16d63e1c605d3d P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:52:56.724713 999 leader_election.cc:336] T a81e1d10ca49459989260fbd6a2b8645 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Network error: Client connection negotiation failed: client connection to 127.0.104.130:38539: connect: Connection refused (error 111)
I20260202 02:52:56.725363 999 leader_election.cc:304] T a81e1d10ca49459989260fbd6a2b8645 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:56.725554 3181 raft_consensus.cc:2749] T a81e1d10ca49459989260fbd6a2b8645 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:56.725620 999 leader_election.cc:336] T 67a8c9a1998b47b3b214a06ce62381e9 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Network error: Client connection negotiation failed: client connection to 127.0.104.130:38539: connect: Connection refused (error 111)
I20260202 02:52:56.725693 999 leader_election.cc:304] T 67a8c9a1998b47b3b214a06ce62381e9 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
W20260202 02:52:56.725837 999 raft_consensus.cc:2686] T 67a8c9a1998b47b3b214a06ce62381e9 P b1f05820b6034228bdad884cc348716d: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260202 02:52:56.725946 1173 raft_consensus.cc:359] T 1d98c7a46139413b8f0d3697b74b1e52 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:56.726150 1173 raft_consensus.cc:385] T 1d98c7a46139413b8f0d3697b74b1e52 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:56.726236 1173 raft_consensus.cc:740] T 1d98c7a46139413b8f0d3697b74b1e52 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
W20260202 02:52:56.726270 999 leader_election.cc:336] T 3dcf4f794ec14b2f9d181d015fc437fb P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Network error: Client connection negotiation failed: client connection to 127.0.104.130:38539: connect: Connection refused (error 111)
I20260202 02:52:56.726357 999 leader_election.cc:304] T 3dcf4f794ec14b2f9d181d015fc437fb P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:56.726545 3181 raft_consensus.cc:2749] T 3dcf4f794ec14b2f9d181d015fc437fb P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:56.726644 999 leader_election.cc:336] T 7f9dfac19c04475aadb45e95a6474eb4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Network error: Client connection negotiation failed: client connection to 127.0.104.130:38539: connect: Connection refused (error 111)
I20260202 02:52:56.726737 999 leader_election.cc:304] T 7f9dfac19c04475aadb45e95a6474eb4 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:56.726892 3181 raft_consensus.cc:2749] T 7f9dfac19c04475aadb45e95a6474eb4 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:56.726953 999 leader_election.cc:336] T 09504070c9ec4bcfbb59e3fb4c189775 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Network error: Client connection negotiation failed: client connection to 127.0.104.130:38539: connect: Connection refused (error 111)
I20260202 02:52:56.727031 999 leader_election.cc:304] T 09504070c9ec4bcfbb59e3fb4c189775 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
W20260202 02:52:56.727155 999 raft_consensus.cc:2686] T 09504070c9ec4bcfbb59e3fb4c189775 P b1f05820b6034228bdad884cc348716d: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260202 02:52:56.727136 1173 consensus_queue.cc:260] T 1d98c7a46139413b8f0d3697b74b1e52 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
W20260202 02:52:56.727600 999 leader_election.cc:336] T 7921daa000f54c8dbed04473e21cf0a5 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Network error: Client connection negotiation failed: client connection to 127.0.104.130:38539: connect: Connection refused (error 111)
I20260202 02:52:56.727690 999 leader_election.cc:304] T 7921daa000f54c8dbed04473e21cf0a5 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:56.727859 2979 raft_consensus.cc:2749] T 7921daa000f54c8dbed04473e21cf0a5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260202 02:52:56.728093 999 leader_election.cc:336] T 9a1d7d173c41462c8e32a95b494bb708 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Network error: Client connection negotiation failed: client connection to 127.0.104.130:38539: connect: Connection refused (error 111)
I20260202 02:52:56.728171 999 leader_election.cc:304] T 9a1d7d173c41462c8e32a95b494bb708 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
W20260202 02:52:56.728319 999 raft_consensus.cc:2686] T 9a1d7d173c41462c8e32a95b494bb708 P b1f05820b6034228bdad884cc348716d: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260202 02:52:56.728439 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet a806e1bd36564d659ff06cbe2016bb2a
W20260202 02:52:56.728749 999 leader_election.cc:336] T f6c2bc531d194a4b8f3062c9b40e3da6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Network error: Client connection negotiation failed: client connection to 127.0.104.130:38539: connect: Connection refused (error 111)
I20260202 02:52:56.728834 999 leader_election.cc:304] T f6c2bc531d194a4b8f3062c9b40e3da6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:56.729003 2979 raft_consensus.cc:2749] T f6c2bc531d194a4b8f3062c9b40e3da6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:56.729415 1040 tablet_service.cc:1552] Processing DeleteTablet for tablet a806e1bd36564d659ff06cbe2016bb2a with delete_type TABLET_DATA_DELETED (Replaced by fa0c6354eca24e8c8f8bcf79f3a0aca3 at 2026-02-02 02:52:56 UTC) from {username='slave'} at 127.0.0.1:49890
W20260202 02:52:56.729643 999 leader_election.cc:336] T e95153ee8f2d40de955e3389d3a58644 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539): Network error: Client connection negotiation failed: client connection to 127.0.104.130:38539: connect: Connection refused (error 111)
I20260202 02:52:56.729734 999 leader_election.cc:304] T e95153ee8f2d40de955e3389d3a58644 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1f05820b6034228bdad884cc348716d; no voters: 1521a6d262b7451bbef8debe970dd6b5, 1f442ef85029461091de847118c34c0d
I20260202 02:52:56.729912 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet b6e44b2a88e649599b398a7c9c4028c5
I20260202 02:52:56.730674 1040 tablet_service.cc:1552] Processing DeleteTablet for tablet b6e44b2a88e649599b398a7c9c4028c5 with delete_type TABLET_DATA_DELETED (Replaced by b95be19c12ac4615aca3b7c1b4aeecdf at 2026-02-02 02:52:56 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:56.730849 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet 8b4dcfed8ec348e4b48a85788d728ca2
I20260202 02:52:56.731526 1040 tablet_service.cc:1552] Processing DeleteTablet for tablet 8b4dcfed8ec348e4b48a85788d728ca2 with delete_type TABLET_DATA_DELETED (Replaced by f6cd9d7fb08e4bfc874ac105b0fafba7 at 2026-02-02 02:52:56 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:56.731678 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet ff5966cfdcdd49adaaa3dbc994f82e57
I20260202 02:52:56.732391 1040 tablet_service.cc:1552] Processing DeleteTablet for tablet ff5966cfdcdd49adaaa3dbc994f82e57 with delete_type TABLET_DATA_DELETED (Replaced by f2eb000e27a84943b86d7a9c2946b046 at 2026-02-02 02:52:56 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:56.732610 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet e99a0003b6b74b99bf2b834c3690c1bb
I20260202 02:52:56.733410 1040 tablet_service.cc:1552] Processing DeleteTablet for tablet e99a0003b6b74b99bf2b834c3690c1bb with delete_type TABLET_DATA_DELETED (Replaced by 92e06040c4064e98886b76849f02d0c2 at 2026-02-02 02:52:56 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:56.733630 682 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 92a8191cf1e14e4d99f1c378c2d3c85b: Sending DeleteTablet for 3 replicas of tablet a472b3b95de1476c99c74dcdddbd022d
I20260202 02:52:56.733757 1173 ts_tablet_manager.cc:1434] T 1d98c7a46139413b8f0d3697b74b1e52 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260202 02:52:56.733937 3181 raft_consensus.cc:2749] T e95153ee8f2d40de955e3389d3a58644 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:52:56.734041 1173 tablet_bootstrap.cc:492] T 3857680ccbdc4dd4ad8247da96254434 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:56.734611 1040 tablet_service.cc:1552] Processing DeleteTablet for tablet a472b3b95de1476c99c74dcdddbd022d with delete_type TABLET_DATA_DELETED (Replaced by 59a1390e38fd4cc19d650a95c4277169 at 2026-02-02 02:52:56 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:56.735531 1173 tablet_bootstrap.cc:654] T 3857680ccbdc4dd4ad8247da96254434 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:56.738699 1040 tablet_service.cc:1552] Processing DeleteTablet for tablet 1d98c7a46139413b8f0d3697b74b1e52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-02 02:52:29 UTC) from {username='slave'} at 127.0.0.1:49890
I20260202 02:52:56.741261 1040 tablet_service.cc:1505] Processing CreateTablet for tablet eeb3deca5bae40b7b772a958f1c2b61c (DEFAULT_TABLE table=default.Table_62e7c5405f2b46248650ac463717478b [id=0e3edffd39bd460b8ca0d02a983b76b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:56.742718 1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeb3deca5bae40b7b772a958f1c2b61c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:56.745157 1036 tablet_service.cc:1505] Processing CreateTablet for tablet c36babb5f2634da189de3a2562e0a818 (DEFAULT_TABLE table=default.Table_3cc05aefd1034220bc3d74f6428e11f7 [id=f4f53cb760134f9198764d93c13a68b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:56.745527 1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c36babb5f2634da189de3a2562e0a818. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:56.747126 1104 generic_service.cc:196] Checking for leaks (request via RPC)
I20260202 02:52:56.751739 3181 raft_consensus.cc:493] T dc4c3f52f4f44a84b0d0e3c9a4a49f09 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:56.756884 1034 tablet_service.cc:1505] Processing CreateTablet for tablet 1f21e3b538234b26af036ff512837e65 (DEFAULT_TABLE table=default.Table_44f181eda9a64d218b9bac7f62831d07 [id=14bbe505bc8b4a4bbde264a2c0dcdf50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:56.761212 1038 tablet_service.cc:1505] Processing CreateTablet for tablet 2e02d741772a41e98afe546e67229d25 (DEFAULT_TABLE table=default.Table_2cf49aa4b72d4448943e3ea50fa21c78 [id=9aa9a92096d2492a84e98022117a91ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:56.761713 1035 tablet_service.cc:1505] Processing CreateTablet for tablet cbdd968fb4a54a848175019b2a660f3a (DEFAULT_TABLE table=default.Table_8a639c50c6674b2da7771e12fe2d7237 [id=576ab2d635b74195835c1c2ee9464acd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:57.206585 3181 raft_consensus.cc:515] T dc4c3f52f4f44a84b0d0e3c9a4a49f09 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:57.222596 1025 tablet_service.cc:1505] Processing CreateTablet for tablet b95be19c12ac4615aca3b7c1b4aeecdf (DEFAULT_TABLE table=default.Table_6596d734c9de42ccbf1af1b077742369 [id=0583f0adc4aa46aa8d25f82f5539c669]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:57.222982 1037 tablet_service.cc:1505] Processing CreateTablet for tablet fa0c6354eca24e8c8f8bcf79f3a0aca3 (DEFAULT_TABLE table=default.Table_ba53f6b46a524a03832adc957671ae98 [id=71a5a7d475ba498099f2bd7cc7289808]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:57.223132 1030 tablet_service.cc:1505] Processing CreateTablet for tablet f6cd9d7fb08e4bfc874ac105b0fafba7 (DEFAULT_TABLE table=default.Table_72656c0c3e7f4157b3c57bcd71d097b7 [id=fb15f49f2a484130aa798949cb3f26e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:57.223469 1043 tablet_service.cc:1505] Processing CreateTablet for tablet f2eb000e27a84943b86d7a9c2946b046 (DEFAULT_TABLE table=default.Table_c65ca0837da14b6baeb2f46f36018548 [id=3ce5b518d0b0428f9624bc81c3b5c36b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:57.223902 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 92e06040c4064e98886b76849f02d0c2 (DEFAULT_TABLE table=default.Table_df0389a8470d4dcfa6d9b90366867bae [id=305ae4f445fe4461aaed24b6dba424e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:57.224282 1042 tablet_service.cc:1505] Processing CreateTablet for tablet 59a1390e38fd4cc19d650a95c4277169 (DEFAULT_TABLE table=default.Table_bcaaa936a01e4aafbb3dc42f01abff56 [id=f3cca9e684db48a08ed7a40cf8683cce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:52:57.224964 3181 leader_election.cc:290] T dc4c3f52f4f44a84b0d0e3c9a4a49f09 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:57.225122 3181 raft_consensus.cc:493] T 5ffdf97dff3e4dca89507f488c5b0f8e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:57.225229 3181 raft_consensus.cc:515] T 5ffdf97dff3e4dca89507f488c5b0f8e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:57.225840 3181 leader_election.cc:290] T 5ffdf97dff3e4dca89507f488c5b0f8e P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:57.226325 3181 raft_consensus.cc:493] T af29c7a2ad9d4797b08052606151a7d5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:57.226430 3181 raft_consensus.cc:515] T af29c7a2ad9d4797b08052606151a7d5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:57.232686 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f21e3b538234b26af036ff512837e65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:57.232955 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbdd968fb4a54a848175019b2a660f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:57.236202 1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e02d741772a41e98afe546e67229d25. 1 dirs total, 0 dirs full, 0 dirs failed
W20260202 02:52:57.239125 1173 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.503s user 0.013s sys 0.025s
W20260202 02:52:57.239236 1173 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.503s user 0.013s sys 0.025s
I20260202 02:52:57.240025 1173 tablet_bootstrap.cc:492] T 3857680ccbdc4dd4ad8247da96254434 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:57.240142 1173 ts_tablet_manager.cc:1403] T 3857680ccbdc4dd4ad8247da96254434 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.506s user 0.014s sys 0.025s
I20260202 02:52:57.241226 1173 raft_consensus.cc:359] T 3857680ccbdc4dd4ad8247da96254434 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:57.241385 1173 raft_consensus.cc:385] T 3857680ccbdc4dd4ad8247da96254434 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:57.241429 1173 raft_consensus.cc:740] T 3857680ccbdc4dd4ad8247da96254434 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:57.241602 1173 consensus_queue.cc:260] T 3857680ccbdc4dd4ad8247da96254434 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:57.241967 1173 ts_tablet_manager.cc:1434] T 3857680ccbdc4dd4ad8247da96254434 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260202 02:52:57.242254 1173 tablet_bootstrap.cc:492] T 3706562a22634eb6b72e36bb6a2d1ae5 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:57.242338 1882 ts_tablet_manager.cc:1929] T 4e90f273439d414caf16d63e1c605d3d P b1f05820b6034228bdad884cc348716d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:52:57.242415 1882 log.cc:1199] T 4e90f273439d414caf16d63e1c605d3d P b1f05820b6034228bdad884cc348716d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/4e90f273439d414caf16d63e1c605d3d
I20260202 02:52:57.242695 1882 ts_tablet_manager.cc:1950] T 4e90f273439d414caf16d63e1c605d3d P b1f05820b6034228bdad884cc348716d: Deleting consensus metadata
I20260202 02:52:57.243317 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b95be19c12ac4615aca3b7c1b4aeecdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:57.243907 1173 tablet_bootstrap.cc:654] T 3706562a22634eb6b72e36bb6a2d1ae5 P b1f05820b6034228bdad884cc348716d: Neither blocks nor log segments found. Creating new log.
I20260202 02:52:57.245334 3289 raft_consensus.cc:493] T 23671408fcbc454cbd918328015cf6e6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:57.245466 3289 raft_consensus.cc:515] T 23671408fcbc454cbd918328015cf6e6 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:57.246184 3289 leader_election.cc:290] T 23671408fcbc454cbd918328015cf6e6 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:57.247005 1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2eb000e27a84943b86d7a9c2946b046. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:57.247239 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92e06040c4064e98886b76849f02d0c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:57.247423 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 1045) took 1750 ms. Trace:
I20260202 02:52:57.247529 1882 rpcz_store.cc:276] 0202 02:52:55.497097 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:55.497138 (+ 41us) service_pool.cc:225] Handling call
0202 02:52:56.722247 (+1225109us) ts_tablet_manager.cc:1151] Deleting tablet 4e90f273439d414caf16d63e1c605d3d
0202 02:52:56.722259 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:56.739329 (+ 17070us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:57.242254 (+502925us) tablet_metadata.cc:641] Metadata flushed
0202 02:52:57.247399 (+ 5145us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1224967}
I20260202 02:52:57.247946 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 1046) took 1736 ms. Trace:
I20260202 02:52:57.248010 1882 rpcz_store.cc:276] 0202 02:52:55.511118 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:55.511165 (+ 47us) service_pool.cc:225] Handling call
0202 02:52:57.247814 (+1736649us) ts_tablet_manager.cc:1151] Deleting tablet 455e7d1197c144eda1cf1ef405aeb262
0202 02:52:57.247824 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:57.247931 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1736056}
I20260202 02:52:57.248273 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 1047) took 1735 ms. Trace:
I20260202 02:52:57.248618 1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59a1390e38fd4cc19d650a95c4277169. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:57.250659 1882 rpcz_store.cc:276] 0202 02:52:55.513057 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:55.513102 (+ 45us) service_pool.cc:225] Handling call
0202 02:52:57.248193 (+1735091us) ts_tablet_manager.cc:1151] Deleting tablet 7f227a89344b4aa6a62f6b58f679fbef
0202 02:52:57.248200 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:57.248262 (+ 62us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1734732}
I20260202 02:52:57.251096 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 1048) took 1737 ms. Trace:
I20260202 02:52:57.251174 1882 rpcz_store.cc:276] 0202 02:52:55.513604 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:55.513642 (+ 38us) service_pool.cc:225] Handling call
0202 02:52:57.250959 (+1737317us) ts_tablet_manager.cc:1151] Deleting tablet 470f9ce5e38f4d22b4850e986a4d5c52
0202 02:52:57.250968 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:57.251079 (+ 111us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1737117}
I20260202 02:52:57.251442 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 1049) took 1737 ms. Trace:
I20260202 02:52:57.251515 1882 rpcz_store.cc:276] 0202 02:52:55.513958 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:55.513994 (+ 36us) service_pool.cc:225] Handling call
0202 02:52:57.251356 (+1737362us) ts_tablet_manager.cc:1151] Deleting tablet b34dc1cab5874fa6aee55d79c6e09ade
0202 02:52:57.251363 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:57.251430 (+ 67us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1737185}
I20260202 02:52:57.251667 1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa0c6354eca24e8c8f8bcf79f3a0aca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:57.251803 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 1050) took 1737 ms. Trace:
I20260202 02:52:57.251878 1882 rpcz_store.cc:276] 0202 02:52:55.514282 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:55.516575 (+ 2293us) service_pool.cc:225] Handling call
0202 02:52:57.251710 (+1735135us) ts_tablet_manager.cc:1151] Deleting tablet bb145df67e1a44569a3dbc91c024b584
0202 02:52:57.251719 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:57.251789 (+ 70us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1734888}
I20260202 02:52:57.252117 3289 raft_consensus.cc:493] T 1c8317adc47a4c399f2db650ed3d2e53 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:52:57.252255 3289 raft_consensus.cc:515] T 1c8317adc47a4c399f2db650ed3d2e53 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:57.252758 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6cd9d7fb08e4bfc874ac105b0fafba7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:52:57.252962 3289 leader_election.cc:290] T 1c8317adc47a4c399f2db650ed3d2e53 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539), 1f442ef85029461091de847118c34c0d (127.0.104.129:39403)
I20260202 02:52:57.253300 1882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49890 (request call id 1051) took 1736 ms. Trace:
I20260202 02:52:57.253387 1882 rpcz_store.cc:276] 0202 02:52:55.516963 (+ 0us) service_pool.cc:168] Inserting onto call queue
0202 02:52:55.521269 (+ 4306us) service_pool.cc:225] Handling call
0202 02:52:57.252062 (+1730793us) ts_tablet_manager.cc:1151] Deleting tablet 353bbad7f24f40e3b6316f619c749017
0202 02:52:57.253155 (+ 1093us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0202 02:52:57.253280 (+ 125us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1730448}
I20260202 02:52:57.253763 1882 tablet_replica.cc:333] T 5ffdf97dff3e4dca89507f488c5b0f8e P b1f05820b6034228bdad884cc348716d: stopping tablet replica
I20260202 02:52:57.256672 3181 leader_election.cc:290] T af29c7a2ad9d4797b08052606151a7d5 P b1f05820b6034228bdad884cc348716d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f442ef85029461091de847118c34c0d (127.0.104.129:39403), 1521a6d262b7451bbef8debe970dd6b5 (127.0.104.130:38539)
I20260202 02:52:57.276062 1173 tablet_bootstrap.cc:492] T 3706562a22634eb6b72e36bb6a2d1ae5 P b1f05820b6034228bdad884cc348716d: No bootstrap required, opened a new log
I20260202 02:52:57.276229 1173 ts_tablet_manager.cc:1403] T 3706562a22634eb6b72e36bb6a2d1ae5 P b1f05820b6034228bdad884cc348716d: Time spent bootstrapping tablet: real 0.034s user 0.003s sys 0.026s
I20260202 02:52:57.277326 1173 raft_consensus.cc:359] T 3706562a22634eb6b72e36bb6a2d1ae5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:57.277547 1173 raft_consensus.cc:385] T 3706562a22634eb6b72e36bb6a2d1ae5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:52:57.277608 1173 raft_consensus.cc:740] T 3706562a22634eb6b72e36bb6a2d1ae5 P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f05820b6034228bdad884cc348716d, State: Initialized, Role: FOLLOWER
I20260202 02:52:57.277930 1882 raft_consensus.cc:2243] T 5ffdf97dff3e4dca89507f488c5b0f8e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:52:57.278075 1882 raft_consensus.cc:2272] T 5ffdf97dff3e4dca89507f488c5b0f8e P b1f05820b6034228bdad884cc348716d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:52:57.278637 1882 ts_tablet_manager.cc:1916] T 5ffdf97dff3e4dca89507f488c5b0f8e P b1f05820b6034228bdad884cc348716d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:52:57.280303 1173 consensus_queue.cc:260] T 3706562a22634eb6b72e36bb6a2d1ae5 P b1f05820b6034228bdad884cc348716d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f442ef85029461091de847118c34c0d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 39403 } } peers { permanent_uuid: "1521a6d262b7451bbef8debe970dd6b5" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38539 } } peers { permanent_uuid: "b1f05820b6034228bdad884cc348716d" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 35257 } }
I20260202 02:52:57.283838 1173 ts_tablet_manager.cc:1434] T 3706562a22634eb6b72e36bb6a2d1ae5 P b1f05820b6034228bdad884cc348716d: Time spent starting tablet: real 0.008s user 0.001s sys 0.002s
I20260202 02:52:57.284215 1173 tablet_bootstrap.cc:492] T 745fc215251e4ccb9a2abed02573dd84 P b1f05820b6034228bdad884cc348716d: Bootstrap starting.
I20260202 02:52:57.284219 418 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskYXxklX/build/asan/bin/kudu with pid 975
W20260202 02:52:57.333171 1384 connection.cc:537] server connection from 127.0.104.131:39717 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260202 02:52:57.333189 2259 connection.cc:537] server connection from 127.0.104.131:48919 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260202 02:52:57.333669 418 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskYXxklX/build/asan/bin/kudu pid 1906
I20260202 02:52:57.348948 2355 generic_service.cc:196] Checking for leaks (request via RPC)
W20260202 02:52:57.450460 512 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet de76ca9a98c840d0967a97154b940b60 (table default.Table_7cdb7249dcfe4b5aa11091f41f7bf50e [id=277a1bdd805e49a794c46be1f9a03316], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet de76ca9a98c840d0967a97154b940b60 consensus state has no leader
I20260202 02:52:57.533234 418 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskYXxklX/build/asan/bin/kudu with pid 1906
I20260202 02:52:57.561355 418 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskYXxklX/build/asan/bin/kudu pid 494
I20260202 02:52:57.562602 558 generic_service.cc:196] Checking for leaks (request via RPC)
I20260202 02:52:57.812373 418 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskYXxklX/build/asan/bin/kudu with pid 494
I20260202 02:52:57.850328 418 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskYXxklX/build/asan/bin/kudu pid 1246
I20260202 02:52:57.858008 1523 generic_service.cc:196] Checking for leaks (request via RPC)
I20260202 02:52:58.027191 3304 raft_consensus.cc:493] T 00000000000000000000000000000000 P a9c3e3612bb94fb7b94c308041acfaeb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92a8191cf1e14e4d99f1c378c2d3c85b)
I20260202 02:52:58.027549 418 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskYXxklX/build/asan/bin/kudu with pid 1246
2026-02-02T02:52:58Z chronyd exiting
I20260202 02:52:58.068385 418 master-stress-test.cc:585] Tables created: 485
I20260202 02:52:58.068480 418 master-stress-test.cc:586] Tables altered: 240
I20260202 02:52:58.068534 418 master-stress-test.cc:587] Tables deleted: 210
I20260202 02:52:58.068574 418 master-stress-test.cc:588] Tablets replaced: 103
I20260202 02:52:58.068622 418 master-stress-test.cc:589] Masters restarted: 2
[ OK ] CatalogManagerConfigurations/MasterStressTest.Test/0 (48433 ms)
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/3
2026-02-02T02:52:58Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-02-02T02:52:58Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskYXxklX/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskYXxklX/thirdparty/src/hadoop-3.4.1/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/security.properties
2026-02-02 02:53:05: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskYXxklX/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskYXxklX/thirdparty/src/hadoop-3.4.1/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260202 02:53:14.296126 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskYXxklX/build/asan/bin/kudu
/tmp/dist-test-taskYXxklX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.190:34577
--webserver_interface=127.0.104.190
--webserver_port=0
--builtin_ntp_servers=127.0.104.148:34531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525
--hive_metastore_uris=thrift://127.0.0.1:46545
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260202 02:53:14.653028 3544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260202 02:53:14.653399 3544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260202 02:53:14.653512 3544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260202 02:53:14.662086 3544 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260202 02:53:14.662330 3544 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260202 02:53:14.662446 3544 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260202 02:53:14.662511 3544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260202 02:53:14.662601 3544 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260202 02:53:14.662678 3544 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260202 02:53:14.673197 3544 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:34531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:46545
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.190:34577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.104.190
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3722ed49c19ce54c7feae2fa944e32b1d4ed8d2d
build type FASTDEBUG
built by None at 02 Feb 2026 02:43:23 UTC on 24a791456cd2
build id 10304
ASAN enabled
I20260202 02:53:14.674968 3544 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260202 02:53:14.676896 3544 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260202 02:53:14.685681 3549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:53:14.685926 3552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:53:14.685771 3550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260202 02:53:14.686978 3544 server_base.cc:1060] running on GCE node
I20260202 02:53:14.687865 3544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260202 02:53:14.689559 3544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260202 02:53:14.690814 3544 hybrid_clock.cc:648] HybridClock initialized: now 1770000794690759 us; error 46 us; skew 500 ppm
I20260202 02:53:14.691241 3544 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260202 02:53:14.693897 3544 webserver.cc:492] Webserver started at http://127.0.104.190:33649/ using document root <none> and password file <none>
I20260202 02:53:14.694761 3544 fs_manager.cc:362] Metadata directory not provided
I20260202 02:53:14.694979 3544 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260202 02:53:14.695426 3544 server_base.cc:908] This appears to be a new deployment of Kudu; creating new FS layout
I20260202 02:53:14.698117 3544 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data/instance:
uuid: "666b7e52b5fc4f3397f04960c8b67100"
format_stamp: "Formatted at 2026-02-02 02:53:14 on dist-test-slave-llj2"
I20260202 02:53:14.699054 3544 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/wal/instance:
uuid: "666b7e52b5fc4f3397f04960c8b67100"
format_stamp: "Formatted at 2026-02-02 02:53:14 on dist-test-slave-llj2"
I20260202 02:53:14.705001 3544 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260202 02:53:14.708734 3558 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260202 02:53:14.710289 3544 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260202 02:53:14.710505 3544 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data,/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/wal
uuid: "666b7e52b5fc4f3397f04960c8b67100"
format_stamp: "Formatted at 2026-02-02 02:53:14 on dist-test-slave-llj2"
I20260202 02:53:14.710759 3544 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260202 02:53:14.727579 3544 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260202 02:53:14.728571 3544 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260202 02:53:14.728854 3544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260202 02:53:14.749228 3544 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.190:34577
I20260202 02:53:14.749248 3609 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.190:34577 every 8 connection(s)
I20260202 02:53:14.750907 3544 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data/info.pb
I20260202 02:53:14.755463 3610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:14.760473 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskYXxklX/build/asan/bin/kudu as pid 3544
I20260202 02:53:14.760694 418 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/wal/instance
I20260202 02:53:14.762133 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskYXxklX/build/asan/bin/kudu
/tmp/dist-test-taskYXxklX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.189:40361
--webserver_interface=127.0.104.189
--webserver_port=0
--builtin_ntp_servers=127.0.104.148:34531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525
--hive_metastore_uris=thrift://127.0.0.1:46545
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260202 02:53:14.764225 3610 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } has no permanent_uuid. Determining permanent_uuid...
I20260202 02:53:14.787364 3610 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } has no permanent_uuid. Determining permanent_uuid...
W20260202 02:53:14.788869 3560 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
W20260202 02:53:14.790751 3610 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.189:40361: Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
I20260202 02:53:14.840185 3610 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } attempt: 1
W20260202 02:53:14.842224 3610 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.189:40361: Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
I20260202 02:53:14.910534 3610 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } attempt: 2
W20260202 02:53:14.912451 3610 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.189:40361: Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
I20260202 02:53:15.003743 3610 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } attempt: 3
W20260202 02:53:15.006054 3610 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.189:40361: Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
W20260202 02:53:15.109246 3611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260202 02:53:15.109603 3611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260202 02:53:15.109719 3611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260202 02:53:15.118615 3611 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260202 02:53:15.118719 3611 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260202 02:53:15.118779 3611 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260202 02:53:15.118842 3611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260202 02:53:15.118918 3611 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260202 02:53:15.118974 3611 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260202 02:53:15.129179 3611 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:34531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:46545
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.189:40361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.104.189
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3722ed49c19ce54c7feae2fa944e32b1d4ed8d2d
build type FASTDEBUG
built by None at 02 Feb 2026 02:43:23 UTC on 24a791456cd2
build id 10304
ASAN enabled
I20260202 02:53:15.130836 3611 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260202 02:53:15.132701 3611 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260202 02:53:15.140728 3620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:53:15.140729 3619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:53:15.141893 3622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260202 02:53:15.142961 3611 server_base.cc:1060] running on GCE node
I20260202 02:53:15.143780 3611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260202 02:53:15.145305 3611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260202 02:53:15.146565 3611 hybrid_clock.cc:648] HybridClock initialized: now 1770000795146497 us; error 52 us; skew 500 ppm
I20260202 02:53:15.146946 3611 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260202 02:53:15.149448 3611 webserver.cc:492] Webserver started at http://127.0.104.189:35647/ using document root <none> and password file <none>
I20260202 02:53:15.149359 3610 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } attempt: 4
I20260202 02:53:15.150363 3611 fs_manager.cc:362] Metadata directory not provided
I20260202 02:53:15.150511 3611 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260202 02:53:15.150887 3611 server_base.cc:908] This appears to be a new deployment of Kudu; creating new FS layout
W20260202 02:53:15.151266 3610 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.189:40361: Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
I20260202 02:53:15.153599 3611 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data/instance:
uuid: "3acf71c31a284578b4f8c918eaa50205"
format_stamp: "Formatted at 2026-02-02 02:53:15 on dist-test-slave-llj2"
I20260202 02:53:15.154443 3611 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/wal/instance:
uuid: "3acf71c31a284578b4f8c918eaa50205"
format_stamp: "Formatted at 2026-02-02 02:53:15 on dist-test-slave-llj2"
I20260202 02:53:15.160202 3611 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260202 02:53:15.164112 3628 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260202 02:53:15.165719 3611 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260202 02:53:15.165933 3611 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data,/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/wal
uuid: "3acf71c31a284578b4f8c918eaa50205"
format_stamp: "Formatted at 2026-02-02 02:53:15 on dist-test-slave-llj2"
I20260202 02:53:15.166170 3611 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260202 02:53:15.185523 3611 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260202 02:53:15.186511 3611 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260202 02:53:15.186792 3611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260202 02:53:15.206862 3611 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.189:40361
I20260202 02:53:15.206880 3679 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.189:40361 every 8 connection(s)
I20260202 02:53:15.208441 3611 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data/info.pb
I20260202 02:53:15.209096 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskYXxklX/build/asan/bin/kudu as pid 3611
I20260202 02:53:15.209333 418 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/wal/instance
I20260202 02:53:15.210374 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskYXxklX/build/asan/bin/kudu
/tmp/dist-test-taskYXxklX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.188:42525
--webserver_interface=127.0.104.188
--webserver_port=0
--builtin_ntp_servers=127.0.104.148:34531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525
--hive_metastore_uris=thrift://127.0.0.1:46545
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260202 02:53:15.213415 3680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:15.221217 3680 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } has no permanent_uuid. Determining permanent_uuid...
I20260202 02:53:15.236181 3680 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } has no permanent_uuid. Determining permanent_uuid...
I20260202 02:53:15.246764 3680 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } has no permanent_uuid. Determining permanent_uuid...
W20260202 02:53:15.247957 3632 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
W20260202 02:53:15.249620 3680 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.188:42525: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:15.298900 3680 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } attempt: 1
W20260202 02:53:15.301554 3680 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.188:42525: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:15.369827 3680 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } attempt: 2
W20260202 02:53:15.371971 3680 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.188:42525: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:15.450549 3610 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } attempt: 5
I20260202 02:53:15.456946 3610 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } has no permanent_uuid. Determining permanent_uuid...
W20260202 02:53:15.459236 3610 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.188:42525: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:15.463201 3680 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } attempt: 3
W20260202 02:53:15.465145 3680 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.188:42525: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:15.510490 3610 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } attempt: 1
W20260202 02:53:15.512709 3610 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.188:42525: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
W20260202 02:53:15.565755 3681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260202 02:53:15.566120 3681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260202 02:53:15.566229 3681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260202 02:53:15.575281 3681 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260202 02:53:15.575402 3681 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260202 02:53:15.575485 3681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260202 02:53:15.575551 3681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260202 02:53:15.575639 3681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260202 02:53:15.575708 3681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260202 02:53:15.580993 3610 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } attempt: 2
W20260202 02:53:15.583171 3610 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.188:42525: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:15.587363 3681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:34531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:46545
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.188:42525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.104.188
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3722ed49c19ce54c7feae2fa944e32b1d4ed8d2d
build type FASTDEBUG
built by None at 02 Feb 2026 02:43:23 UTC on 24a791456cd2
build id 10304
ASAN enabled
I20260202 02:53:15.589135 3681 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260202 02:53:15.591063 3681 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260202 02:53:15.600270 3690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:53:15.600356 3689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:53:15.600490 3692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260202 02:53:15.601300 3681 server_base.cc:1060] running on GCE node
I20260202 02:53:15.602145 3681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260202 02:53:15.603801 3681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260202 02:53:15.605079 3681 hybrid_clock.cc:648] HybridClock initialized: now 1770000795605035 us; error 45 us; skew 500 ppm
I20260202 02:53:15.605522 3681 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260202 02:53:15.607939 3681 webserver.cc:492] Webserver started at http://127.0.104.188:42683/ using document root <none> and password file <none>
I20260202 02:53:15.608429 3680 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } attempt: 4
I20260202 02:53:15.608860 3681 fs_manager.cc:362] Metadata directory not provided
I20260202 02:53:15.609004 3681 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260202 02:53:15.609458 3681 server_base.cc:908] This appears to be a new deployment of Kudu; creating new FS layout
W20260202 02:53:15.610304 3680 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.188:42525: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:15.612181 3681 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-2/data/instance:
uuid: "879caa0e6c114ab4855b7a756e252c9e"
format_stamp: "Formatted at 2026-02-02 02:53:15 on dist-test-slave-llj2"
I20260202 02:53:15.613085 3681 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-2/wal/instance:
uuid: "879caa0e6c114ab4855b7a756e252c9e"
format_stamp: "Formatted at 2026-02-02 02:53:15 on dist-test-slave-llj2"
I20260202 02:53:15.619045 3681 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260202 02:53:15.623131 3698 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260202 02:53:15.624881 3681 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260202 02:53:15.625094 3681 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-2/data,/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-2/wal
uuid: "879caa0e6c114ab4855b7a756e252c9e"
format_stamp: "Formatted at 2026-02-02 02:53:15 on dist-test-slave-llj2"
I20260202 02:53:15.625407 3681 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260202 02:53:15.643952 3681 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260202 02:53:15.645042 3681 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260202 02:53:15.645335 3681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260202 02:53:15.665406 3681 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.188:42525
I20260202 02:53:15.665424 3749 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.188:42525 every 8 connection(s)
I20260202 02:53:15.667014 3681 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-2/data/info.pb
I20260202 02:53:15.671765 3750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:15.675493 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskYXxklX/build/asan/bin/kudu as pid 3681
I20260202 02:53:15.675690 418 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-2/wal/instance
I20260202 02:53:15.678476 3750 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } has no permanent_uuid. Determining permanent_uuid...
I20260202 02:53:15.689432 3610 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } attempt: 3
I20260202 02:53:15.695622 3750 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } has no permanent_uuid. Determining permanent_uuid...
I20260202 02:53:15.703227 3750 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } has no permanent_uuid. Determining permanent_uuid...
I20260202 02:53:15.704793 3610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: Bootstrap starting.
I20260202 02:53:15.708024 3610 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:15.709396 3610 log.cc:826] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: Log is configured to *not* fsync() on all Append() calls
I20260202 02:53:15.712236 3610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: No bootstrap required, opened a new log
I20260202 02:53:15.718899 3750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: Bootstrap starting.
I20260202 02:53:15.719821 3610 raft_consensus.cc:359] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:15.720187 3610 raft_consensus.cc:385] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:15.720363 3610 raft_consensus.cc:740] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666b7e52b5fc4f3397f04960c8b67100, State: Initialized, Role: FOLLOWER
I20260202 02:53:15.721149 3610 consensus_queue.cc:260] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:15.722218 3757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:15.722680 3757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:53:15.723698 3610 sys_catalog.cc:565] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: configured and running, proceeding with master startup.
I20260202 02:53:15.724040 3750 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:15.725577 3750 log.cc:826] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: Log is configured to *not* fsync() on all Append() calls
I20260202 02:53:15.729131 3750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: No bootstrap required, opened a new log
W20260202 02:53:15.731927 3564 tablet.cc:2378] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260202 02:53:15.738369 3750 raft_consensus.cc:359] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:15.738968 3750 raft_consensus.cc:385] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:15.739194 3750 raft_consensus.cc:740] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 879caa0e6c114ab4855b7a756e252c9e, State: Initialized, Role: FOLLOWER
I20260202 02:53:15.740933 3750 consensus_queue.cc:260] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:15.742908 3774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:15.743333 3774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:53:15.744422 3750 sys_catalog.cc:565] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [sys.catalog]: configured and running, proceeding with master startup.
W20260202 02:53:15.756170 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:15.756457 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260202 02:53:15.744418 3544 master_runner.cc:186] Error getting master registration for 127.0.104.189:40361: OK, instance_id { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" instance_seqno: 1770000795201270 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260202 02:53:15.764209 3544 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:53:15.768106 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:15.768266 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:15.780049 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:15.780236 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:15.794878 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:15.795038 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:15.807868 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:15.808035 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:15.819572 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:15.819734 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260202 02:53:15.824976 3789 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260202 02:53:15.831267 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:15.831434 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260202 02:53:15.833127 3681 master_runner.cc:186] Error getting master registration for 127.0.104.189:40361: OK, instance_id { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" instance_seqno: 1770000795201270 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260202 02:53:15.839908 3681 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:53:15.842919 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:15.843065 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:15.854787 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:15.854961 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:15.867416 3789 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:15.867627 3789 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:15.869647 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:15.869799 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:15.879171 3789 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:15.879377 3789 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:15.881323 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:15.881466 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:15.890853 3789 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:15.891031 3789 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:15.892977 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:15.893112 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:15.897708 3704 tablet.cc:2378] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260202 02:53:15.902652 3789 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:15.902814 3789 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:15.904703 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:15.904842 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260202 02:53:15.909525 3680 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } attempt: 5
W20260202 02:53:15.914194 3789 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:15.914355 3789 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:15.916227 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:15.916429 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260202 02:53:15.923738 3680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: Bootstrap starting.
W20260202 02:53:15.925737 3789 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:15.925901 3789 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:15.928277 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:15.928452 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260202 02:53:15.929132 3680 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:15.930625 3680 log.cc:826] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: Log is configured to *not* fsync() on all Append() calls
W20260202 02:53:15.937515 3789 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:15.937695 3789 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:15.939950 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:15.940093 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260202 02:53:15.940558 3680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: No bootstrap required, opened a new log
W20260202 02:53:15.950766 3789 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:15.950939 3789 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:15.951476 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:15.951587 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260202 02:53:15.953079 3680 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:15.953476 3680 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:15.953610 3680 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3acf71c31a284578b4f8c918eaa50205, State: Initialized, Role: FOLLOWER
I20260202 02:53:15.954438 3680 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:15.955572 3793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:15.955967 3793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:53:15.957001 3680 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: configured and running, proceeding with master startup.
W20260202 02:53:15.962522 3789 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:15.962695 3789 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:15.964006 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:15.964125 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:15.974287 3789 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:15.974478 3789 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:15.976624 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:15.976759 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:15.987334 3789 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:15.987504 3789 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:15.989125 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:15.989259 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:15.999015 3789 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:15.999444 3789 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.004817 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.005029 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.016697 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.016844 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.018072 3789 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.018204 3789 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.029572 3789 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.029748 3789 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.030253 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.030385 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260202 02:53:16.033010 3611 master_runner.cc:263] No leader master found from master 3acf71c31a284578b4f8c918eaa50205
I20260202 02:53:16.033136 3611 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:53:16.039211 3808 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.039381 3808 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.041236 3789 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.041390 3789 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.041791 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.041898 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.050953 3808 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.051116 3808 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.052934 3789 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.053113 3789 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.053579 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.053696 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.062673 3808 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.062836 3808 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.064741 3789 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.064987 3789 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.065033 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.065146 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.074373 3808 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.074544 3808 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.076774 3789 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.076941 3789 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.078317 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.078449 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.087350 3808 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.087494 3808 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.088979 3789 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.089140 3789 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.090483 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.090621 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.098987 3808 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.099143 3808 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.100623 3789 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260202 02:53:16.101105 3757 raft_consensus.cc:493] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:16.101397 3757 raft_consensus.cc:515] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
W20260202 02:53:16.102263 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.102387 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.102551 3789 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260202 02:53:16.103256 3757 leader_election.cc:290] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
I20260202 02:53:16.104482 3725 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "666b7e52b5fc4f3397f04960c8b67100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879caa0e6c114ab4855b7a756e252c9e" is_pre_election: true
I20260202 02:53:16.105070 3725 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666b7e52b5fc4f3397f04960c8b67100 in term 0.
I20260202 02:53:16.105871 3562 leader_election.cc:304] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e; no voters:
I20260202 02:53:16.107197 3655 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "666b7e52b5fc4f3397f04960c8b67100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3acf71c31a284578b4f8c918eaa50205" is_pre_election: true
I20260202 02:53:16.107797 3655 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666b7e52b5fc4f3397f04960c8b67100 in term 0.
I20260202 02:53:16.108832 3757 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:53:16.109014 3757 raft_consensus.cc:493] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:53:16.109131 3757 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:53:16.110664 3808 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.110795 3808 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260202 02:53:16.113832 3757 raft_consensus.cc:515] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
W20260202 02:53:16.114264 3789 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.114409 3789 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260202 02:53:16.115095 3655 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "666b7e52b5fc4f3397f04960c8b67100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3acf71c31a284578b4f8c918eaa50205"
I20260202 02:53:16.115302 3655 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 0 FOLLOWER]: Advancing to term 1
W20260202 02:53:16.116422 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.116544 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260202 02:53:16.118683 3655 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666b7e52b5fc4f3397f04960c8b67100 in term 1.
I20260202 02:53:16.120262 3560 leader_election.cc:304] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205, 666b7e52b5fc4f3397f04960c8b67100; no voters:
W20260202 02:53:16.122175 3808 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.122301 3808 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260202 02:53:16.122964 3725 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "666b7e52b5fc4f3397f04960c8b67100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879caa0e6c114ab4855b7a756e252c9e"
I20260202 02:53:16.123214 3725 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:16.126605 3725 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666b7e52b5fc4f3397f04960c8b67100 in term 1.
W20260202 02:53:16.127908 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.128033 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.128818 3789 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.128954 3789 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260202 02:53:16.129899 3757 leader_election.cc:290] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 1 election: Requested vote from peers 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
I20260202 02:53:16.130187 3757 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:53:16.130342 3757 raft_consensus.cc:697] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 1 LEADER]: Becoming Leader. State: Replica: 666b7e52b5fc4f3397f04960c8b67100, State: Running, Role: LEADER
I20260202 02:53:16.131500 3757 consensus_queue.cc:237] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
W20260202 02:53:16.133719 3808 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.133839 3808 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.140421 3789 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.140596 3789 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.145311 3808 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.145454 3808 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.146844 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.146986 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260202 02:53:16.147519 3810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 666b7e52b5fc4f3397f04960c8b67100. Latest consensus state: current_term: 1 leader_uuid: "666b7e52b5fc4f3397f04960c8b67100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:16.147873 3810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: This master's current role is: LEADER
W20260202 02:53:16.152833 3789 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.153007 3789 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.156870 3808 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.157021 3808 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.158228 3772 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.158354 3772 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260202 02:53:16.162309 3812 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
W20260202 02:53:16.164418 3789 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.164579 3789 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260202 02:53:16.166579 3812 catalog_manager.cc:1494] Initializing Kudu cluster ID...
W20260202 02:53:16.168407 3808 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.168545 3808 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.176113 3789 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.176296 3789 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.180104 3808 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.180253 3808 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260202 02:53:16.187371 3725 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 1 FOLLOWER]: Refusing update from remote peer 666b7e52b5fc4f3397f04960c8b67100: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260202 02:53:16.188759 3789 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.188938 3789 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260202 02:53:16.188858 3810 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260202 02:53:16.189819 3634 tablet.cc:2378] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260202 02:53:16.191843 3808 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.191980 3808 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260202 02:53:16.199733 3655 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 1 FOLLOWER]: Refusing update from remote peer 666b7e52b5fc4f3397f04960c8b67100: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260202 02:53:16.200826 3810 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:53:16.257153 3793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 666b7e52b5fc4f3397f04960c8b67100. Latest consensus state: current_term: 1 leader_uuid: "666b7e52b5fc4f3397f04960c8b67100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:16.257475 3793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: This master's current role is: FOLLOWER
W20260202 02:53:16.260674 3808 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.260818 3808 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260202 02:53:16.262986 3813 mvcc.cc:204] Tried to move back new op lower bound from 7249923261170315264 to 7249923260966772736. Current Snapshot: MvccSnapshot[applied={T|T < 7249923261170315264}]
I20260202 02:53:16.263886 3815 mvcc.cc:204] Tried to move back new op lower bound from 7249923261170315264 to 7249923260966772736. Current Snapshot: MvccSnapshot[applied={T|T < 7249923261170315264}]
W20260202 02:53:16.267431 3789 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.267599 3789 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260202 02:53:16.268195 3774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 666b7e52b5fc4f3397f04960c8b67100. Latest consensus state: current_term: 1 leader_uuid: "666b7e52b5fc4f3397f04960c8b67100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:16.268453 3774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:53:16.268980 3810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "666b7e52b5fc4f3397f04960c8b67100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:16.269215 3810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: This master's current role is: LEADER
I20260202 02:53:16.273005 3757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "666b7e52b5fc4f3397f04960c8b67100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:16.273312 3757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: This master's current role is: LEADER
I20260202 02:53:16.276356 3814 mvcc.cc:204] Tried to move back new op lower bound from 7249923261170315264 to 7249923260966772736. Current Snapshot: MvccSnapshot[applied={T|T < 7249923261170315264}]
I20260202 02:53:16.277057 3793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "666b7e52b5fc4f3397f04960c8b67100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:16.277288 3793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:53:16.277684 3812 catalog_manager.cc:1357] Generated new cluster ID: 0d95b889b53247a0813a4fda5550ae30
I20260202 02:53:16.277796 3812 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260202 02:53:16.283512 3808 catalog_manager.cc:1269] Loaded cluster ID: 0d95b889b53247a0813a4fda5550ae30
I20260202 02:53:16.283641 3808 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: loading cluster ID for follower catalog manager: success
W20260202 02:53:16.284864 3808 catalog_manager.cc:1586] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260202 02:53:16.285001 3808 catalog_manager.cc:883] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.296648 3808 catalog_manager.cc:1586] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260202 02:53:16.296805 3808 catalog_manager.cc:883] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.302098 3789 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.302248 3789 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260202 02:53:16.302548 3774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "666b7e52b5fc4f3397f04960c8b67100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:16.302754 3774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [sys.catalog]: This master's current role is: FOLLOWER
W20260202 02:53:16.308612 3808 catalog_manager.cc:1586] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260202 02:53:16.308769 3808 catalog_manager.cc:883] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.314200 3789 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260202 02:53:16.314375 3789 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.320541 3808 catalog_manager.cc:1586] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260202 02:53:16.320705 3808 catalog_manager.cc:883] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260202 02:53:16.327178 3789 catalog_manager.cc:1269] Loaded cluster ID: 0d95b889b53247a0813a4fda5550ae30
I20260202 02:53:16.327327 3789 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: loading cluster ID for follower catalog manager: success
W20260202 02:53:16.328817 3789 catalog_manager.cc:1586] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260202 02:53:16.329488 3789 catalog_manager.cc:883] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260202 02:53:16.330281 3812 catalog_manager.cc:1380] Generated new certificate authority record
W20260202 02:53:16.332360 3808 catalog_manager.cc:1586] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260202 02:53:16.332486 3808 catalog_manager.cc:883] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260202 02:53:16.340591 3812 catalog_manager.cc:1514] Loading token signing keys...
I20260202 02:53:16.344066 3789 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: acquiring CA information for follower catalog manager: success
W20260202 02:53:16.345589 3789 catalog_manager.cc:883] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260202 02:53:16.346717 3808 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: acquiring CA information for follower catalog manager: success
W20260202 02:53:16.348454 3808 catalog_manager.cc:883] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.357293 3789 catalog_manager.cc:883] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.360347 3808 catalog_manager.cc:883] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.369050 3789 catalog_manager.cc:883] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.372898 3808 catalog_manager.cc:883] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.380906 3789 catalog_manager.cc:883] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
W20260202 02:53:16.391466 3808 catalog_manager.cc:883] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260202 02:53:16.391920 3812 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: Generated new TSK 0
I20260202 02:53:16.393190 3812 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260202 02:53:16.393797 3789 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260202 02:53:16.394558 3812 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260202 02:53:16.396065 3812 catalog_manager.cc:5876] Last processed Hive Metastore notification event ID: -1
I20260202 02:53:16.404902 3808 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260202 02:53:16.415197 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskYXxklX/build/asan/bin/kudu
/tmp/dist-test-taskYXxklX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.104.129:0
--local_ip_for_outbound_sockets=127.0.104.129
--webserver_interface=127.0.104.129
--webserver_port=0
--tserver_master_addrs=127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525
--builtin_ntp_servers=127.0.104.148:34531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
W20260202 02:53:16.846748 3822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260202 02:53:16.847067 3822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260202 02:53:16.847164 3822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260202 02:53:16.855820 3822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260202 02:53:16.856021 3822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.104.129
I20260202 02:53:16.866531 3822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:34531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.104.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.104.129
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.104.129
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3722ed49c19ce54c7feae2fa944e32b1d4ed8d2d
build type FASTDEBUG
built by None at 02 Feb 2026 02:43:23 UTC on 24a791456cd2
build id 10304
ASAN enabled
I20260202 02:53:16.868278 3822 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260202 02:53:16.870460 3822 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260202 02:53:16.881176 3828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:53:16.881199 3827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:53:16.881263 3830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260202 02:53:16.881805 3822 server_base.cc:1060] running on GCE node
I20260202 02:53:16.882455 3822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260202 02:53:16.883546 3822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260202 02:53:16.884790 3822 hybrid_clock.cc:648] HybridClock initialized: now 1770000796884731 us; error 54 us; skew 500 ppm
I20260202 02:53:16.885121 3822 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260202 02:53:16.887351 3833 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 666b7e52b5fc4f3397f04960c8b67100)
I20260202 02:53:16.887535 3833 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:16.888763 3833 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
I20260202 02:53:16.889438 3724 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3acf71c31a284578b4f8c918eaa50205" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "879caa0e6c114ab4855b7a756e252c9e" is_pre_election: true
I20260202 02:53:16.889402 3585 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3acf71c31a284578b4f8c918eaa50205" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "666b7e52b5fc4f3397f04960c8b67100" is_pre_election: true
I20260202 02:53:16.889673 3724 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3acf71c31a284578b4f8c918eaa50205 in term 1.
I20260202 02:53:16.890237 3632 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205, 879caa0e6c114ab4855b7a756e252c9e; no voters:
I20260202 02:53:16.890642 3833 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260202 02:53:16.890777 3833 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 666b7e52b5fc4f3397f04960c8b67100)
I20260202 02:53:16.890878 3833 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:53:16.891538 3822 webserver.cc:492] Webserver started at http://127.0.104.129:37227/ using document root <none> and password file <none>
I20260202 02:53:16.892674 3822 fs_manager.cc:362] Metadata directory not provided
I20260202 02:53:16.892829 3822 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260202 02:53:16.893271 3822 server_base.cc:908] This appears to be a new deployment of Kudu; creating new FS layout
I20260202 02:53:16.893793 3833 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:16.894546 3833 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 2 election: Requested vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
I20260202 02:53:16.895278 3724 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3acf71c31a284578b4f8c918eaa50205" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "879caa0e6c114ab4855b7a756e252c9e"
I20260202 02:53:16.895501 3724 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 1 FOLLOWER]: Advancing to term 2
I20260202 02:53:16.896570 3585 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3acf71c31a284578b4f8c918eaa50205" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "666b7e52b5fc4f3397f04960c8b67100"
I20260202 02:53:16.896617 3822 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-0/data/instance:
uuid: "d9570a47043b4891b2392af17f405e3d"
format_stamp: "Formatted at 2026-02-02 02:53:16 on dist-test-slave-llj2"
I20260202 02:53:16.897738 3822 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-0/wal/instance:
uuid: "d9570a47043b4891b2392af17f405e3d"
format_stamp: "Formatted at 2026-02-02 02:53:16 on dist-test-slave-llj2"
I20260202 02:53:16.898883 3724 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3acf71c31a284578b4f8c918eaa50205 in term 2.
I20260202 02:53:16.899408 3632 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205, 879caa0e6c114ab4855b7a756e252c9e; no voters: 666b7e52b5fc4f3397f04960c8b67100
I20260202 02:53:16.899744 3833 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 2 FOLLOWER]: Leader election won for term 2
I20260202 02:53:16.900578 3833 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 2 LEADER]: Becoming Leader. State: Replica: 3acf71c31a284578b4f8c918eaa50205, State: Running, Role: LEADER
I20260202 02:53:16.901057 3833 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:16.902558 3836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3acf71c31a284578b4f8c918eaa50205. Latest consensus state: current_term: 2 leader_uuid: "3acf71c31a284578b4f8c918eaa50205" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:16.902778 3836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: This master's current role is: LEADER
I20260202 02:53:16.903368 3838 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260202 02:53:16.904757 3822 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260202 02:53:16.905929 3838 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260202 02:53:16.906975 3838 catalog_manager.cc:1269] Loaded cluster ID: 0d95b889b53247a0813a4fda5550ae30
I20260202 02:53:16.907087 3838 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260202 02:53:16.908587 3838 catalog_manager.cc:1514] Loading token signing keys...
I20260202 02:53:16.909627 3838 catalog_manager.cc:6038] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: Loaded TSK: 0
I20260202 02:53:16.909660 3839 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260202 02:53:16.910683 3838 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260202 02:53:16.911556 3822 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260202 02:53:16.911756 3838 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260202 02:53:16.911770 3822 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-0/wal
uuid: "d9570a47043b4891b2392af17f405e3d"
format_stamp: "Formatted at 2026-02-02 02:53:16 on dist-test-slave-llj2"
I20260202 02:53:16.912050 3822 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260202 02:53:16.912850 3838 catalog_manager.cc:5876] Last processed Hive Metastore notification event ID: -1
I20260202 02:53:16.925213 3822 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260202 02:53:16.926106 3822 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260202 02:53:16.926347 3822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260202 02:53:16.927314 3822 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260202 02:53:16.929150 3822 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260202 02:53:16.929247 3822 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260202 02:53:16.929332 3822 ts_tablet_manager.cc:616] Registered 0 tablets
I20260202 02:53:16.929392 3822 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260202 02:53:16.939085 3724 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 2 FOLLOWER]: Rejecting Update request from peer 666b7e52b5fc4f3397f04960c8b67100 for earlier term 1. Current term is 2. Ops: []
I20260202 02:53:16.939824 3865 consensus_queue.cc:1059] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260202 02:53:16.940871 3873 raft_consensus.cc:3055] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 1 LEADER]: Stepping down as leader of term 1
I20260202 02:53:16.940976 3873 raft_consensus.cc:740] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 666b7e52b5fc4f3397f04960c8b67100, State: Running, Role: LEADER
I20260202 02:53:16.941398 3873 consensus_queue.cc:260] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:16.941911 3873 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 1 FOLLOWER]: Advancing to term 2
W20260202 02:53:16.941929 3560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260202 02:53:16.946393 3772 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260202 02:53:16.966666 3822 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.129:44425
I20260202 02:53:16.966712 3953 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.129:44425 every 8 connection(s)
I20260202 02:53:16.968132 3822 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-0/data/info.pb
I20260202 02:53:16.968583 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskYXxklX/build/asan/bin/kudu as pid 3822
I20260202 02:53:16.968808 418 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-0/wal/instance
I20260202 02:53:16.978382 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskYXxklX/build/asan/bin/kudu
/tmp/dist-test-taskYXxklX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.104.130:0
--local_ip_for_outbound_sockets=127.0.104.130
--webserver_interface=127.0.104.130
--webserver_port=0
--tserver_master_addrs=127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525
--builtin_ntp_servers=127.0.104.148:34531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260202 02:53:17.024399 3954 heartbeater.cc:344] Connected to a master server at 127.0.104.188:42525
I20260202 02:53:17.024773 3954 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:17.025889 3956 heartbeater.cc:344] Connected to a master server at 127.0.104.189:40361
I20260202 02:53:17.025938 3954 heartbeater.cc:507] Master 127.0.104.188:42525 requested a full tablet report, sending...
I20260202 02:53:17.026142 3956 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:17.026754 3956 heartbeater.cc:507] Master 127.0.104.189:40361 requested a full tablet report, sending...
I20260202 02:53:17.029071 3645 ts_manager.cc:194] Registered new tserver with Master: d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425)
I20260202 02:53:17.029146 3714 ts_manager.cc:194] Registered new tserver with Master: d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425)
I20260202 02:53:17.030956 3955 heartbeater.cc:344] Connected to a master server at 127.0.104.190:34577
I20260202 02:53:17.031090 3955 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:17.031396 3645 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.104.129:55807
I20260202 02:53:17.031530 3955 heartbeater.cc:507] Master 127.0.104.190:34577 requested a full tablet report, sending...
I20260202 02:53:17.033429 3574 ts_manager.cc:194] Registered new tserver with Master: d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425)
I20260202 02:53:17.290444 3585 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 2 FOLLOWER]: Refusing update from remote peer 3acf71c31a284578b4f8c918eaa50205: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260202 02:53:17.291270 3836 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260202 02:53:17.293174 3873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3acf71c31a284578b4f8c918eaa50205. Latest consensus state: current_term: 2 leader_uuid: "3acf71c31a284578b4f8c918eaa50205" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:17.293398 3873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:53:17.295339 3833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "3acf71c31a284578b4f8c918eaa50205" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:17.295583 3833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: This master's current role is: LEADER
I20260202 02:53:17.295990 3724 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 2 FOLLOWER]: Refusing update from remote peer 3acf71c31a284578b4f8c918eaa50205: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260202 02:53:17.296747 3836 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260202 02:53:17.297224 3873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "3acf71c31a284578b4f8c918eaa50205" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:17.297430 3873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:53:17.299192 3969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "3acf71c31a284578b4f8c918eaa50205" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:17.299557 3969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:53:17.299629 3967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3acf71c31a284578b4f8c918eaa50205. Latest consensus state: current_term: 2 leader_uuid: "3acf71c31a284578b4f8c918eaa50205" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:17.299880 3967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [sys.catalog]: This master's current role is: FOLLOWER
W20260202 02:53:17.367446 3962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260202 02:53:17.367763 3962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260202 02:53:17.367882 3962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260202 02:53:17.376583 3962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260202 02:53:17.376858 3962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.104.130
I20260202 02:53:17.387863 3962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:34531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.104.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.104.130
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.104.130
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3722ed49c19ce54c7feae2fa944e32b1d4ed8d2d
build type FASTDEBUG
built by None at 02 Feb 2026 02:43:23 UTC on 24a791456cd2
build id 10304
ASAN enabled
I20260202 02:53:17.389760 3962 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260202 02:53:17.391898 3962 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260202 02:53:17.403250 3975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:53:17.403218 3974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:53:17.403640 3977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260202 02:53:17.404342 3962 server_base.cc:1060] running on GCE node
I20260202 02:53:17.404997 3962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260202 02:53:17.406086 3962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260202 02:53:17.407366 3962 hybrid_clock.cc:648] HybridClock initialized: now 1770000797407314 us; error 48 us; skew 500 ppm
I20260202 02:53:17.407716 3962 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260202 02:53:17.410303 3962 webserver.cc:492] Webserver started at http://127.0.104.130:39857/ using document root <none> and password file <none>
I20260202 02:53:17.411132 3962 fs_manager.cc:362] Metadata directory not provided
I20260202 02:53:17.411260 3962 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260202 02:53:17.411701 3962 server_base.cc:908] This appears to be a new deployment of Kudu; creating new FS layout
I20260202 02:53:17.414295 3962 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-1/data/instance:
uuid: "a34c77c818054f87bc19600949531ad0"
format_stamp: "Formatted at 2026-02-02 02:53:17 on dist-test-slave-llj2"
I20260202 02:53:17.415169 3962 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-1/wal/instance:
uuid: "a34c77c818054f87bc19600949531ad0"
format_stamp: "Formatted at 2026-02-02 02:53:17 on dist-test-slave-llj2"
I20260202 02:53:17.421804 3962 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260202 02:53:17.425868 3983 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260202 02:53:17.427572 3962 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260202 02:53:17.427773 3962 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-1/wal
uuid: "a34c77c818054f87bc19600949531ad0"
format_stamp: "Formatted at 2026-02-02 02:53:17 on dist-test-slave-llj2"
I20260202 02:53:17.428014 3962 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260202 02:53:17.451445 3962 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260202 02:53:17.452632 3962 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260202 02:53:17.452955 3962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260202 02:53:17.454034 3962 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260202 02:53:17.456035 3962 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260202 02:53:17.456152 3962 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260202 02:53:17.456271 3962 ts_tablet_manager.cc:616] Registered 0 tablets
I20260202 02:53:17.456373 3962 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260202 02:53:17.489128 3962 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.130:38525
I20260202 02:53:17.489156 4095 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.130:38525 every 8 connection(s)
I20260202 02:53:17.490635 3962 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-1/data/info.pb
I20260202 02:53:17.496842 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskYXxklX/build/asan/bin/kudu as pid 3962
I20260202 02:53:17.497026 418 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-1/wal/instance
I20260202 02:53:17.501348 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskYXxklX/build/asan/bin/kudu
/tmp/dist-test-taskYXxklX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.104.131:0
--local_ip_for_outbound_sockets=127.0.104.131
--webserver_interface=127.0.104.131
--webserver_port=0
--tserver_master_addrs=127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525
--builtin_ntp_servers=127.0.104.148:34531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260202 02:53:17.511404 4096 heartbeater.cc:344] Connected to a master server at 127.0.104.188:42525
I20260202 02:53:17.511521 4097 heartbeater.cc:344] Connected to a master server at 127.0.104.190:34577
I20260202 02:53:17.511338 4098 heartbeater.cc:344] Connected to a master server at 127.0.104.189:40361
I20260202 02:53:17.511880 4097 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:17.511974 4098 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:17.511873 4096 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:17.513082 4096 heartbeater.cc:507] Master 127.0.104.188:42525 requested a full tablet report, sending...
I20260202 02:53:17.514187 4097 heartbeater.cc:507] Master 127.0.104.190:34577 requested a full tablet report, sending...
I20260202 02:53:17.514930 3714 ts_manager.cc:194] Registered new tserver with Master: a34c77c818054f87bc19600949531ad0 (127.0.104.130:38525)
I20260202 02:53:17.514966 4098 heartbeater.cc:507] Master 127.0.104.189:40361 requested a full tablet report, sending...
I20260202 02:53:17.515635 3574 ts_manager.cc:194] Registered new tserver with Master: a34c77c818054f87bc19600949531ad0 (127.0.104.130:38525)
I20260202 02:53:17.516844 3645 ts_manager.cc:194] Registered new tserver with Master: a34c77c818054f87bc19600949531ad0 (127.0.104.130:38525)
I20260202 02:53:17.517719 3645 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.104.130:41623
I20260202 02:53:17.534628 3956 heartbeater.cc:499] Master 127.0.104.189:40361 was elected leader, sending a full tablet report...
I20260202 02:53:17.795826 3873 raft_consensus.cc:493] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3acf71c31a284578b4f8c918eaa50205)
I20260202 02:53:17.796082 3873 raft_consensus.cc:515] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:17.797173 3873 leader_election.cc:290] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
I20260202 02:53:17.797585 3724 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "666b7e52b5fc4f3397f04960c8b67100" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "879caa0e6c114ab4855b7a756e252c9e" is_pre_election: true
I20260202 02:53:17.797614 3654 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "666b7e52b5fc4f3397f04960c8b67100" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3acf71c31a284578b4f8c918eaa50205" is_pre_election: true
I20260202 02:53:17.798388 3562 leader_election.cc:304] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666b7e52b5fc4f3397f04960c8b67100; no voters: 3acf71c31a284578b4f8c918eaa50205, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:17.798877 3873 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260202 02:53:17.799423 3967 raft_consensus.cc:493] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3acf71c31a284578b4f8c918eaa50205)
I20260202 02:53:17.799628 3967 raft_consensus.cc:515] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:17.801100 3967 leader_election.cc:290] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361)
I20260202 02:53:17.801602 3585 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "879caa0e6c114ab4855b7a756e252c9e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "666b7e52b5fc4f3397f04960c8b67100" is_pre_election: true
I20260202 02:53:17.802107 3654 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "879caa0e6c114ab4855b7a756e252c9e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3acf71c31a284578b4f8c918eaa50205" is_pre_election: true
I20260202 02:53:17.802853 3700 leader_election.cc:304] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 879caa0e6c114ab4855b7a756e252c9e; no voters: 3acf71c31a284578b4f8c918eaa50205, 666b7e52b5fc4f3397f04960c8b67100
I20260202 02:53:17.803309 3967 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260202 02:53:17.862259 4103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260202 02:53:17.862612 4103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260202 02:53:17.862740 4103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260202 02:53:17.872488 4103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260202 02:53:17.872740 4103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.104.131
I20260202 02:53:17.883515 4103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:34531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.104.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.104.131
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.104.131
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3722ed49c19ce54c7feae2fa944e32b1d4ed8d2d
build type FASTDEBUG
built by None at 02 Feb 2026 02:43:23 UTC on 24a791456cd2
build id 10304
ASAN enabled
I20260202 02:53:17.885557 4103 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260202 02:53:17.885999 3833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "3acf71c31a284578b4f8c918eaa50205" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:17.886452 3833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: This master's current role is: LEADER
I20260202 02:53:17.888286 4103 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260202 02:53:17.898507 4110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:53:17.899300 4111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:53:17.900038 4113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260202 02:53:17.900307 4103 server_base.cc:1060] running on GCE node
I20260202 02:53:17.901031 4103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260202 02:53:17.902127 4103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260202 02:53:17.903396 4103 hybrid_clock.cc:648] HybridClock initialized: now 1770000797903315 us; error 72 us; skew 500 ppm
I20260202 02:53:17.903761 4103 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260202 02:53:17.906976 4103 webserver.cc:492] Webserver started at http://127.0.104.131:38027/ using document root <none> and password file <none>
I20260202 02:53:17.907864 4103 fs_manager.cc:362] Metadata directory not provided
I20260202 02:53:17.907991 4103 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260202 02:53:17.908454 4103 server_base.cc:908] This appears to be a new deployment of Kudu; creating new FS layout
I20260202 02:53:17.911120 4103 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-2/data/instance:
uuid: "68f130042e0c4b8292b138ff8a322517"
format_stamp: "Formatted at 2026-02-02 02:53:17 on dist-test-slave-llj2"
I20260202 02:53:17.912050 4103 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-2/wal/instance:
uuid: "68f130042e0c4b8292b138ff8a322517"
format_stamp: "Formatted at 2026-02-02 02:53:17 on dist-test-slave-llj2"
I20260202 02:53:17.918008 4103 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260202 02:53:17.921780 4119 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260202 02:53:17.923297 4103 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260202 02:53:17.923491 4103 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-2/wal
uuid: "68f130042e0c4b8292b138ff8a322517"
format_stamp: "Formatted at 2026-02-02 02:53:17 on dist-test-slave-llj2"
I20260202 02:53:17.923734 4103 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260202 02:53:17.938304 4103 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260202 02:53:17.939195 4103 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260202 02:53:17.939463 4103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260202 02:53:17.940501 4103 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260202 02:53:17.942297 4103 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260202 02:53:17.942399 4103 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260202 02:53:17.942513 4103 ts_tablet_manager.cc:616] Registered 0 tablets
I20260202 02:53:17.942595 4103 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260202 02:53:17.978171 4103 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.131:46661
I20260202 02:53:17.978230 4231 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.131:46661 every 8 connection(s)
I20260202 02:53:17.979722 4103 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-2/data/info.pb
I20260202 02:53:17.988969 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskYXxklX/build/asan/bin/kudu as pid 4103
I20260202 02:53:17.989193 418 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-2/wal/instance
I20260202 02:53:17.997851 4233 heartbeater.cc:344] Connected to a master server at 127.0.104.190:34577
I20260202 02:53:17.998270 4233 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:17.999186 4233 heartbeater.cc:507] Master 127.0.104.190:34577 requested a full tablet report, sending...
I20260202 02:53:18.000164 4234 heartbeater.cc:344] Connected to a master server at 127.0.104.189:40361
I20260202 02:53:18.000365 4234 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:18.000932 3574 ts_manager.cc:194] Registered new tserver with Master: 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661)
I20260202 02:53:18.000970 4234 heartbeater.cc:507] Master 127.0.104.189:40361 requested a full tablet report, sending...
I20260202 02:53:18.001261 4232 heartbeater.cc:344] Connected to a master server at 127.0.104.188:42525
I20260202 02:53:18.001411 4232 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:18.001821 4232 heartbeater.cc:507] Master 127.0.104.188:42525 requested a full tablet report, sending...
I20260202 02:53:18.002692 3645 ts_manager.cc:194] Registered new tserver with Master: 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661)
I20260202 02:53:18.002820 3714 ts_manager.cc:194] Registered new tserver with Master: 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661)
I20260202 02:53:18.003723 3645 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.104.131:52029
I20260202 02:53:18.018790 418 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20260202 02:53:18.020073 4098 heartbeater.cc:499] Master 127.0.104.189:40361 was elected leader, sending a full tablet report...
I20260202 02:53:18.050137 3643 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52118:
name: "default.Table_dc0bdab2115f4027a86e680d2995f726"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:18.050017 3644 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52118:
name: "default.Table_2d03ff86a95f4175b2225ecc0d899058"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:18.055655 3636 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52118:
name: "default.Table_a728b4dd173b4f0c8c6b9cce0bcda94d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:18.050752 3641 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52118:
name: "default.Table_2faba70635fa4e55ae56458e06dba641"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:18.051865 3640 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52118:
name: "default.Table_de1272fe656e46328aa237aec594394a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:18.052459 3642 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52118:
name: "default.Table_8d32aa2872ab405fa4f2e7b1dac33f56"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:18.052278 3639 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52118:
name: "default.Table_2c330ab9e5984ba5980e7264050e7a41"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:18.054359 3637 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52118:
name: "default.Table_6a23cf9a48824f7e8d243c355f458ce8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:18.054167 3638 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52118:
name: "default.Table_470455fd011c452b9cceafbc413b1cfe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:18.049911 3645 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52118:
name: "default.Table_4e64388f537d44aaa6d541b90e924c65"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:53:18.063935 3643 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_dc0bdab2115f4027a86e680d2995f726 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:53:18.064761 3644 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_2d03ff86a95f4175b2225ecc0d899058 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:53:18.065230 3636 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_a728b4dd173b4f0c8c6b9cce0bcda94d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:53:18.065659 3641 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_2faba70635fa4e55ae56458e06dba641 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:53:18.078058 3640 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_de1272fe656e46328aa237aec594394a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:53:18.078361 3639 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_2c330ab9e5984ba5980e7264050e7a41 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:53:18.078060 3642 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_8d32aa2872ab405fa4f2e7b1dac33f56 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:53:18.079617 3637 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_6a23cf9a48824f7e8d243c355f458ce8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:53:18.080708 3638 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_470455fd011c452b9cceafbc413b1cfe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:53:18.081702 3645 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_4e64388f537d44aaa6d541b90e924c65 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:53:18.092163 418 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskYXxklX/build/asan/bin/kudu pid 3611
I20260202 02:53:18.093444 3675 generic_service.cc:196] Checking for leaks (request via RPC)
I20260202 02:53:18.240481 418 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskYXxklX/build/asan/bin/kudu with pid 3611
W20260202 02:53:18.267118 4240 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.104.189:40361)
W20260202 02:53:18.275005 4243 negotiation.cc:337] Failed RPC negotiation. Trace:
0202 02:53:18.268640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.104.189:40361 (local address 127.0.0.1:52134)
0202 02:53:18.269403 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0202 02:53:18.269427 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0202 02:53:18.269641 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0202 02:53:18.273254 (+ 3613us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":448}
I20260202 02:53:18.302641 4279 raft_consensus.cc:493] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3acf71c31a284578b4f8c918eaa50205)
I20260202 02:53:18.302837 4279 raft_consensus.cc:515] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:18.304072 3724 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "666b7e52b5fc4f3397f04960c8b67100" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "879caa0e6c114ab4855b7a756e252c9e" is_pre_election: true
I20260202 02:53:18.305053 4279 leader_election.cc:290] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:53:18.314139 3560 leader_election.cc:336] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361): Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
I20260202 02:53:18.314328 3560 leader_election.cc:304] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666b7e52b5fc4f3397f04960c8b67100; no voters: 3acf71c31a284578b4f8c918eaa50205, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:18.314642 4279 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260202 02:53:18.396732 4282 raft_consensus.cc:493] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3acf71c31a284578b4f8c918eaa50205)
I20260202 02:53:18.396927 4282 raft_consensus.cc:515] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:18.398182 3585 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "879caa0e6c114ab4855b7a756e252c9e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "666b7e52b5fc4f3397f04960c8b67100" is_pre_election: true
I20260202 02:53:18.398479 3585 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 879caa0e6c114ab4855b7a756e252c9e in term 2.
I20260202 02:53:18.398650 4282 leader_election.cc:290] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361)
I20260202 02:53:18.399125 3702 leader_election.cc:304] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e; no voters:
W20260202 02:53:18.399266 3700 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
I20260202 02:53:18.399933 4282 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260202 02:53:18.401028 4282 raft_consensus.cc:493] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3acf71c31a284578b4f8c918eaa50205)
I20260202 02:53:18.401232 4282 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 2 FOLLOWER]: Advancing to term 3
I20260202 02:53:18.405030 4282 raft_consensus.cc:515] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:18.406142 4282 leader_election.cc:290] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 3 election: Requested vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361)
I20260202 02:53:18.406720 3585 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "879caa0e6c114ab4855b7a756e252c9e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "666b7e52b5fc4f3397f04960c8b67100"
I20260202 02:53:18.406931 3585 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 2 FOLLOWER]: Advancing to term 3
W20260202 02:53:18.407018 3700 leader_election.cc:336] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361): Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
W20260202 02:53:18.407963 3700 leader_election.cc:336] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361): Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
I20260202 02:53:18.409288 3585 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 879caa0e6c114ab4855b7a756e252c9e in term 3.
I20260202 02:53:18.409765 3702 leader_election.cc:304] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e; no voters: 3acf71c31a284578b4f8c918eaa50205
I20260202 02:53:18.410136 4282 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 3 FOLLOWER]: Leader election won for term 3
I20260202 02:53:18.410722 4282 raft_consensus.cc:697] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 3 LEADER]: Becoming Leader. State: Replica: 879caa0e6c114ab4855b7a756e252c9e, State: Running, Role: LEADER
I20260202 02:53:18.411186 4282 consensus_queue.cc:237] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:18.414355 4285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 879caa0e6c114ab4855b7a756e252c9e. Latest consensus state: current_term: 3 leader_uuid: "879caa0e6c114ab4855b7a756e252c9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:18.414598 4285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [sys.catalog]: This master's current role is: LEADER
I20260202 02:53:18.415292 4287 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260202 02:53:18.418270 4287 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260202 02:53:18.419173 4287 catalog_manager.cc:1269] Loaded cluster ID: 0d95b889b53247a0813a4fda5550ae30
I20260202 02:53:18.419260 4287 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260202 02:53:18.420482 4287 catalog_manager.cc:1514] Loading token signing keys...
I20260202 02:53:18.421346 4287 catalog_manager.cc:6038] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: Loaded TSK: 0
I20260202 02:53:18.422210 4287 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260202 02:53:18.423230 4287 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260202 02:53:18.424074 4287 catalog_manager.cc:5876] Last processed Hive Metastore notification event ID: -1
I20260202 02:53:18.441892 3712 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52366:
name: "default.Table_2c330ab9e5984ba5980e7264050e7a41"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:18.442575 3710 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52366:
name: "default.Table_4e64388f537d44aaa6d541b90e924c65"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:18.443140 3713 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52366:
name: "default.Table_470455fd011c452b9cceafbc413b1cfe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:18.443200 3709 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52366:
name: "default.Table_dc0bdab2115f4027a86e680d2995f726"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:18.443790 3715 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52366:
name: "default.Table_6a23cf9a48824f7e8d243c355f458ce8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:18.443758 3708 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52366:
name: "default.Table_8d32aa2872ab405fa4f2e7b1dac33f56"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:18.444365 3707 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52366:
name: "default.Table_2faba70635fa4e55ae56458e06dba641"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:18.444386 3714 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52366:
name: "default.Table_a728b4dd173b4f0c8c6b9cce0bcda94d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:18.442068 3711 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52366:
name: "default.Table_2d03ff86a95f4175b2225ecc0d899058"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:18.444929 3706 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52366:
name: "default.Table_de1272fe656e46328aa237aec594394a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260202 02:53:18.452520 3710 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_4e64388f537d44aaa6d541b90e924c65 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:53:18.452762 3713 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_470455fd011c452b9cceafbc413b1cfe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:53:18.453058 3712 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_2c330ab9e5984ba5980e7264050e7a41 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:53:18.455302 3708 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_8d32aa2872ab405fa4f2e7b1dac33f56 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:53:18.452523 3715 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_6a23cf9a48824f7e8d243c355f458ce8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:53:18.506029 4234 heartbeater.cc:499] Master 127.0.104.189:40361 was elected leader, sending a full tablet report...
W20260202 02:53:18.508466 4234 heartbeater.cc:646] Failed to heartbeat to 127.0.104.189:40361 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
W20260202 02:53:18.523521 4098 heartbeater.cc:646] Failed to heartbeat to 127.0.104.189:40361 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
W20260202 02:53:18.541662 3956 heartbeater.cc:646] Failed to heartbeat to 127.0.104.189:40361 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
I20260202 02:53:18.623698 3710 catalog_manager.cc:2174] created HMS catalog entry for table [id=4f41a1a5a6844fbfb43b6798b2c6f67d]
I20260202 02:53:18.629369 3585 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 3 FOLLOWER]: Refusing update from remote peer 879caa0e6c114ab4855b7a756e252c9e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 7. (index mismatch)
W20260202 02:53:18.630218 3700 consensus_peers.cc:597] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e -> Peer 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361): Couldn't send request to peer 3acf71c31a284578b4f8c918eaa50205. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260202 02:53:18.630491 4285 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260202 02:53:18.650309 4279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 879caa0e6c114ab4855b7a756e252c9e. Latest consensus state: current_term: 3 leader_uuid: "879caa0e6c114ab4855b7a756e252c9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:18.650547 4279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:53:18.652165 4285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "879caa0e6c114ab4855b7a756e252c9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:18.652463 4285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [sys.catalog]: This master's current role is: LEADER
I20260202 02:53:18.664556 3710 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:52366:
table { table_name: "default.Table_4e64388f537d44aaa6d541b90e924c65" } new_table_name: "default.Table_4ad4563b34454572a4930ad034763780" modify_external_catalogs: true
I20260202 02:53:18.677309 4279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "879caa0e6c114ab4855b7a756e252c9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:18.677515 4279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:53:18.678427 3708 catalog_manager.cc:2174] created HMS catalog entry for table [id=2ab6a7a17803446898f275ae3faa032c]
I20260202 02:53:18.689944 3708 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:52366:
table { table_name: "default.Table_8d32aa2872ab405fa4f2e7b1dac33f56" } new_table_name: "default.Table_042b89c2475b4966b48af7927ba62e21" modify_external_catalogs: true
I20260202 02:53:18.729795 4167 tablet_service.cc:1505] Processing CreateTablet for tablet 704bab1b8eb34766a8f328a0acbca7a0 (DEFAULT_TABLE table=default.table_4e64388f537d44aaa6d541b90e924c65 [id=4f41a1a5a6844fbfb43b6798b2c6f67d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:18.730995 4166 tablet_service.cc:1505] Processing CreateTablet for tablet 33bfead3692c4836baf5b9816464156f (DEFAULT_TABLE table=default.table_8d32aa2872ab405fa4f2e7b1dac33f56 [id=2ab6a7a17803446898f275ae3faa032c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:18.732003 4166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33bfead3692c4836baf5b9816464156f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:18.733661 3889 tablet_service.cc:1505] Processing CreateTablet for tablet 704bab1b8eb34766a8f328a0acbca7a0 (DEFAULT_TABLE table=default.table_4e64388f537d44aaa6d541b90e924c65 [id=4f41a1a5a6844fbfb43b6798b2c6f67d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:18.735267 3889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 704bab1b8eb34766a8f328a0acbca7a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:18.735045 4030 tablet_service.cc:1505] Processing CreateTablet for tablet 33bfead3692c4836baf5b9816464156f (DEFAULT_TABLE table=default.table_8d32aa2872ab405fa4f2e7b1dac33f56 [id=2ab6a7a17803446898f275ae3faa032c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:18.736614 3888 tablet_service.cc:1505] Processing CreateTablet for tablet 33bfead3692c4836baf5b9816464156f (DEFAULT_TABLE table=default.table_8d32aa2872ab405fa4f2e7b1dac33f56 [id=2ab6a7a17803446898f275ae3faa032c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:18.736838 4031 tablet_service.cc:1505] Processing CreateTablet for tablet 704bab1b8eb34766a8f328a0acbca7a0 (DEFAULT_TABLE table=default.table_4e64388f537d44aaa6d541b90e924c65 [id=4f41a1a5a6844fbfb43b6798b2c6f67d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:18.737144 4030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33bfead3692c4836baf5b9816464156f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:18.737159 3888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33bfead3692c4836baf5b9816464156f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:18.737582 4031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 704bab1b8eb34766a8f328a0acbca7a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:18.739400 4167 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 704bab1b8eb34766a8f328a0acbca7a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:18.763617 4303 tablet_bootstrap.cc:492] T 33bfead3692c4836baf5b9816464156f P d9570a47043b4891b2392af17f405e3d: Bootstrap starting.
I20260202 02:53:18.767522 3715 catalog_manager.cc:2174] created HMS catalog entry for table [id=0048dd71261d4204ac5548c190957aa9]
I20260202 02:53:18.767688 4303 tablet_bootstrap.cc:654] T 33bfead3692c4836baf5b9816464156f P d9570a47043b4891b2392af17f405e3d: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:18.770445 4301 tablet_bootstrap.cc:492] T 704bab1b8eb34766a8f328a0acbca7a0 P a34c77c818054f87bc19600949531ad0: Bootstrap starting.
I20260202 02:53:18.773638 4302 tablet_bootstrap.cc:492] T 704bab1b8eb34766a8f328a0acbca7a0 P 68f130042e0c4b8292b138ff8a322517: Bootstrap starting.
I20260202 02:53:18.776525 3715 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:52366:
table { table_name: "default.Table_6a23cf9a48824f7e8d243c355f458ce8" } new_table_name: "default.Table_824602d24d0345759eda1443b7232548" modify_external_catalogs: true
I20260202 02:53:18.780061 4301 tablet_bootstrap.cc:654] T 704bab1b8eb34766a8f328a0acbca7a0 P a34c77c818054f87bc19600949531ad0: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:18.781661 4301 log.cc:826] T 704bab1b8eb34766a8f328a0acbca7a0 P a34c77c818054f87bc19600949531ad0: Log is configured to *not* fsync() on all Append() calls
I20260202 02:53:18.785737 3889 tablet_service.cc:1505] Processing CreateTablet for tablet 313ff5ec7131498aaaa0c832abea2c56 (DEFAULT_TABLE table=default.table_6a23cf9a48824f7e8d243c355f458ce8 [id=0048dd71261d4204ac5548c190957aa9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:18.786237 3889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 313ff5ec7131498aaaa0c832abea2c56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:18.786398 4031 tablet_service.cc:1505] Processing CreateTablet for tablet 313ff5ec7131498aaaa0c832abea2c56 (DEFAULT_TABLE table=default.table_6a23cf9a48824f7e8d243c355f458ce8 [id=0048dd71261d4204ac5548c190957aa9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:18.786871 4031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 313ff5ec7131498aaaa0c832abea2c56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:18.794149 4167 tablet_service.cc:1505] Processing CreateTablet for tablet 313ff5ec7131498aaaa0c832abea2c56 (DEFAULT_TABLE table=default.table_6a23cf9a48824f7e8d243c355f458ce8 [id=0048dd71261d4204ac5548c190957aa9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:18.794621 4302 tablet_bootstrap.cc:654] T 704bab1b8eb34766a8f328a0acbca7a0 P 68f130042e0c4b8292b138ff8a322517: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:18.794736 4167 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 313ff5ec7131498aaaa0c832abea2c56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:18.796257 4302 log.cc:826] T 704bab1b8eb34766a8f328a0acbca7a0 P 68f130042e0c4b8292b138ff8a322517: Log is configured to *not* fsync() on all Append() calls
I20260202 02:53:18.797791 4303 log.cc:826] T 33bfead3692c4836baf5b9816464156f P d9570a47043b4891b2392af17f405e3d: Log is configured to *not* fsync() on all Append() calls
I20260202 02:53:18.807824 4302 tablet_bootstrap.cc:492] T 704bab1b8eb34766a8f328a0acbca7a0 P 68f130042e0c4b8292b138ff8a322517: No bootstrap required, opened a new log
I20260202 02:53:18.808827 4302 ts_tablet_manager.cc:1403] T 704bab1b8eb34766a8f328a0acbca7a0 P 68f130042e0c4b8292b138ff8a322517: Time spent bootstrapping tablet: real 0.035s user 0.010s sys 0.000s
I20260202 02:53:18.809634 4301 tablet_bootstrap.cc:492] T 704bab1b8eb34766a8f328a0acbca7a0 P a34c77c818054f87bc19600949531ad0: No bootstrap required, opened a new log
I20260202 02:53:18.810048 4301 ts_tablet_manager.cc:1403] T 704bab1b8eb34766a8f328a0acbca7a0 P a34c77c818054f87bc19600949531ad0: Time spent bootstrapping tablet: real 0.040s user 0.015s sys 0.018s
I20260202 02:53:18.816701 4303 tablet_bootstrap.cc:492] T 33bfead3692c4836baf5b9816464156f P d9570a47043b4891b2392af17f405e3d: No bootstrap required, opened a new log
I20260202 02:53:18.817135 4303 ts_tablet_manager.cc:1403] T 33bfead3692c4836baf5b9816464156f P d9570a47043b4891b2392af17f405e3d: Time spent bootstrapping tablet: real 0.054s user 0.009s sys 0.000s
I20260202 02:53:18.817552 4301 raft_consensus.cc:359] T 704bab1b8eb34766a8f328a0acbca7a0 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } }
I20260202 02:53:18.818140 4301 raft_consensus.cc:385] T 704bab1b8eb34766a8f328a0acbca7a0 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:18.818298 4301 raft_consensus.cc:740] T 704bab1b8eb34766a8f328a0acbca7a0 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a34c77c818054f87bc19600949531ad0, State: Initialized, Role: FOLLOWER
I20260202 02:53:18.819200 4301 consensus_queue.cc:260] T 704bab1b8eb34766a8f328a0acbca7a0 P a34c77c818054f87bc19600949531ad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } }
I20260202 02:53:18.825558 4303 raft_consensus.cc:359] T 33bfead3692c4836baf5b9816464156f P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
I20260202 02:53:18.826135 4303 raft_consensus.cc:385] T 33bfead3692c4836baf5b9816464156f P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:18.826251 4303 raft_consensus.cc:740] T 33bfead3692c4836baf5b9816464156f P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9570a47043b4891b2392af17f405e3d, State: Initialized, Role: FOLLOWER
I20260202 02:53:18.830996 4302 raft_consensus.cc:359] T 704bab1b8eb34766a8f328a0acbca7a0 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } }
I20260202 02:53:18.831574 4302 raft_consensus.cc:385] T 704bab1b8eb34766a8f328a0acbca7a0 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:18.831707 4302 raft_consensus.cc:740] T 704bab1b8eb34766a8f328a0acbca7a0 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f130042e0c4b8292b138ff8a322517, State: Initialized, Role: FOLLOWER
I20260202 02:53:18.832775 4302 consensus_queue.cc:260] T 704bab1b8eb34766a8f328a0acbca7a0 P 68f130042e0c4b8292b138ff8a322517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } }
I20260202 02:53:18.833932 4303 consensus_queue.cc:260] T 33bfead3692c4836baf5b9816464156f P d9570a47043b4891b2392af17f405e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
I20260202 02:53:18.838138 4301 ts_tablet_manager.cc:1434] T 704bab1b8eb34766a8f328a0acbca7a0 P a34c77c818054f87bc19600949531ad0: Time spent starting tablet: real 0.028s user 0.007s sys 0.018s
I20260202 02:53:18.838587 4096 heartbeater.cc:499] Master 127.0.104.188:42525 was elected leader, sending a full tablet report...
I20260202 02:53:18.838888 4301 tablet_bootstrap.cc:492] T 33bfead3692c4836baf5b9816464156f P a34c77c818054f87bc19600949531ad0: Bootstrap starting.
I20260202 02:53:18.841063 4301 tablet_bootstrap.cc:654] T 33bfead3692c4836baf5b9816464156f P a34c77c818054f87bc19600949531ad0: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:18.853093 4303 ts_tablet_manager.cc:1434] T 33bfead3692c4836baf5b9816464156f P d9570a47043b4891b2392af17f405e3d: Time spent starting tablet: real 0.036s user 0.010s sys 0.004s
I20260202 02:53:18.854614 4232 heartbeater.cc:499] Master 127.0.104.188:42525 was elected leader, sending a full tablet report...
I20260202 02:53:18.859251 4302 ts_tablet_manager.cc:1434] T 704bab1b8eb34766a8f328a0acbca7a0 P 68f130042e0c4b8292b138ff8a322517: Time spent starting tablet: real 0.050s user 0.014s sys 0.000s
I20260202 02:53:18.864691 4302 tablet_bootstrap.cc:492] T 33bfead3692c4836baf5b9816464156f P 68f130042e0c4b8292b138ff8a322517: Bootstrap starting.
I20260202 02:53:18.866921 4302 tablet_bootstrap.cc:654] T 33bfead3692c4836baf5b9816464156f P 68f130042e0c4b8292b138ff8a322517: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:18.869364 3954 heartbeater.cc:499] Master 127.0.104.188:42525 was elected leader, sending a full tablet report...
I20260202 02:53:18.871549 3713 catalog_manager.cc:2174] created HMS catalog entry for table [id=e28252bb319446c39056986e8b98de30]
I20260202 02:53:18.873152 4303 tablet_bootstrap.cc:492] T 704bab1b8eb34766a8f328a0acbca7a0 P d9570a47043b4891b2392af17f405e3d: Bootstrap starting.
I20260202 02:53:18.875326 4303 tablet_bootstrap.cc:654] T 704bab1b8eb34766a8f328a0acbca7a0 P d9570a47043b4891b2392af17f405e3d: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:18.877856 4302 tablet_bootstrap.cc:492] T 33bfead3692c4836baf5b9816464156f P 68f130042e0c4b8292b138ff8a322517: No bootstrap required, opened a new log
I20260202 02:53:18.878044 4302 ts_tablet_manager.cc:1403] T 33bfead3692c4836baf5b9816464156f P 68f130042e0c4b8292b138ff8a322517: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.000s
I20260202 02:53:18.879136 4302 raft_consensus.cc:359] T 33bfead3692c4836baf5b9816464156f P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
I20260202 02:53:18.879355 4302 raft_consensus.cc:385] T 33bfead3692c4836baf5b9816464156f P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:18.879439 4302 raft_consensus.cc:740] T 33bfead3692c4836baf5b9816464156f P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f130042e0c4b8292b138ff8a322517, State: Initialized, Role: FOLLOWER
I20260202 02:53:18.880311 4302 consensus_queue.cc:260] T 33bfead3692c4836baf5b9816464156f P 68f130042e0c4b8292b138ff8a322517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
I20260202 02:53:18.880893 4301 tablet_bootstrap.cc:492] T 33bfead3692c4836baf5b9816464156f P a34c77c818054f87bc19600949531ad0: No bootstrap required, opened a new log
I20260202 02:53:18.881049 4301 ts_tablet_manager.cc:1403] T 33bfead3692c4836baf5b9816464156f P a34c77c818054f87bc19600949531ad0: Time spent bootstrapping tablet: real 0.042s user 0.028s sys 0.013s
I20260202 02:53:18.882072 4301 raft_consensus.cc:359] T 33bfead3692c4836baf5b9816464156f P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
I20260202 02:53:18.882352 4301 raft_consensus.cc:385] T 33bfead3692c4836baf5b9816464156f P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:18.882449 4301 raft_consensus.cc:740] T 33bfead3692c4836baf5b9816464156f P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a34c77c818054f87bc19600949531ad0, State: Initialized, Role: FOLLOWER
I20260202 02:53:18.882761 4301 consensus_queue.cc:260] T 33bfead3692c4836baf5b9816464156f P a34c77c818054f87bc19600949531ad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
I20260202 02:53:18.883283 4301 ts_tablet_manager.cc:1434] T 33bfead3692c4836baf5b9816464156f P a34c77c818054f87bc19600949531ad0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:53:18.883561 4301 tablet_bootstrap.cc:492] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0: Bootstrap starting.
I20260202 02:53:18.885344 4301 tablet_bootstrap.cc:654] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:18.890156 4302 ts_tablet_manager.cc:1434] T 33bfead3692c4836baf5b9816464156f P 68f130042e0c4b8292b138ff8a322517: Time spent starting tablet: real 0.012s user 0.003s sys 0.000s
I20260202 02:53:18.893628 4301 tablet_bootstrap.cc:492] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0: No bootstrap required, opened a new log
I20260202 02:53:18.893834 4301 ts_tablet_manager.cc:1403] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260202 02:53:18.894166 4302 tablet_bootstrap.cc:492] T 313ff5ec7131498aaaa0c832abea2c56 P 68f130042e0c4b8292b138ff8a322517: Bootstrap starting.
I20260202 02:53:18.894678 4301 raft_consensus.cc:359] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
I20260202 02:53:18.894888 4301 raft_consensus.cc:385] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:18.894948 3889 tablet_service.cc:1505] Processing CreateTablet for tablet b5a121d55d9d498ba76bb4dcb24d0e20 (DEFAULT_TABLE table=default.table_470455fd011c452b9cceafbc413b1cfe [id=e28252bb319446c39056986e8b98de30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:18.895298 4167 tablet_service.cc:1505] Processing CreateTablet for tablet b5a121d55d9d498ba76bb4dcb24d0e20 (DEFAULT_TABLE table=default.table_470455fd011c452b9cceafbc413b1cfe [id=e28252bb319446c39056986e8b98de30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:18.895403 3889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5a121d55d9d498ba76bb4dcb24d0e20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:18.895697 4167 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5a121d55d9d498ba76bb4dcb24d0e20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:18.895965 4302 tablet_bootstrap.cc:654] T 313ff5ec7131498aaaa0c832abea2c56 P 68f130042e0c4b8292b138ff8a322517: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:18.897876 4303 tablet_bootstrap.cc:492] T 704bab1b8eb34766a8f328a0acbca7a0 P d9570a47043b4891b2392af17f405e3d: No bootstrap required, opened a new log
I20260202 02:53:18.898041 4303 ts_tablet_manager.cc:1403] T 704bab1b8eb34766a8f328a0acbca7a0 P d9570a47043b4891b2392af17f405e3d: Time spent bootstrapping tablet: real 0.025s user 0.003s sys 0.000s
I20260202 02:53:18.899078 4303 raft_consensus.cc:359] T 704bab1b8eb34766a8f328a0acbca7a0 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } }
I20260202 02:53:18.899269 4303 raft_consensus.cc:385] T 704bab1b8eb34766a8f328a0acbca7a0 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:18.899366 4303 raft_consensus.cc:740] T 704bab1b8eb34766a8f328a0acbca7a0 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9570a47043b4891b2392af17f405e3d, State: Initialized, Role: FOLLOWER
I20260202 02:53:18.899698 4303 consensus_queue.cc:260] T 704bab1b8eb34766a8f328a0acbca7a0 P d9570a47043b4891b2392af17f405e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } }
I20260202 02:53:18.899299 4031 tablet_service.cc:1505] Processing CreateTablet for tablet b5a121d55d9d498ba76bb4dcb24d0e20 (DEFAULT_TABLE table=default.table_470455fd011c452b9cceafbc413b1cfe [id=e28252bb319446c39056986e8b98de30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:18.903375 4031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5a121d55d9d498ba76bb4dcb24d0e20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:18.904289 4303 ts_tablet_manager.cc:1434] T 704bab1b8eb34766a8f328a0acbca7a0 P d9570a47043b4891b2392af17f405e3d: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260202 02:53:18.907524 4303 tablet_bootstrap.cc:492] T 313ff5ec7131498aaaa0c832abea2c56 P d9570a47043b4891b2392af17f405e3d: Bootstrap starting.
I20260202 02:53:18.909318 4303 tablet_bootstrap.cc:654] T 313ff5ec7131498aaaa0c832abea2c56 P d9570a47043b4891b2392af17f405e3d: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:18.910297 4301 raft_consensus.cc:740] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a34c77c818054f87bc19600949531ad0, State: Initialized, Role: FOLLOWER
I20260202 02:53:18.910641 4301 consensus_queue.cc:260] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
I20260202 02:53:18.911275 4301 ts_tablet_manager.cc:1434] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0: Time spent starting tablet: real 0.017s user 0.001s sys 0.001s
I20260202 02:53:18.911648 4301 tablet_bootstrap.cc:492] T b5a121d55d9d498ba76bb4dcb24d0e20 P a34c77c818054f87bc19600949531ad0: Bootstrap starting.
I20260202 02:53:18.913515 4301 tablet_bootstrap.cc:654] T b5a121d55d9d498ba76bb4dcb24d0e20 P a34c77c818054f87bc19600949531ad0: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:18.918172 4302 tablet_bootstrap.cc:492] T 313ff5ec7131498aaaa0c832abea2c56 P 68f130042e0c4b8292b138ff8a322517: No bootstrap required, opened a new log
I20260202 02:53:18.918402 4302 ts_tablet_manager.cc:1403] T 313ff5ec7131498aaaa0c832abea2c56 P 68f130042e0c4b8292b138ff8a322517: Time spent bootstrapping tablet: real 0.025s user 0.003s sys 0.000s
I20260202 02:53:18.919282 4302 raft_consensus.cc:359] T 313ff5ec7131498aaaa0c832abea2c56 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
I20260202 02:53:18.919520 4302 raft_consensus.cc:385] T 313ff5ec7131498aaaa0c832abea2c56 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:18.919636 4302 raft_consensus.cc:740] T 313ff5ec7131498aaaa0c832abea2c56 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f130042e0c4b8292b138ff8a322517, State: Initialized, Role: FOLLOWER
I20260202 02:53:18.919950 4302 consensus_queue.cc:260] T 313ff5ec7131498aaaa0c832abea2c56 P 68f130042e0c4b8292b138ff8a322517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
I20260202 02:53:18.922971 4302 ts_tablet_manager.cc:1434] T 313ff5ec7131498aaaa0c832abea2c56 P 68f130042e0c4b8292b138ff8a322517: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260202 02:53:18.927526 4302 tablet_bootstrap.cc:492] T b5a121d55d9d498ba76bb4dcb24d0e20 P 68f130042e0c4b8292b138ff8a322517: Bootstrap starting.
I20260202 02:53:18.930819 4303 tablet_bootstrap.cc:492] T 313ff5ec7131498aaaa0c832abea2c56 P d9570a47043b4891b2392af17f405e3d: No bootstrap required, opened a new log
I20260202 02:53:18.931037 4303 ts_tablet_manager.cc:1403] T 313ff5ec7131498aaaa0c832abea2c56 P d9570a47043b4891b2392af17f405e3d: Time spent bootstrapping tablet: real 0.024s user 0.000s sys 0.003s
I20260202 02:53:18.931892 4301 tablet_bootstrap.cc:492] T b5a121d55d9d498ba76bb4dcb24d0e20 P a34c77c818054f87bc19600949531ad0: No bootstrap required, opened a new log
I20260202 02:53:18.931927 4303 raft_consensus.cc:359] T 313ff5ec7131498aaaa0c832abea2c56 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
I20260202 02:53:18.932139 4303 raft_consensus.cc:385] T 313ff5ec7131498aaaa0c832abea2c56 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:18.932230 4303 raft_consensus.cc:740] T 313ff5ec7131498aaaa0c832abea2c56 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9570a47043b4891b2392af17f405e3d, State: Initialized, Role: FOLLOWER
I20260202 02:53:18.932238 4301 ts_tablet_manager.cc:1403] T b5a121d55d9d498ba76bb4dcb24d0e20 P a34c77c818054f87bc19600949531ad0: Time spent bootstrapping tablet: real 0.021s user 0.016s sys 0.004s
I20260202 02:53:18.932708 4303 consensus_queue.cc:260] T 313ff5ec7131498aaaa0c832abea2c56 P d9570a47043b4891b2392af17f405e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
I20260202 02:53:18.933528 4301 raft_consensus.cc:359] T b5a121d55d9d498ba76bb4dcb24d0e20 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:18.933815 4301 raft_consensus.cc:385] T b5a121d55d9d498ba76bb4dcb24d0e20 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:18.933956 4301 raft_consensus.cc:740] T b5a121d55d9d498ba76bb4dcb24d0e20 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a34c77c818054f87bc19600949531ad0, State: Initialized, Role: FOLLOWER
I20260202 02:53:18.934266 4301 consensus_queue.cc:260] T b5a121d55d9d498ba76bb4dcb24d0e20 P a34c77c818054f87bc19600949531ad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:18.935796 4301 ts_tablet_manager.cc:1434] T b5a121d55d9d498ba76bb4dcb24d0e20 P a34c77c818054f87bc19600949531ad0: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260202 02:53:18.936620 4303 ts_tablet_manager.cc:1434] T 313ff5ec7131498aaaa0c832abea2c56 P d9570a47043b4891b2392af17f405e3d: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260202 02:53:18.939172 4303 tablet_bootstrap.cc:492] T b5a121d55d9d498ba76bb4dcb24d0e20 P d9570a47043b4891b2392af17f405e3d: Bootstrap starting.
I20260202 02:53:18.940929 4303 tablet_bootstrap.cc:654] T b5a121d55d9d498ba76bb4dcb24d0e20 P d9570a47043b4891b2392af17f405e3d: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:18.942648 4302 tablet_bootstrap.cc:654] T b5a121d55d9d498ba76bb4dcb24d0e20 P 68f130042e0c4b8292b138ff8a322517: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:18.945502 4303 tablet_bootstrap.cc:492] T b5a121d55d9d498ba76bb4dcb24d0e20 P d9570a47043b4891b2392af17f405e3d: No bootstrap required, opened a new log
I20260202 02:53:18.945713 4303 ts_tablet_manager.cc:1403] T b5a121d55d9d498ba76bb4dcb24d0e20 P d9570a47043b4891b2392af17f405e3d: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260202 02:53:18.946493 4303 raft_consensus.cc:359] T b5a121d55d9d498ba76bb4dcb24d0e20 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:18.946650 4303 raft_consensus.cc:385] T b5a121d55d9d498ba76bb4dcb24d0e20 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:18.946736 4303 raft_consensus.cc:740] T b5a121d55d9d498ba76bb4dcb24d0e20 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9570a47043b4891b2392af17f405e3d, State: Initialized, Role: FOLLOWER
I20260202 02:53:18.946954 4303 consensus_queue.cc:260] T b5a121d55d9d498ba76bb4dcb24d0e20 P d9570a47043b4891b2392af17f405e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:18.947732 4303 ts_tablet_manager.cc:1434] T b5a121d55d9d498ba76bb4dcb24d0e20 P d9570a47043b4891b2392af17f405e3d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:53:18.952057 4302 tablet_bootstrap.cc:492] T b5a121d55d9d498ba76bb4dcb24d0e20 P 68f130042e0c4b8292b138ff8a322517: No bootstrap required, opened a new log
I20260202 02:53:18.952237 4302 ts_tablet_manager.cc:1403] T b5a121d55d9d498ba76bb4dcb24d0e20 P 68f130042e0c4b8292b138ff8a322517: Time spent bootstrapping tablet: real 0.025s user 0.006s sys 0.000s
I20260202 02:53:18.954170 4302 raft_consensus.cc:359] T b5a121d55d9d498ba76bb4dcb24d0e20 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:18.954365 4302 raft_consensus.cc:385] T b5a121d55d9d498ba76bb4dcb24d0e20 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:18.954454 4302 raft_consensus.cc:740] T b5a121d55d9d498ba76bb4dcb24d0e20 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f130042e0c4b8292b138ff8a322517, State: Initialized, Role: FOLLOWER
I20260202 02:53:18.954687 4302 consensus_queue.cc:260] T b5a121d55d9d498ba76bb4dcb24d0e20 P 68f130042e0c4b8292b138ff8a322517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:18.954983 3713 master_service.cc:954] ReplaceTablet: received request to replace tablet 704bab1b8eb34766a8f328a0acbca7a0 from {username='slave'} at 127.0.0.1:52366
I20260202 02:53:18.956125 4302 ts_tablet_manager.cc:1434] T b5a121d55d9d498ba76bb4dcb24d0e20 P 68f130042e0c4b8292b138ff8a322517: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260202 02:53:18.964566 3713 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: Sending DeleteTablet for 3 replicas of tablet 704bab1b8eb34766a8f328a0acbca7a0
I20260202 02:53:18.965970 4167 tablet_service.cc:1552] Processing DeleteTablet for tablet 704bab1b8eb34766a8f328a0acbca7a0 with delete_type TABLET_DATA_DELETED (replaced by tablet 76c092d3e26c4906964e0f852ac24448) from {username='slave'} at 127.0.0.1:37318
I20260202 02:53:18.966635 3889 tablet_service.cc:1552] Processing DeleteTablet for tablet 704bab1b8eb34766a8f328a0acbca7a0 with delete_type TABLET_DATA_DELETED (replaced by tablet 76c092d3e26c4906964e0f852ac24448) from {username='slave'} at 127.0.0.1:39530
I20260202 02:53:18.966779 4031 tablet_service.cc:1552] Processing DeleteTablet for tablet 704bab1b8eb34766a8f328a0acbca7a0 with delete_type TABLET_DATA_DELETED (replaced by tablet 76c092d3e26c4906964e0f852ac24448) from {username='slave'} at 127.0.0.1:38036
W20260202 02:53:18.982764 4235 tablet.cc:2378] T 313ff5ec7131498aaaa0c832abea2c56 P 68f130042e0c4b8292b138ff8a322517: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260202 02:53:18.984437 4030 tablet_service.cc:1505] Processing CreateTablet for tablet 76c092d3e26c4906964e0f852ac24448 (DEFAULT_TABLE table=default.table_4e64388f537d44aaa6d541b90e924c65 [id=4f41a1a5a6844fbfb43b6798b2c6f67d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:18.984719 3888 tablet_service.cc:1505] Processing CreateTablet for tablet 76c092d3e26c4906964e0f852ac24448 (DEFAULT_TABLE table=default.table_4e64388f537d44aaa6d541b90e924c65 [id=4f41a1a5a6844fbfb43b6798b2c6f67d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:18.984968 4030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76c092d3e26c4906964e0f852ac24448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:18.985185 3888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76c092d3e26c4906964e0f852ac24448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:18.987692 4319 tablet_replica.cc:333] T 704bab1b8eb34766a8f328a0acbca7a0 P 68f130042e0c4b8292b138ff8a322517: stopping tablet replica
I20260202 02:53:18.988202 4319 raft_consensus.cc:2243] T 704bab1b8eb34766a8f328a0acbca7a0 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:53:18.988670 4319 raft_consensus.cc:2272] T 704bab1b8eb34766a8f328a0acbca7a0 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:53:18.989581 4320 tablet_replica.cc:333] T 704bab1b8eb34766a8f328a0acbca7a0 P d9570a47043b4891b2392af17f405e3d: stopping tablet replica
I20260202 02:53:18.990100 4320 raft_consensus.cc:2243] T 704bab1b8eb34766a8f328a0acbca7a0 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:53:18.990563 4320 raft_consensus.cc:2272] T 704bab1b8eb34766a8f328a0acbca7a0 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:53:18.991406 4321 tablet_replica.cc:333] T 704bab1b8eb34766a8f328a0acbca7a0 P a34c77c818054f87bc19600949531ad0: stopping tablet replica
I20260202 02:53:18.991572 4303 tablet_bootstrap.cc:492] T 76c092d3e26c4906964e0f852ac24448 P d9570a47043b4891b2392af17f405e3d: Bootstrap starting.
I20260202 02:53:18.991961 4321 raft_consensus.cc:2243] T 704bab1b8eb34766a8f328a0acbca7a0 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:53:18.992554 4321 raft_consensus.cc:2272] T 704bab1b8eb34766a8f328a0acbca7a0 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:53:18.993374 4303 tablet_bootstrap.cc:654] T 76c092d3e26c4906964e0f852ac24448 P d9570a47043b4891b2392af17f405e3d: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:18.994542 4320 ts_tablet_manager.cc:1916] T 704bab1b8eb34766a8f328a0acbca7a0 P d9570a47043b4891b2392af17f405e3d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260202 02:53:18.994590 4099 tablet.cc:2378] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260202 02:53:18.994172 4167 tablet_service.cc:1505] Processing CreateTablet for tablet 76c092d3e26c4906964e0f852ac24448 (DEFAULT_TABLE table=default.table_4e64388f537d44aaa6d541b90e924c65 [id=4f41a1a5a6844fbfb43b6798b2c6f67d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:18.995787 4167 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76c092d3e26c4906964e0f852ac24448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:18.996289 4319 ts_tablet_manager.cc:1916] T 704bab1b8eb34766a8f328a0acbca7a0 P 68f130042e0c4b8292b138ff8a322517: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:53:18.996764 4301 tablet_bootstrap.cc:492] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0: Bootstrap starting.
I20260202 02:53:18.999038 4301 tablet_bootstrap.cc:654] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:19.002532 4302 tablet_bootstrap.cc:492] T 76c092d3e26c4906964e0f852ac24448 P 68f130042e0c4b8292b138ff8a322517: Bootstrap starting.
I20260202 02:53:19.005627 3713 catalog_manager.cc:6724] ReplaceTablet: tablet 704bab1b8eb34766a8f328a0acbca7a0 deleted and replaced by tablet 76c092d3e26c4906964e0f852ac24448
I20260202 02:53:19.008616 3713 master_service.cc:954] ReplaceTablet: received request to replace tablet 33bfead3692c4836baf5b9816464156f from {username='slave'} at 127.0.0.1:52366
I20260202 02:53:19.010267 4321 ts_tablet_manager.cc:1916] T 704bab1b8eb34766a8f328a0acbca7a0 P a34c77c818054f87bc19600949531ad0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:53:19.016076 4302 tablet_bootstrap.cc:654] T 76c092d3e26c4906964e0f852ac24448 P 68f130042e0c4b8292b138ff8a322517: Neither blocks nor log segments found. Creating new log.
W20260202 02:53:19.018359 3957 tablet.cc:2378] T 313ff5ec7131498aaaa0c832abea2c56 P d9570a47043b4891b2392af17f405e3d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260202 02:53:19.018872 4301 tablet_bootstrap.cc:492] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0: No bootstrap required, opened a new log
I20260202 02:53:19.019137 4301 ts_tablet_manager.cc:1403] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0: Time spent bootstrapping tablet: real 0.023s user 0.003s sys 0.006s
I20260202 02:53:19.020117 4301 raft_consensus.cc:359] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
I20260202 02:53:19.020463 4301 raft_consensus.cc:385] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:19.020586 4301 raft_consensus.cc:740] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a34c77c818054f87bc19600949531ad0, State: Initialized, Role: FOLLOWER
I20260202 02:53:19.020928 4301 consensus_queue.cc:260] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
I20260202 02:53:19.023468 4302 tablet_bootstrap.cc:492] T 76c092d3e26c4906964e0f852ac24448 P 68f130042e0c4b8292b138ff8a322517: No bootstrap required, opened a new log
I20260202 02:53:19.023684 4302 ts_tablet_manager.cc:1403] T 76c092d3e26c4906964e0f852ac24448 P 68f130042e0c4b8292b138ff8a322517: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.005s
I20260202 02:53:19.024538 4302 raft_consensus.cc:359] T 76c092d3e26c4906964e0f852ac24448 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
I20260202 02:53:19.024748 4302 raft_consensus.cc:385] T 76c092d3e26c4906964e0f852ac24448 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:19.024818 4302 raft_consensus.cc:740] T 76c092d3e26c4906964e0f852ac24448 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f130042e0c4b8292b138ff8a322517, State: Initialized, Role: FOLLOWER
I20260202 02:53:19.025146 4302 consensus_queue.cc:260] T 76c092d3e26c4906964e0f852ac24448 P 68f130042e0c4b8292b138ff8a322517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
I20260202 02:53:19.026321 4301 ts_tablet_manager.cc:1434] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260202 02:53:19.029295 4302 ts_tablet_manager.cc:1434] T 76c092d3e26c4906964e0f852ac24448 P 68f130042e0c4b8292b138ff8a322517: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260202 02:53:19.031359 4320 ts_tablet_manager.cc:1929] T 704bab1b8eb34766a8f328a0acbca7a0 P d9570a47043b4891b2392af17f405e3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:53:19.031553 4320 log.cc:1199] T 704bab1b8eb34766a8f328a0acbca7a0 P d9570a47043b4891b2392af17f405e3d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/704bab1b8eb34766a8f328a0acbca7a0
I20260202 02:53:19.032039 4320 ts_tablet_manager.cc:1950] T 704bab1b8eb34766a8f328a0acbca7a0 P d9570a47043b4891b2392af17f405e3d: Deleting consensus metadata
I20260202 02:53:19.040697 3713 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: Sending DeleteTablet for 3 replicas of tablet 33bfead3692c4836baf5b9816464156f
I20260202 02:53:19.041621 3888 tablet_service.cc:1552] Processing DeleteTablet for tablet 33bfead3692c4836baf5b9816464156f with delete_type TABLET_DATA_DELETED (replaced by tablet 906f9148074848da8456b204a587fa8a) from {username='slave'} at 127.0.0.1:39530
I20260202 02:53:19.042613 4030 tablet_service.cc:1552] Processing DeleteTablet for tablet 33bfead3692c4836baf5b9816464156f with delete_type TABLET_DATA_DELETED (replaced by tablet 906f9148074848da8456b204a587fa8a) from {username='slave'} at 127.0.0.1:38036
I20260202 02:53:19.043104 4167 tablet_service.cc:1552] Processing DeleteTablet for tablet 33bfead3692c4836baf5b9816464156f with delete_type TABLET_DATA_DELETED (replaced by tablet 906f9148074848da8456b204a587fa8a) from {username='slave'} at 127.0.0.1:37318
I20260202 02:53:19.043143 4319 ts_tablet_manager.cc:1929] T 704bab1b8eb34766a8f328a0acbca7a0 P 68f130042e0c4b8292b138ff8a322517: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:53:19.043372 4319 log.cc:1199] T 704bab1b8eb34766a8f328a0acbca7a0 P 68f130042e0c4b8292b138ff8a322517: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/704bab1b8eb34766a8f328a0acbca7a0
I20260202 02:53:19.043918 4319 ts_tablet_manager.cc:1950] T 704bab1b8eb34766a8f328a0acbca7a0 P 68f130042e0c4b8292b138ff8a322517: Deleting consensus metadata
I20260202 02:53:19.047565 4303 tablet_bootstrap.cc:492] T 76c092d3e26c4906964e0f852ac24448 P d9570a47043b4891b2392af17f405e3d: No bootstrap required, opened a new log
I20260202 02:53:19.047798 4303 ts_tablet_manager.cc:1403] T 76c092d3e26c4906964e0f852ac24448 P d9570a47043b4891b2392af17f405e3d: Time spent bootstrapping tablet: real 0.056s user 0.019s sys 0.016s
I20260202 02:53:19.048705 4303 raft_consensus.cc:359] T 76c092d3e26c4906964e0f852ac24448 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
I20260202 02:53:19.048933 4303 raft_consensus.cc:385] T 76c092d3e26c4906964e0f852ac24448 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:19.049010 4303 raft_consensus.cc:740] T 76c092d3e26c4906964e0f852ac24448 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9570a47043b4891b2392af17f405e3d, State: Initialized, Role: FOLLOWER
I20260202 02:53:19.049283 4303 consensus_queue.cc:260] T 76c092d3e26c4906964e0f852ac24448 P d9570a47043b4891b2392af17f405e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
I20260202 02:53:19.053701 4303 ts_tablet_manager.cc:1434] T 76c092d3e26c4906964e0f852ac24448 P d9570a47043b4891b2392af17f405e3d: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260202 02:53:19.056072 4321 ts_tablet_manager.cc:1929] T 704bab1b8eb34766a8f328a0acbca7a0 P a34c77c818054f87bc19600949531ad0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:53:19.056198 3888 tablet_service.cc:1505] Processing CreateTablet for tablet 906f9148074848da8456b204a587fa8a (DEFAULT_TABLE table=default.table_8d32aa2872ab405fa4f2e7b1dac33f56 [id=2ab6a7a17803446898f275ae3faa032c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:19.056308 4321 log.cc:1199] T 704bab1b8eb34766a8f328a0acbca7a0 P a34c77c818054f87bc19600949531ad0: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/704bab1b8eb34766a8f328a0acbca7a0
I20260202 02:53:19.056715 3888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 906f9148074848da8456b204a587fa8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:19.056888 4321 ts_tablet_manager.cc:1950] T 704bab1b8eb34766a8f328a0acbca7a0 P a34c77c818054f87bc19600949531ad0: Deleting consensus metadata
I20260202 02:53:19.058929 3700 catalog_manager.cc:4985] TS a34c77c818054f87bc19600949531ad0 (127.0.104.130:38525): tablet 704bab1b8eb34766a8f328a0acbca7a0 (table default.table_4e64388f537d44aaa6d541b90e924c65 [id=4f41a1a5a6844fbfb43b6798b2c6f67d]) successfully deleted
I20260202 02:53:19.060449 3713 catalog_manager.cc:6724] ReplaceTablet: tablet 33bfead3692c4836baf5b9816464156f deleted and replaced by tablet 906f9148074848da8456b204a587fa8a
I20260202 02:53:19.060663 4319 tablet_replica.cc:333] T 33bfead3692c4836baf5b9816464156f P 68f130042e0c4b8292b138ff8a322517: stopping tablet replica
I20260202 02:53:19.060876 4319 raft_consensus.cc:2243] T 33bfead3692c4836baf5b9816464156f P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:53:19.061062 4319 raft_consensus.cc:2272] T 33bfead3692c4836baf5b9816464156f P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:53:19.061753 4321 tablet_replica.cc:333] T 33bfead3692c4836baf5b9816464156f P a34c77c818054f87bc19600949531ad0: stopping tablet replica
I20260202 02:53:19.062031 4321 raft_consensus.cc:2243] T 33bfead3692c4836baf5b9816464156f P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:53:19.062098 3699 catalog_manager.cc:4985] TS 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): tablet 704bab1b8eb34766a8f328a0acbca7a0 (table default.table_4e64388f537d44aaa6d541b90e924c65 [id=4f41a1a5a6844fbfb43b6798b2c6f67d]) successfully deleted
I20260202 02:53:19.062274 4321 raft_consensus.cc:2272] T 33bfead3692c4836baf5b9816464156f P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:53:19.061648 4030 tablet_service.cc:1505] Processing CreateTablet for tablet 906f9148074848da8456b204a587fa8a (DEFAULT_TABLE table=default.table_8d32aa2872ab405fa4f2e7b1dac33f56 [id=2ab6a7a17803446898f275ae3faa032c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:19.062489 4319 ts_tablet_manager.cc:1916] T 33bfead3692c4836baf5b9816464156f P 68f130042e0c4b8292b138ff8a322517: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:53:19.062739 4030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 906f9148074848da8456b204a587fa8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:19.063104 4321 ts_tablet_manager.cc:1916] T 33bfead3692c4836baf5b9816464156f P a34c77c818054f87bc19600949531ad0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:53:19.064100 3713 master_service.cc:954] ReplaceTablet: received request to replace tablet 704bab1b8eb34766a8f328a0acbca7a0 from {username='slave'} at 127.0.0.1:52366
I20260202 02:53:19.065053 3713 master_service.cc:954] ReplaceTablet: received request to replace tablet 704bab1b8eb34766a8f328a0acbca7a0 from {username='slave'} at 127.0.0.1:52366
I20260202 02:53:19.065750 3713 master_service.cc:954] ReplaceTablet: received request to replace tablet 704bab1b8eb34766a8f328a0acbca7a0 from {username='slave'} at 127.0.0.1:52366
I20260202 02:53:19.066524 3713 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52366:
name: "default.Table_fb2a5c4107db446aba29a7f07a46c1a8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:19.069487 4319 ts_tablet_manager.cc:1929] T 33bfead3692c4836baf5b9816464156f P 68f130042e0c4b8292b138ff8a322517: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:53:19.069629 4321 ts_tablet_manager.cc:1929] T 33bfead3692c4836baf5b9816464156f P a34c77c818054f87bc19600949531ad0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:53:19.069679 4319 log.cc:1199] T 33bfead3692c4836baf5b9816464156f P 68f130042e0c4b8292b138ff8a322517: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/33bfead3692c4836baf5b9816464156f
I20260202 02:53:19.069769 4321 log.cc:1199] T 33bfead3692c4836baf5b9816464156f P a34c77c818054f87bc19600949531ad0: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/33bfead3692c4836baf5b9816464156f
I20260202 02:53:19.070011 4319 ts_tablet_manager.cc:1950] T 33bfead3692c4836baf5b9816464156f P 68f130042e0c4b8292b138ff8a322517: Deleting consensus metadata
I20260202 02:53:19.070081 4321 ts_tablet_manager.cc:1950] T 33bfead3692c4836baf5b9816464156f P a34c77c818054f87bc19600949531ad0: Deleting consensus metadata
I20260202 02:53:19.071077 3700 catalog_manager.cc:4985] TS a34c77c818054f87bc19600949531ad0 (127.0.104.130:38525): tablet 33bfead3692c4836baf5b9816464156f (table default.table_8d32aa2872ab405fa4f2e7b1dac33f56 [id=2ab6a7a17803446898f275ae3faa032c]) successfully deleted
I20260202 02:53:19.071283 3699 catalog_manager.cc:4985] TS 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): tablet 33bfead3692c4836baf5b9816464156f (table default.table_8d32aa2872ab405fa4f2e7b1dac33f56 [id=2ab6a7a17803446898f275ae3faa032c]) successfully deleted
I20260202 02:53:19.071538 4167 tablet_service.cc:1505] Processing CreateTablet for tablet 906f9148074848da8456b204a587fa8a (DEFAULT_TABLE table=default.table_8d32aa2872ab405fa4f2e7b1dac33f56 [id=2ab6a7a17803446898f275ae3faa032c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:19.071955 4167 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 906f9148074848da8456b204a587fa8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:19.072467 3700 catalog_manager.cc:4985] TS d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): tablet 704bab1b8eb34766a8f328a0acbca7a0 (table default.table_4e64388f537d44aaa6d541b90e924c65 [id=4f41a1a5a6844fbfb43b6798b2c6f67d]) successfully deleted
I20260202 02:53:19.072604 4320 tablet_replica.cc:333] T 33bfead3692c4836baf5b9816464156f P d9570a47043b4891b2392af17f405e3d: stopping tablet replica
I20260202 02:53:19.072901 4301 tablet_bootstrap.cc:492] T 906f9148074848da8456b204a587fa8a P a34c77c818054f87bc19600949531ad0: Bootstrap starting.
I20260202 02:53:19.072907 4320 raft_consensus.cc:2243] T 33bfead3692c4836baf5b9816464156f P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260202 02:53:19.073099 4320 raft_consensus.cc:2272] T 33bfead3692c4836baf5b9816464156f P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260202 02:53:19.073946 4320 ts_tablet_manager.cc:1916] T 33bfead3692c4836baf5b9816464156f P d9570a47043b4891b2392af17f405e3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:53:19.074643 4301 tablet_bootstrap.cc:654] T 906f9148074848da8456b204a587fa8a P a34c77c818054f87bc19600949531ad0: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:19.078388 4301 tablet_bootstrap.cc:492] T 906f9148074848da8456b204a587fa8a P a34c77c818054f87bc19600949531ad0: No bootstrap required, opened a new log
I20260202 02:53:19.078579 4301 ts_tablet_manager.cc:1403] T 906f9148074848da8456b204a587fa8a P a34c77c818054f87bc19600949531ad0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260202 02:53:19.079586 4303 tablet_bootstrap.cc:492] T 906f9148074848da8456b204a587fa8a P d9570a47043b4891b2392af17f405e3d: Bootstrap starting.
I20260202 02:53:19.079435 4301 raft_consensus.cc:359] T 906f9148074848da8456b204a587fa8a P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
I20260202 02:53:19.079598 4302 tablet_bootstrap.cc:492] T 906f9148074848da8456b204a587fa8a P 68f130042e0c4b8292b138ff8a322517: Bootstrap starting.
I20260202 02:53:19.079685 4301 raft_consensus.cc:385] T 906f9148074848da8456b204a587fa8a P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:19.079770 4301 raft_consensus.cc:740] T 906f9148074848da8456b204a587fa8a P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a34c77c818054f87bc19600949531ad0, State: Initialized, Role: FOLLOWER
I20260202 02:53:19.080063 4301 consensus_queue.cc:260] T 906f9148074848da8456b204a587fa8a P a34c77c818054f87bc19600949531ad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
I20260202 02:53:19.081434 4302 tablet_bootstrap.cc:654] T 906f9148074848da8456b204a587fa8a P 68f130042e0c4b8292b138ff8a322517: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:19.081516 4303 tablet_bootstrap.cc:654] T 906f9148074848da8456b204a587fa8a P d9570a47043b4891b2392af17f405e3d: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:19.082762 4320 ts_tablet_manager.cc:1929] T 33bfead3692c4836baf5b9816464156f P d9570a47043b4891b2392af17f405e3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:53:19.082940 4320 log.cc:1199] T 33bfead3692c4836baf5b9816464156f P d9570a47043b4891b2392af17f405e3d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/33bfead3692c4836baf5b9816464156f
I20260202 02:53:19.083297 4320 ts_tablet_manager.cc:1950] T 33bfead3692c4836baf5b9816464156f P d9570a47043b4891b2392af17f405e3d: Deleting consensus metadata
I20260202 02:53:19.084519 4301 ts_tablet_manager.cc:1434] T 906f9148074848da8456b204a587fa8a P a34c77c818054f87bc19600949531ad0: Time spent starting tablet: real 0.006s user 0.002s sys 0.001s
I20260202 02:53:19.084568 4308 raft_consensus.cc:493] T b5a121d55d9d498ba76bb4dcb24d0e20 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:19.084803 4308 raft_consensus.cc:515] T b5a121d55d9d498ba76bb4dcb24d0e20 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:19.086861 4308 leader_election.cc:290] T b5a121d55d9d498ba76bb4dcb24d0e20 P 68f130042e0c4b8292b138ff8a322517 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425), a34c77c818054f87bc19600949531ad0 (127.0.104.130:38525)
I20260202 02:53:19.087379 3712 catalog_manager.cc:2174] created HMS catalog entry for table [id=3852a850379342c69688c71b95873308]
W20260202 02:53:19.090523 3707 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_2faba70635fa4e55ae56458e06dba641 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:53:19.091814 3714 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_a728b4dd173b4f0c8c6b9cce0bcda94d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260202 02:53:19.093088 3711 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_2d03ff86a95f4175b2225ecc0d899058 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:53:19.093837 4307 raft_consensus.cc:493] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260202 02:53:19.094277 3709 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_dc0bdab2115f4027a86e680d2995f726 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:53:19.094141 4307 raft_consensus.cc:515] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
W20260202 02:53:19.095379 3706 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_de1272fe656e46328aa237aec594394a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260202 02:53:19.096663 4307 leader_election.cc:290] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425), 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661)
I20260202 02:53:19.097674 3700 catalog_manager.cc:4985] TS d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): tablet 33bfead3692c4836baf5b9816464156f (table default.table_8d32aa2872ab405fa4f2e7b1dac33f56 [id=2ab6a7a17803446898f275ae3faa032c]) successfully deleted
F20260202 02:53:19.106119 3788 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 6 received events: NotificationEvent(eventId=1, eventTime=1770000798, eventType=CREATE_TABLE, dbName=default, tableName=table_4e64388f537d44aaa6d541b90e924c65, message={"server":"","servicePrincipal":"","db":"default","table":"table_4e64388f537d44aaa6d541b90e924c65","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_4e64388f537d44aaa6d541b90e924c65\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770000798},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/warehouse/table_4e64388f537d44aaa6d541b90e924c65\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_4e64388f537d44aaa6d541b90e924c65\",\"kudu.table_id\":\"4f41a1a5a6844fbfb43b6798b2c6f67d\",\"transient_lastDdlTime\":\"1770000798\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"0d95b889b53247a0813a4fda5550ae30\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770000798,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1770000798, eventType=CREATE_TABLE, dbName=default, tableName=table_a728b4dd173b4f0c8c6b9cce0bcda94d, message={"server":"","servicePrincipal":"","db":"default","table":"table_a728b4dd173b4f0c8c6b9cce0bcda94d","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_a728b4dd173b4f0c8c6b9cce0bcda94d\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770000798},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/warehouse/table_a728b4dd173b4f0c8c6b9cce0bcda94d\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_a728b4dd173b4f0c8c6b9cce0bcda94d\",\"kudu.table_id\":\"a54f76c7d1ff4769b2d84a5fe325d81c\",\"transient_lastDdlTime\":\"1770000798\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"0d95b889b53247a0813a4fda5550ae30\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770000798,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7f57e382adcd google::LogMessage::Fail() at ??:0
I20260202 02:53:19.111439 4187 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "76c092d3e26c4906964e0f852ac24448" candidate_uuid: "a34c77c818054f87bc19600949531ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f130042e0c4b8292b138ff8a322517" is_pre_election: true
I20260202 02:53:19.112006 4187 raft_consensus.cc:2468] T 76c092d3e26c4906964e0f852ac24448 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c77c818054f87bc19600949531ad0 in term 0.
@ 0x7f57e382eb93 google::LogMessage::SendToLog() at ??:0
I20260202 02:53:19.113116 3984 leader_election.cc:304] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68f130042e0c4b8292b138ff8a322517, a34c77c818054f87bc19600949531ad0; no voters:
@ 0x7f57e382a7cc google::LogMessage::Flush() at ??:0
I20260202 02:53:19.113721 4307 raft_consensus.cc:2804] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:53:19.113911 4307 raft_consensus.cc:493] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:53:19.114022 4307 raft_consensus.cc:3060] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Advancing to term 1
@ 0x7f57e382aa11 google::LogMessage::~LogMessage() at ??:0
I20260202 02:53:19.119699 3712 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:52366:
table { table_name: "default.Table_470455fd011c452b9cceafbc413b1cfe" } new_table_name: "default.Table_d4d44071ee424b9fa646dd6fe1c3ecd8" modify_external_catalogs: true
I20260202 02:53:19.126665 4307 raft_consensus.cc:515] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
I20260202 02:53:19.128265 4187 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "76c092d3e26c4906964e0f852ac24448" candidate_uuid: "a34c77c818054f87bc19600949531ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f130042e0c4b8292b138ff8a322517"
I20260202 02:53:19.128557 4187 raft_consensus.cc:3060] T 76c092d3e26c4906964e0f852ac24448 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:19.132736 4187 raft_consensus.cc:2468] T 76c092d3e26c4906964e0f852ac24448 P 68f130042e0c4b8292b138ff8a322517 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c77c818054f87bc19600949531ad0 in term 1.
I20260202 02:53:19.133330 3984 leader_election.cc:304] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68f130042e0c4b8292b138ff8a322517, a34c77c818054f87bc19600949531ad0; no voters:
I20260202 02:53:19.135370 4166 tablet_service.cc:1505] Processing CreateTablet for tablet 8d1373239ae14791a720a0a7ca86d09a (DEFAULT_TABLE table=default.table_2c330ab9e5984ba5980e7264050e7a41 [id=3852a850379342c69688c71b95873308]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:19.135633 4307 leader_election.cc:290] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 [CANDIDATE]: Term 1 election: Requested vote from peers d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425), 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661)
I20260202 02:53:19.135845 4166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d1373239ae14791a720a0a7ca86d09a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:19.135926 4307 raft_consensus.cc:2804] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:53:19.136093 4307 raft_consensus.cc:697] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 [term 1 LEADER]: Becoming Leader. State: Replica: a34c77c818054f87bc19600949531ad0, State: Running, Role: LEADER
I20260202 02:53:19.136595 4307 consensus_queue.cc:237] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
I20260202 02:53:19.137563 4030 tablet_service.cc:1505] Processing CreateTablet for tablet 8d1373239ae14791a720a0a7ca86d09a (DEFAULT_TABLE table=default.table_2c330ab9e5984ba5980e7264050e7a41 [id=3852a850379342c69688c71b95873308]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:19.138077 4030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d1373239ae14791a720a0a7ca86d09a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:19.141042 3888 tablet_service.cc:1505] Processing CreateTablet for tablet 8d1373239ae14791a720a0a7ca86d09a (DEFAULT_TABLE table=default.table_2c330ab9e5984ba5980e7264050e7a41 [id=3852a850379342c69688c71b95873308]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:19.141539 3888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d1373239ae14791a720a0a7ca86d09a. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f57fa2217f4 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260202 02:53:19.153196 4301 tablet_bootstrap.cc:492] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0: Bootstrap starting.
I20260202 02:53:19.155208 4301 tablet_bootstrap.cc:654] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:19.157708 4309 raft_consensus.cc:493] T 76c092d3e26c4906964e0f852ac24448 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:19.157984 4309 raft_consensus.cc:515] T 76c092d3e26c4906964e0f852ac24448 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
I20260202 02:53:19.160820 4302 tablet_bootstrap.cc:492] T 906f9148074848da8456b204a587fa8a P 68f130042e0c4b8292b138ff8a322517: No bootstrap required, opened a new log
I20260202 02:53:19.160990 4302 ts_tablet_manager.cc:1403] T 906f9148074848da8456b204a587fa8a P 68f130042e0c4b8292b138ff8a322517: Time spent bootstrapping tablet: real 0.082s user 0.005s sys 0.005s
I20260202 02:53:19.160219 4309 leader_election.cc:290] T 76c092d3e26c4906964e0f852ac24448 P d9570a47043b4891b2392af17f405e3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a34c77c818054f87bc19600949531ad0 (127.0.104.130:38525), 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661)
I20260202 02:53:19.161859 4302 raft_consensus.cc:359] T 906f9148074848da8456b204a587fa8a P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
I20260202 02:53:19.162070 4302 raft_consensus.cc:385] T 906f9148074848da8456b204a587fa8a P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:19.162150 4302 raft_consensus.cc:740] T 906f9148074848da8456b204a587fa8a P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f130042e0c4b8292b138ff8a322517, State: Initialized, Role: FOLLOWER
I20260202 02:53:19.162384 4302 consensus_queue.cc:260] T 906f9148074848da8456b204a587fa8a P 68f130042e0c4b8292b138ff8a322517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
I20260202 02:53:19.163830 4302 ts_tablet_manager.cc:1434] T 906f9148074848da8456b204a587fa8a P 68f130042e0c4b8292b138ff8a322517: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260202 02:53:19.164144 4302 tablet_bootstrap.cc:492] T 8d1373239ae14791a720a0a7ca86d09a P 68f130042e0c4b8292b138ff8a322517: Bootstrap starting.
I20260202 02:53:19.166007 4302 tablet_bootstrap.cc:654] T 8d1373239ae14791a720a0a7ca86d09a P 68f130042e0c4b8292b138ff8a322517: Neither blocks nor log segments found. Creating new log.
@ 0x7f57fa2200a1 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260202 02:53:19.172758 4303 tablet_bootstrap.cc:492] T 906f9148074848da8456b204a587fa8a P d9570a47043b4891b2392af17f405e3d: No bootstrap required, opened a new log
I20260202 02:53:19.172945 4303 ts_tablet_manager.cc:1403] T 906f9148074848da8456b204a587fa8a P d9570a47043b4891b2392af17f405e3d: Time spent bootstrapping tablet: real 0.093s user 0.023s sys 0.023s
I20260202 02:53:19.173833 4303 raft_consensus.cc:359] T 906f9148074848da8456b204a587fa8a P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
I20260202 02:53:19.174059 4303 raft_consensus.cc:385] T 906f9148074848da8456b204a587fa8a P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:19.174142 4303 raft_consensus.cc:740] T 906f9148074848da8456b204a587fa8a P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9570a47043b4891b2392af17f405e3d, State: Initialized, Role: FOLLOWER
@ 0x7f57fa227d1c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260202 02:53:19.178858 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskYXxklX/build/asan/bin/kudu
/tmp/dist-test-taskYXxklX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.189:40361
--webserver_interface=127.0.104.189
--webserver_port=35647
--builtin_ntp_servers=127.0.104.148:34531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525
--hive_metastore_uris=thrift://127.0.0.1:46545
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260202 02:53:19.179262 4303 consensus_queue.cc:260] T 906f9148074848da8456b204a587fa8a P d9570a47043b4891b2392af17f405e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
I20260202 02:53:19.182673 3909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "76c092d3e26c4906964e0f852ac24448" candidate_uuid: "a34c77c818054f87bc19600949531ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9570a47043b4891b2392af17f405e3d" is_pre_election: true
I20260202 02:53:19.183167 3909 raft_consensus.cc:2468] T 76c092d3e26c4906964e0f852ac24448 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c77c818054f87bc19600949531ad0 in term 0.
I20260202 02:53:19.183804 3909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "76c092d3e26c4906964e0f852ac24448" candidate_uuid: "a34c77c818054f87bc19600949531ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9570a47043b4891b2392af17f405e3d"
I20260202 02:53:19.183986 3909 raft_consensus.cc:3060] T 76c092d3e26c4906964e0f852ac24448 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:19.186962 3909 raft_consensus.cc:2468] T 76c092d3e26c4906964e0f852ac24448 P d9570a47043b4891b2392af17f405e3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c77c818054f87bc19600949531ad0 in term 1.
I20260202 02:53:19.188732 4303 ts_tablet_manager.cc:1434] T 906f9148074848da8456b204a587fa8a P d9570a47043b4891b2392af17f405e3d: Time spent starting tablet: real 0.016s user 0.003s sys 0.000s
I20260202 02:53:19.189145 4303 tablet_bootstrap.cc:492] T 8d1373239ae14791a720a0a7ca86d09a P d9570a47043b4891b2392af17f405e3d: Bootstrap starting.
I20260202 02:53:19.189380 4302 tablet_bootstrap.cc:492] T 8d1373239ae14791a720a0a7ca86d09a P 68f130042e0c4b8292b138ff8a322517: No bootstrap required, opened a new log
I20260202 02:53:19.189594 4302 ts_tablet_manager.cc:1403] T 8d1373239ae14791a720a0a7ca86d09a P 68f130042e0c4b8292b138ff8a322517: Time spent bootstrapping tablet: real 0.025s user 0.003s sys 0.000s
I20260202 02:53:19.190475 4302 raft_consensus.cc:359] T 8d1373239ae14791a720a0a7ca86d09a P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:19.190711 4302 raft_consensus.cc:385] T 8d1373239ae14791a720a0a7ca86d09a P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:19.190799 4302 raft_consensus.cc:740] T 8d1373239ae14791a720a0a7ca86d09a P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f130042e0c4b8292b138ff8a322517, State: Initialized, Role: FOLLOWER
I20260202 02:53:19.191097 4302 consensus_queue.cc:260] T 8d1373239ae14791a720a0a7ca86d09a P 68f130042e0c4b8292b138ff8a322517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:19.192951 4051 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b5a121d55d9d498ba76bb4dcb24d0e20" candidate_uuid: "68f130042e0c4b8292b138ff8a322517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a34c77c818054f87bc19600949531ad0" is_pre_election: true
I20260202 02:53:19.193150 4302 ts_tablet_manager.cc:1434] T 8d1373239ae14791a720a0a7ca86d09a P 68f130042e0c4b8292b138ff8a322517: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260202 02:53:19.193418 4051 raft_consensus.cc:2468] T b5a121d55d9d498ba76bb4dcb24d0e20 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68f130042e0c4b8292b138ff8a322517 in term 0.
I20260202 02:53:19.193763 4303 tablet_bootstrap.cc:654] T 8d1373239ae14791a720a0a7ca86d09a P d9570a47043b4891b2392af17f405e3d: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:19.196197 4301 tablet_bootstrap.cc:492] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0: No bootstrap required, opened a new log
I20260202 02:53:19.196439 4301 ts_tablet_manager.cc:1403] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0: Time spent bootstrapping tablet: real 0.043s user 0.001s sys 0.010s
I20260202 02:53:19.198104 4301 raft_consensus.cc:359] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:19.198313 4301 raft_consensus.cc:385] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:19.198395 4301 raft_consensus.cc:740] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a34c77c818054f87bc19600949531ad0, State: Initialized, Role: FOLLOWER
I20260202 02:53:19.198679 4301 consensus_queue.cc:260] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:19.198956 4121 leader_election.cc:304] T b5a121d55d9d498ba76bb4dcb24d0e20 P 68f130042e0c4b8292b138ff8a322517 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68f130042e0c4b8292b138ff8a322517, a34c77c818054f87bc19600949531ad0; no voters:
I20260202 02:53:19.199237 4301 ts_tablet_manager.cc:1434] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260202 02:53:19.202239 4303 tablet_bootstrap.cc:492] T 8d1373239ae14791a720a0a7ca86d09a P d9570a47043b4891b2392af17f405e3d: No bootstrap required, opened a new log
I20260202 02:53:19.202461 4303 ts_tablet_manager.cc:1403] T 8d1373239ae14791a720a0a7ca86d09a P d9570a47043b4891b2392af17f405e3d: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20260202 02:53:19.203410 4303 raft_consensus.cc:359] T 8d1373239ae14791a720a0a7ca86d09a P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:19.203637 4303 raft_consensus.cc:385] T 8d1373239ae14791a720a0a7ca86d09a P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:19.203732 4303 raft_consensus.cc:740] T 8d1373239ae14791a720a0a7ca86d09a P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9570a47043b4891b2392af17f405e3d, State: Initialized, Role: FOLLOWER
I20260202 02:53:19.203994 4303 consensus_queue.cc:260] T 8d1373239ae14791a720a0a7ca86d09a P d9570a47043b4891b2392af17f405e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:19.204645 3909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b5a121d55d9d498ba76bb4dcb24d0e20" candidate_uuid: "68f130042e0c4b8292b138ff8a322517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9570a47043b4891b2392af17f405e3d" is_pre_election: true
I20260202 02:53:19.204881 3909 raft_consensus.cc:2468] T b5a121d55d9d498ba76bb4dcb24d0e20 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68f130042e0c4b8292b138ff8a322517 in term 0.
I20260202 02:53:19.206501 4303 ts_tablet_manager.cc:1434] T 8d1373239ae14791a720a0a7ca86d09a P d9570a47043b4891b2392af17f405e3d: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
@ 0x7f57fa227b99 std::_Function_handler<>::_M_invoke() at ??:0
@ 0xa8970d std::function<>::operator()() at ??:?
I20260202 02:53:19.211432 4308 raft_consensus.cc:2804] T b5a121d55d9d498ba76bb4dcb24d0e20 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:53:19.211598 4308 raft_consensus.cc:493] T b5a121d55d9d498ba76bb4dcb24d0e20 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:53:19.211696 4308 raft_consensus.cc:3060] T b5a121d55d9d498ba76bb4dcb24d0e20 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:19.211796 4051 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "76c092d3e26c4906964e0f852ac24448" candidate_uuid: "d9570a47043b4891b2392af17f405e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a34c77c818054f87bc19600949531ad0" is_pre_election: true
@ 0x7f57e659ff99 kudu::Thread::SuperviseThread() at ??:0
I20260202 02:53:19.214702 4308 raft_consensus.cc:515] T b5a121d55d9d498ba76bb4dcb24d0e20 P 68f130042e0c4b8292b138ff8a322517 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
@ 0x7f57e9bac6db start_thread at ??:0
I20260202 02:53:19.215466 4308 leader_election.cc:290] T b5a121d55d9d498ba76bb4dcb24d0e20 P 68f130042e0c4b8292b138ff8a322517 [CANDIDATE]: Term 1 election: Requested vote from peers d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425), a34c77c818054f87bc19600949531ad0 (127.0.104.130:38525)
@ 0x7f57e26db71f clone at ??:0
I20260202 02:53:19.217044 4051 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b5a121d55d9d498ba76bb4dcb24d0e20" candidate_uuid: "68f130042e0c4b8292b138ff8a322517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a34c77c818054f87bc19600949531ad0"
I20260202 02:53:19.217319 4051 raft_consensus.cc:3060] T b5a121d55d9d498ba76bb4dcb24d0e20 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:19.217445 3909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b5a121d55d9d498ba76bb4dcb24d0e20" candidate_uuid: "68f130042e0c4b8292b138ff8a322517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9570a47043b4891b2392af17f405e3d"
I20260202 02:53:19.217675 3909 raft_consensus.cc:3060] T b5a121d55d9d498ba76bb4dcb24d0e20 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:19.220455 3909 raft_consensus.cc:2468] T b5a121d55d9d498ba76bb4dcb24d0e20 P d9570a47043b4891b2392af17f405e3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68f130042e0c4b8292b138ff8a322517 in term 1.
I20260202 02:53:19.221089 4051 raft_consensus.cc:2468] T b5a121d55d9d498ba76bb4dcb24d0e20 P a34c77c818054f87bc19600949531ad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68f130042e0c4b8292b138ff8a322517 in term 1.
I20260202 02:53:19.221271 4187 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "76c092d3e26c4906964e0f852ac24448" candidate_uuid: "d9570a47043b4891b2392af17f405e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f130042e0c4b8292b138ff8a322517" is_pre_election: true
I20260202 02:53:19.221649 4187 raft_consensus.cc:2393] T 76c092d3e26c4906964e0f852ac24448 P 68f130042e0c4b8292b138ff8a322517 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9570a47043b4891b2392af17f405e3d in current term 1: Already voted for candidate a34c77c818054f87bc19600949531ad0 in this term.
I20260202 02:53:19.221808 4121 leader_election.cc:304] T b5a121d55d9d498ba76bb4dcb24d0e20 P 68f130042e0c4b8292b138ff8a322517 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68f130042e0c4b8292b138ff8a322517, a34c77c818054f87bc19600949531ad0; no voters:
I20260202 02:53:19.222095 4308 raft_consensus.cc:2804] T b5a121d55d9d498ba76bb4dcb24d0e20 P 68f130042e0c4b8292b138ff8a322517 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:53:19.222556 4308 raft_consensus.cc:697] T b5a121d55d9d498ba76bb4dcb24d0e20 P 68f130042e0c4b8292b138ff8a322517 [term 1 LEADER]: Becoming Leader. State: Replica: 68f130042e0c4b8292b138ff8a322517, State: Running, Role: LEADER
I20260202 02:53:19.223016 4308 consensus_queue.cc:237] T b5a121d55d9d498ba76bb4dcb24d0e20 P 68f130042e0c4b8292b138ff8a322517 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:19.224862 3840 leader_election.cc:304] T 76c092d3e26c4906964e0f852ac24448 P d9570a47043b4891b2392af17f405e3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9570a47043b4891b2392af17f405e3d; no voters: 68f130042e0c4b8292b138ff8a322517, a34c77c818054f87bc19600949531ad0
I20260202 02:53:19.225703 4309 raft_consensus.cc:2749] T 76c092d3e26c4906964e0f852ac24448 P d9570a47043b4891b2392af17f405e3d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260202 02:53:19.285086 4309 raft_consensus.cc:493] T 906f9148074848da8456b204a587fa8a P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:19.285332 4309 raft_consensus.cc:515] T 906f9148074848da8456b204a587fa8a P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
I20260202 02:53:19.286085 4309 leader_election.cc:290] T 906f9148074848da8456b204a587fa8a P d9570a47043b4891b2392af17f405e3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a34c77c818054f87bc19600949531ad0 (127.0.104.130:38525), 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661)
I20260202 02:53:19.286883 4187 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "906f9148074848da8456b204a587fa8a" candidate_uuid: "d9570a47043b4891b2392af17f405e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f130042e0c4b8292b138ff8a322517" is_pre_election: true
I20260202 02:53:19.286902 4051 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "906f9148074848da8456b204a587fa8a" candidate_uuid: "d9570a47043b4891b2392af17f405e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a34c77c818054f87bc19600949531ad0" is_pre_election: true
I20260202 02:53:19.287117 4051 raft_consensus.cc:2468] T 906f9148074848da8456b204a587fa8a P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9570a47043b4891b2392af17f405e3d in term 0.
I20260202 02:53:19.287134 4187 raft_consensus.cc:2468] T 906f9148074848da8456b204a587fa8a P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9570a47043b4891b2392af17f405e3d in term 0.
I20260202 02:53:19.287652 3841 leader_election.cc:304] T 906f9148074848da8456b204a587fa8a P d9570a47043b4891b2392af17f405e3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 68f130042e0c4b8292b138ff8a322517, a34c77c818054f87bc19600949531ad0, d9570a47043b4891b2392af17f405e3d; no voters:
I20260202 02:53:19.288021 4309 raft_consensus.cc:2804] T 906f9148074848da8456b204a587fa8a P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:53:19.288142 4309 raft_consensus.cc:493] T 906f9148074848da8456b204a587fa8a P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:53:19.288232 4309 raft_consensus.cc:3060] T 906f9148074848da8456b204a587fa8a P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:19.290334 4309 raft_consensus.cc:515] T 906f9148074848da8456b204a587fa8a P d9570a47043b4891b2392af17f405e3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
I20260202 02:53:19.291527 4051 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "906f9148074848da8456b204a587fa8a" candidate_uuid: "d9570a47043b4891b2392af17f405e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a34c77c818054f87bc19600949531ad0"
I20260202 02:53:19.291507 4187 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "906f9148074848da8456b204a587fa8a" candidate_uuid: "d9570a47043b4891b2392af17f405e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f130042e0c4b8292b138ff8a322517"
I20260202 02:53:19.291730 4051 raft_consensus.cc:3060] T 906f9148074848da8456b204a587fa8a P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:19.291749 4187 raft_consensus.cc:3060] T 906f9148074848da8456b204a587fa8a P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:19.293697 4309 leader_election.cc:290] T 906f9148074848da8456b204a587fa8a P d9570a47043b4891b2392af17f405e3d [CANDIDATE]: Term 1 election: Requested vote from peers a34c77c818054f87bc19600949531ad0 (127.0.104.130:38525), 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661)
I20260202 02:53:19.296944 4051 raft_consensus.cc:2468] T 906f9148074848da8456b204a587fa8a P a34c77c818054f87bc19600949531ad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9570a47043b4891b2392af17f405e3d in term 1.
I20260202 02:53:19.297472 4187 raft_consensus.cc:2468] T 906f9148074848da8456b204a587fa8a P 68f130042e0c4b8292b138ff8a322517 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9570a47043b4891b2392af17f405e3d in term 1.
I20260202 02:53:19.297616 3841 leader_election.cc:304] T 906f9148074848da8456b204a587fa8a P d9570a47043b4891b2392af17f405e3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c77c818054f87bc19600949531ad0, d9570a47043b4891b2392af17f405e3d; no voters:
I20260202 02:53:19.298158 4309 raft_consensus.cc:2804] T 906f9148074848da8456b204a587fa8a P d9570a47043b4891b2392af17f405e3d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:53:19.298668 4309 raft_consensus.cc:697] T 906f9148074848da8456b204a587fa8a P d9570a47043b4891b2392af17f405e3d [term 1 LEADER]: Becoming Leader. State: Replica: d9570a47043b4891b2392af17f405e3d, State: Running, Role: LEADER
I20260202 02:53:19.299504 4309 consensus_queue.cc:237] T 906f9148074848da8456b204a587fa8a P d9570a47043b4891b2392af17f405e3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
I20260202 02:53:19.368880 4307 raft_consensus.cc:493] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:19.369115 4307 raft_consensus.cc:515] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:19.369813 4307 leader_election.cc:290] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661), d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425)
I20260202 02:53:19.370437 4187 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8d1373239ae14791a720a0a7ca86d09a" candidate_uuid: "a34c77c818054f87bc19600949531ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f130042e0c4b8292b138ff8a322517" is_pre_election: true
I20260202 02:53:19.370555 3909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8d1373239ae14791a720a0a7ca86d09a" candidate_uuid: "a34c77c818054f87bc19600949531ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9570a47043b4891b2392af17f405e3d" is_pre_election: true
I20260202 02:53:19.370668 4187 raft_consensus.cc:2468] T 8d1373239ae14791a720a0a7ca86d09a P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c77c818054f87bc19600949531ad0 in term 0.
I20260202 02:53:19.370747 3909 raft_consensus.cc:2468] T 8d1373239ae14791a720a0a7ca86d09a P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c77c818054f87bc19600949531ad0 in term 0.
I20260202 02:53:19.371248 3985 leader_election.cc:304] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 68f130042e0c4b8292b138ff8a322517, a34c77c818054f87bc19600949531ad0, d9570a47043b4891b2392af17f405e3d; no voters:
I20260202 02:53:19.371608 4307 raft_consensus.cc:2804] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:53:19.371721 4307 raft_consensus.cc:493] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:53:19.371834 4307 raft_consensus.cc:3060] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:19.373762 4307 raft_consensus.cc:515] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:19.374833 4307 leader_election.cc:290] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 [CANDIDATE]: Term 1 election: Requested vote from peers 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661), d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425)
I20260202 02:53:19.374953 3909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8d1373239ae14791a720a0a7ca86d09a" candidate_uuid: "a34c77c818054f87bc19600949531ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9570a47043b4891b2392af17f405e3d"
I20260202 02:53:19.375157 3909 raft_consensus.cc:3060] T 8d1373239ae14791a720a0a7ca86d09a P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:19.375213 4187 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8d1373239ae14791a720a0a7ca86d09a" candidate_uuid: "a34c77c818054f87bc19600949531ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f130042e0c4b8292b138ff8a322517"
I20260202 02:53:19.375589 4187 raft_consensus.cc:3060] T 8d1373239ae14791a720a0a7ca86d09a P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:19.378113 3909 raft_consensus.cc:2468] T 8d1373239ae14791a720a0a7ca86d09a P d9570a47043b4891b2392af17f405e3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c77c818054f87bc19600949531ad0 in term 1.
I20260202 02:53:19.378178 4187 raft_consensus.cc:2468] T 8d1373239ae14791a720a0a7ca86d09a P 68f130042e0c4b8292b138ff8a322517 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c77c818054f87bc19600949531ad0 in term 1.
I20260202 02:53:19.378957 3985 leader_election.cc:304] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 68f130042e0c4b8292b138ff8a322517, a34c77c818054f87bc19600949531ad0, d9570a47043b4891b2392af17f405e3d; no voters:
I20260202 02:53:19.379405 4307 raft_consensus.cc:2804] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:53:19.380237 4307 raft_consensus.cc:697] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 [term 1 LEADER]: Becoming Leader. State: Replica: a34c77c818054f87bc19600949531ad0, State: Running, Role: LEADER
I20260202 02:53:19.380594 4307 consensus_queue.cc:237] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:19.487561 4307 raft_consensus.cc:493] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:19.487766 4307 raft_consensus.cc:515] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
I20260202 02:53:19.488970 4307 leader_election.cc:290] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425), 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661)
I20260202 02:53:19.489053 4187 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "313ff5ec7131498aaaa0c832abea2c56" candidate_uuid: "a34c77c818054f87bc19600949531ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f130042e0c4b8292b138ff8a322517" is_pre_election: true
I20260202 02:53:19.489306 4187 raft_consensus.cc:2468] T 313ff5ec7131498aaaa0c832abea2c56 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c77c818054f87bc19600949531ad0 in term 0.
I20260202 02:53:19.489374 3909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "313ff5ec7131498aaaa0c832abea2c56" candidate_uuid: "a34c77c818054f87bc19600949531ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9570a47043b4891b2392af17f405e3d" is_pre_election: true
I20260202 02:53:19.489595 3909 raft_consensus.cc:2468] T 313ff5ec7131498aaaa0c832abea2c56 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c77c818054f87bc19600949531ad0 in term 0.
I20260202 02:53:19.489944 3984 leader_election.cc:304] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68f130042e0c4b8292b138ff8a322517, a34c77c818054f87bc19600949531ad0; no voters:
I20260202 02:53:19.490319 4307 raft_consensus.cc:2804] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:53:19.490499 4307 raft_consensus.cc:493] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:53:19.490641 4307 raft_consensus.cc:3060] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:19.493935 4307 raft_consensus.cc:515] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
I20260202 02:53:19.495656 3909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "313ff5ec7131498aaaa0c832abea2c56" candidate_uuid: "a34c77c818054f87bc19600949531ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9570a47043b4891b2392af17f405e3d"
I20260202 02:53:19.495954 3909 raft_consensus.cc:3060] T 313ff5ec7131498aaaa0c832abea2c56 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:19.499320 3909 raft_consensus.cc:2468] T 313ff5ec7131498aaaa0c832abea2c56 P d9570a47043b4891b2392af17f405e3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c77c818054f87bc19600949531ad0 in term 1.
I20260202 02:53:19.500010 3985 leader_election.cc:304] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c77c818054f87bc19600949531ad0, d9570a47043b4891b2392af17f405e3d; no voters:
I20260202 02:53:19.500475 4332 raft_consensus.cc:2804] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:53:19.507830 4187 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "313ff5ec7131498aaaa0c832abea2c56" candidate_uuid: "a34c77c818054f87bc19600949531ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f130042e0c4b8292b138ff8a322517"
I20260202 02:53:19.508149 4187 raft_consensus.cc:3060] T 313ff5ec7131498aaaa0c832abea2c56 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:19.511214 4187 raft_consensus.cc:2468] T 313ff5ec7131498aaaa0c832abea2c56 P 68f130042e0c4b8292b138ff8a322517 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c77c818054f87bc19600949531ad0 in term 1.
I20260202 02:53:19.512560 4307 leader_election.cc:290] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 [CANDIDATE]: Term 1 election: Requested vote from peers d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425), 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661)
I20260202 02:53:19.512850 4332 raft_consensus.cc:697] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 [term 1 LEADER]: Becoming Leader. State: Replica: a34c77c818054f87bc19600949531ad0, State: Running, Role: LEADER
I20260202 02:53:19.514747 4332 consensus_queue.cc:237] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } }
W20260202 02:53:19.522544 4242 connection.cc:537] client connection to 127.0.104.188:42525 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260202 02:53:19.523298 4242 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.104.188:42525)
W20260202 02:53:19.523651 4242 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.104.188:42525)
W20260202 02:53:19.524217 4242 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.104.188:42525)
W20260202 02:53:19.529186 4329 negotiation.cc:337] Failed RPC negotiation. Trace:
0202 02:53:19.527385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.104.188:42525 (local address 127.0.104.130:48185)
0202 02:53:19.527664 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0202 02:53:19.527685 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0202 02:53:19.527840 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0202 02:53:19.528085 (+ 245us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":146}
I20260202 02:53:19.633163 4359 raft_consensus.cc:493] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 879caa0e6c114ab4855b7a756e252c9e)
I20260202 02:53:19.633399 4359 raft_consensus.cc:515] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:19.634308 4359 leader_election.cc:290] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:53:19.636282 3560 leader_election.cc:336] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361): Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
W20260202 02:53:19.636881 3562 leader_election.cc:336] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:19.637048 3562 leader_election.cc:304] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666b7e52b5fc4f3397f04960c8b67100; no voters: 3acf71c31a284578b4f8c918eaa50205, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:19.637346 4359 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260202 02:53:19.677013 4338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260202 02:53:19.677363 4338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260202 02:53:19.677567 4338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260202 02:53:19.687311 4338 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260202 02:53:19.687426 4338 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260202 02:53:19.687521 4338 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260202 02:53:19.687602 4338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260202 02:53:19.687680 4338 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260202 02:53:19.687737 4338 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260202 02:53:19.698601 4338 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:34531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:46545
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.189:40361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.104.189
--webserver_port=35647
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3722ed49c19ce54c7feae2fa944e32b1d4ed8d2d
build type FASTDEBUG
built by None at 02 Feb 2026 02:43:23 UTC on 24a791456cd2
build id 10304
ASAN enabled
I20260202 02:53:19.700230 4338 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260202 02:53:19.702322 4338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260202 02:53:19.711658 4367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:53:19.712101 4370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:53:19.711724 4368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260202 02:53:19.713083 4338 server_base.cc:1060] running on GCE node
I20260202 02:53:19.713891 4338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260202 02:53:19.715410 4338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260202 02:53:19.716660 4338 hybrid_clock.cc:648] HybridClock initialized: now 1770000799716602 us; error 47 us; skew 500 ppm
I20260202 02:53:19.717024 4338 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260202 02:53:19.719318 4338 webserver.cc:492] Webserver started at http://127.0.104.189:35647/ using document root <none> and password file <none>
I20260202 02:53:19.720098 4338 fs_manager.cc:362] Metadata directory not provided
I20260202 02:53:19.720223 4338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260202 02:53:19.726667 4338 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260202 02:53:19.730126 4376 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260202 02:53:19.731662 4338 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260202 02:53:19.731845 4338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data,/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/wal
uuid: "3acf71c31a284578b4f8c918eaa50205"
format_stamp: "Formatted at 2026-02-02 02:53:15 on dist-test-slave-llj2"
I20260202 02:53:19.732702 4338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260202 02:53:19.752607 4338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260202 02:53:19.753548 4338 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260202 02:53:19.753835 4338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260202 02:53:19.774977 4338 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.189:40361
I20260202 02:53:19.774972 4427 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.189:40361 every 8 connection(s)
I20260202 02:53:19.776880 4338 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data/info.pb
I20260202 02:53:19.781596 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskYXxklX/build/asan/bin/kudu as pid 4338
I20260202 02:53:19.788312 4428 sys_catalog.cc:263] Verifying existing consensus state
I20260202 02:53:19.796378 4428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: Bootstrap starting.
I20260202 02:53:19.811309 4428 log.cc:826] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: Log is configured to *not* fsync() on all Append() calls
I20260202 02:53:19.815269 3956 heartbeater.cc:344] Connected to a master server at 127.0.104.189:40361
I20260202 02:53:19.823864 4428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260202 02:53:19.824571 4428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: Bootstrap complete.
I20260202 02:53:19.830361 4428 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:19.831158 4428 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3acf71c31a284578b4f8c918eaa50205, State: Initialized, Role: FOLLOWER
I20260202 02:53:19.831869 4428 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:19.832769 4434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:19.833093 4434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:53:19.833948 4428 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: configured and running, proceeding with master startup.
I20260202 02:53:19.859956 4448 catalog_manager.cc:1269] Loaded cluster ID: 0d95b889b53247a0813a4fda5550ae30
I20260202 02:53:19.860077 4448 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: loading cluster ID for follower catalog manager: success
I20260202 02:53:19.865723 4448 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: acquiring CA information for follower catalog manager: success
I20260202 02:53:19.868283 4448 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260202 02:53:19.882267 4338 master_runner.cc:186] Error getting master registration for 127.0.104.188:42525: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111),
I20260202 02:53:19.885993 4338 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:53:20.021198 4098 heartbeater.cc:344] Connected to a master server at 127.0.104.189:40361
I20260202 02:53:20.023622 4393 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "a34c77c818054f87bc19600949531ad0" instance_seqno: 1770000797481684) as {username='slave'} at 127.0.104.130:40957; Asking this server to re-register.
I20260202 02:53:20.024379 4098 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:20.026139 4393 ts_manager.cc:194] Registered new tserver with Master: a34c77c818054f87bc19600949531ad0 (127.0.104.130:38525)
I20260202 02:53:20.098722 4453 consensus_queue.cc:1048] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:53:20.103425 4393 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "d9570a47043b4891b2392af17f405e3d" instance_seqno: 1770000796958858) as {username='slave'} at 127.0.104.129:50189; Asking this server to re-register.
I20260202 02:53:20.104444 3956 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:20.106294 4393 ts_manager.cc:194] Registered new tserver with Master: d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425)
I20260202 02:53:20.112179 4453 consensus_queue.cc:1048] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:53:20.124168 4234 heartbeater.cc:344] Connected to a master server at 127.0.104.189:40361
I20260202 02:53:20.124468 4234 heartbeater.cc:499] Master 127.0.104.189:40361 was elected leader, sending a full tablet report...
I20260202 02:53:20.126144 4393 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "68f130042e0c4b8292b138ff8a322517" instance_seqno: 1770000797970665) as {username='slave'} at 127.0.104.131:43701; Asking this server to re-register.
I20260202 02:53:20.126809 4234 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:20.128235 4393 ts_manager.cc:194] Registered new tserver with Master: 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661)
I20260202 02:53:20.149763 4465 raft_consensus.cc:493] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 879caa0e6c114ab4855b7a756e252c9e)
I20260202 02:53:20.149950 4465 raft_consensus.cc:515] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:20.150635 4465 leader_election.cc:290] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:53:20.151985 3562 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111) [suppressed 10 similar messages]
W20260202 02:53:20.154330 3562 leader_election.cc:336] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:20.159700 4403 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "666b7e52b5fc4f3397f04960c8b67100" candidate_term: 4 candidate_status { last_received { term: 3 index: 21 } } ignore_live_leader: false dest_uuid: "3acf71c31a284578b4f8c918eaa50205" is_pre_election: true
I20260202 02:53:20.160460 4403 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666b7e52b5fc4f3397f04960c8b67100 in term 2.
I20260202 02:53:20.161087 3560 leader_election.cc:304] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205, 666b7e52b5fc4f3397f04960c8b67100; no voters: 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:20.161412 4465 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260202 02:53:20.161535 4465 raft_consensus.cc:493] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 879caa0e6c114ab4855b7a756e252c9e)
I20260202 02:53:20.161628 4465 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 3 FOLLOWER]: Advancing to term 4
I20260202 02:53:20.163564 4465 raft_consensus.cc:515] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:20.164238 4465 leader_election.cc:290] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 4 election: Requested vote from peers 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
I20260202 02:53:20.164850 4403 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "666b7e52b5fc4f3397f04960c8b67100" candidate_term: 4 candidate_status { last_received { term: 3 index: 21 } } ignore_live_leader: false dest_uuid: "3acf71c31a284578b4f8c918eaa50205"
I20260202 02:53:20.165082 4403 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 2 FOLLOWER]: Advancing to term 4
W20260202 02:53:20.166064 3562 leader_election.cc:336] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:20.170435 4403 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666b7e52b5fc4f3397f04960c8b67100 in term 4.
I20260202 02:53:20.170980 3560 leader_election.cc:304] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205, 666b7e52b5fc4f3397f04960c8b67100; no voters: 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:20.171380 4465 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 4 FOLLOWER]: Leader election won for term 4
I20260202 02:53:20.171862 4465 raft_consensus.cc:697] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 4 LEADER]: Becoming Leader. State: Replica: 666b7e52b5fc4f3397f04960c8b67100, State: Running, Role: LEADER
I20260202 02:53:20.172201 4465 consensus_queue.cc:237] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 3.21, Last appended by leader: 21, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:20.174005 4469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 666b7e52b5fc4f3397f04960c8b67100. Latest consensus state: current_term: 4 leader_uuid: "666b7e52b5fc4f3397f04960c8b67100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:20.174319 4469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: This master's current role is: LEADER
I20260202 02:53:20.174996 4471 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260202 02:53:20.177482 4471 catalog_manager.cc:679] Loaded metadata for table default.table_6a23cf9a48824f7e8d243c355f458ce8 [id=0048dd71261d4204ac5548c190957aa9]
I20260202 02:53:20.177968 4471 catalog_manager.cc:679] Loaded metadata for table default.table_8d32aa2872ab405fa4f2e7b1dac33f56 [id=2ab6a7a17803446898f275ae3faa032c]
I20260202 02:53:20.178332 4471 catalog_manager.cc:679] Loaded metadata for table default.table_2c330ab9e5984ba5980e7264050e7a41 [id=3852a850379342c69688c71b95873308]
I20260202 02:53:20.178706 4471 catalog_manager.cc:679] Loaded metadata for table default.table_4e64388f537d44aaa6d541b90e924c65 [id=4f41a1a5a6844fbfb43b6798b2c6f67d]
I20260202 02:53:20.179033 4471 catalog_manager.cc:679] Loaded metadata for table default.table_470455fd011c452b9cceafbc413b1cfe [id=e28252bb319446c39056986e8b98de30]
I20260202 02:53:20.180866 4471 tablet_loader.cc:96] loaded metadata for tablet 313ff5ec7131498aaaa0c832abea2c56 (table default.table_6a23cf9a48824f7e8d243c355f458ce8 [id=0048dd71261d4204ac5548c190957aa9])
I20260202 02:53:20.181418 4471 tablet_loader.cc:96] loaded metadata for tablet 76c092d3e26c4906964e0f852ac24448 (table default.table_4e64388f537d44aaa6d541b90e924c65 [id=4f41a1a5a6844fbfb43b6798b2c6f67d])
I20260202 02:53:20.181766 4471 tablet_loader.cc:96] loaded metadata for tablet 8d1373239ae14791a720a0a7ca86d09a (table default.table_2c330ab9e5984ba5980e7264050e7a41 [id=3852a850379342c69688c71b95873308])
I20260202 02:53:20.182071 4471 tablet_loader.cc:96] loaded metadata for tablet 906f9148074848da8456b204a587fa8a (table default.table_8d32aa2872ab405fa4f2e7b1dac33f56 [id=2ab6a7a17803446898f275ae3faa032c])
I20260202 02:53:20.182411 4471 tablet_loader.cc:96] loaded metadata for tablet b5a121d55d9d498ba76bb4dcb24d0e20 (table default.table_470455fd011c452b9cceafbc413b1cfe [id=e28252bb319446c39056986e8b98de30])
I20260202 02:53:20.182663 4471 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260202 02:53:20.183696 4471 catalog_manager.cc:1269] Loaded cluster ID: 0d95b889b53247a0813a4fda5550ae30
I20260202 02:53:20.183773 4471 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260202 02:53:20.185199 4471 catalog_manager.cc:1514] Loading token signing keys...
I20260202 02:53:20.186103 4471 catalog_manager.cc:6038] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: Loaded TSK: 0
I20260202 02:53:20.186792 4471 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260202 02:53:20.187683 4471 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260202 02:53:20.188576 4471 catalog_manager.cc:5876] Last processed Hive Metastore notification event ID: -1
W20260202 02:53:20.194948 3562 consensus_peers.cc:597] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 -> Peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Couldn't send request to peer 879caa0e6c114ab4855b7a756e252c9e. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260202 02:53:20.194949 4403 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 4 FOLLOWER]: Refusing update from remote peer 666b7e52b5fc4f3397f04960c8b67100: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 23. (index mismatch)
I20260202 02:53:20.195886 4469 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 5, Time since last communication: 0.000s
I20260202 02:53:20.197523 4461 consensus_queue.cc:1048] T b5a121d55d9d498ba76bb4dcb24d0e20 P 68f130042e0c4b8292b138ff8a322517 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:53:20.203080 3575 catalog_manager.cc:5654] T b5a121d55d9d498ba76bb4dcb24d0e20 P d9570a47043b4891b2392af17f405e3d reported cstate change: term changed from 0 to 1, leader changed from <none> to 68f130042e0c4b8292b138ff8a322517 (127.0.104.131). New cstate: current_term: 1 leader_uuid: "68f130042e0c4b8292b138ff8a322517" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } }
I20260202 02:53:20.212690 3574 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58490:
name: "default.Table_fabda1a38a254c23a907bc93e3777342"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:20.214861 3573 master_service.cc:954] ReplaceTablet: received request to replace tablet 8d1373239ae14791a720a0a7ca86d09a from {username='slave'} at 127.0.0.1:58490
I20260202 02:53:20.215240 3572 master_service.cc:954] ReplaceTablet: received request to replace tablet 76c092d3e26c4906964e0f852ac24448 from {username='slave'} at 127.0.0.1:58490
I20260202 02:53:20.215883 3570 master_service.cc:954] ReplaceTablet: received request to replace tablet 313ff5ec7131498aaaa0c832abea2c56 from {username='slave'} at 127.0.0.1:58490
I20260202 02:53:20.217268 3571 master_service.cc:954] ReplaceTablet: received request to replace tablet 313ff5ec7131498aaaa0c832abea2c56 from {username='slave'} at 127.0.0.1:58490
I20260202 02:53:20.217675 3569 master_service.cc:954] ReplaceTablet: received request to replace tablet 8d1373239ae14791a720a0a7ca86d09a from {username='slave'} at 127.0.0.1:58490
I20260202 02:53:20.218012 3568 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:58490:
table { table_name: "default.Table_2c330ab9e5984ba5980e7264050e7a41" } new_table_name: "default.Table_b435fc4b22d0426ebfb76e842d4b43f1" modify_external_catalogs: true
I20260202 02:53:20.228231 4454 consensus_queue.cc:1048] T 906f9148074848da8456b204a587fa8a P d9570a47043b4891b2392af17f405e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:53:20.218243 3567 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58490:
name: "default.Table_4ec28fc3e85f44128ce25471c8cec8b9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260202 02:53:20.235227 3771 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 7 received events: NotificationEvent(eventId=1, eventTime=1770000798, eventType=CREATE_TABLE, dbName=default, tableName=table_4e64388f537d44aaa6d541b90e924c65, message={"server":"","servicePrincipal":"","db":"default","table":"table_4e64388f537d44aaa6d541b90e924c65","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_4e64388f537d44aaa6d541b90e924c65\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770000798},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/warehouse/table_4e64388f537d44aaa6d541b90e924c65\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_4e64388f537d44aaa6d541b90e924c65\",\"kudu.table_id\":\"4f41a1a5a6844fbfb43b6798b2c6f67d\",\"transient_lastDdlTime\":\"1770000798\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"0d95b889b53247a0813a4fda5550ae30\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770000798,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1770000798, eventType=CREATE_TABLE, dbName=default, tableName=table_a728b4dd173b4f0c8c6b9cce0bcda94d, message={"server":"","servicePrincipal":"","db":"default","table":"table_a728b4dd173b4f0c8c6b9cce0bcda94d","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_a728b4dd173b4f0c8c6b9cce0bcda94d\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770000798},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/warehouse/table_a728b4dd173b4f0c8c6b9cce0bcda94d\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_a728b4dd173b4f0c8c6b9cce0bcda94d\",\"kudu.table_id\":\"a54f76c7d1ff4769b2d84a5fe325d81c\",\"transient_lastDdlTime\":\"1770000798\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"0d95b889b53247a0813a4fda5550ae30\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770000798,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
I20260202 02:53:20.233558 3566 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58490:
name: "default.Table_851c0addb9fd444eb3b9dd3a3ddd4623"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
@ 0x7f3459ff4dcd google::LogMessage::Fail() at ??:0
@ 0x7f3459ff8b93 google::LogMessage::SendToLog() at ??:0
@ 0x7f3459ff47cc google::LogMessage::Flush() at ??:0
@ 0x7f3459ff4a11 google::LogMessage::~LogMessage() at ??:0
I20260202 02:53:20.243402 4434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 666b7e52b5fc4f3397f04960c8b67100. Latest consensus state: current_term: 4 leader_uuid: "666b7e52b5fc4f3397f04960c8b67100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:20.243707 4434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:53:20.246901 4477 consensus_queue.cc:1048] T b5a121d55d9d498ba76bb4dcb24d0e20 P 68f130042e0c4b8292b138ff8a322517 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
@ 0x7f34709eb7f4 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260202 02:53:20.258457 4479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "666b7e52b5fc4f3397f04960c8b67100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:20.258846 4479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7f34709ea0a1 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f34709f1d1c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260202 02:53:20.273854 4479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "666b7e52b5fc4f3397f04960c8b67100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:20.274114 4479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7f34709f1b99 std::_Function_handler<>::_M_invoke() at ??:0
I20260202 02:53:20.276733 4469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "666b7e52b5fc4f3397f04960c8b67100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:20.276988 4469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: This master's current role is: LEADER
I20260202 02:53:20.281077 3955 heartbeater.cc:499] Master 127.0.104.190:34577 was elected leader, sending a full tablet report...
I20260202 02:53:20.281895 4475 mvcc.cc:204] Tried to move back new op lower bound from 7249923277588406272 to 7249923270296547328. Current Snapshot: MvccSnapshot[applied={T|T < 7249923273218785280}]
@ 0xa8970d std::function<>::operator()() at ??:?
@ 0x7f345cd69f99 kudu::Thread::SuperviseThread() at ??:0
I20260202 02:53:20.287118 3573 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: Sending DeleteTablet for 3 replicas of tablet 8d1373239ae14791a720a0a7ca86d09a
I20260202 02:53:20.289305 3573 catalog_manager.cc:6724] ReplaceTablet: tablet 8d1373239ae14791a720a0a7ca86d09a deleted and replaced by tablet 7b76f6283a604527946408b2a8983ee5
I20260202 02:53:20.297859 4489 consensus_queue.cc:1048] T 906f9148074848da8456b204a587fa8a P d9570a47043b4891b2392af17f405e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:53:20.301046 3570 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: Sending DeleteTablet for 3 replicas of tablet 313ff5ec7131498aaaa0c832abea2c56
I20260202 02:53:20.301615 3570 catalog_manager.cc:6724] ReplaceTablet: tablet 313ff5ec7131498aaaa0c832abea2c56 deleted and replaced by tablet aa58249c51134388b92c95408bd8c923
I20260202 02:53:20.302031 3570 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58490:
name: "default.Table_2a3cc2b06aae483d8e2b84d018d13310"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:20.302791 3572 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: Sending DeleteTablet for 3 replicas of tablet 76c092d3e26c4906964e0f852ac24448
I20260202 02:53:20.303454 3572 catalog_manager.cc:6724] ReplaceTablet: tablet 76c092d3e26c4906964e0f852ac24448 deleted and replaced by tablet c58cc255392e4a74b94c1cfae918c70e
I20260202 02:53:20.303817 3572 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58490:
name: "default.Table_2d03ff86a95f4175b2225ecc0d899058"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
@ 0x7f34603766db start_thread at ??:0
I20260202 02:53:20.322929 3575 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58490:
name: "default.Table_a728b4dd173b4f0c8c6b9cce0bcda94d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
@ 0x7f3458ea571f clone at ??:0
I20260202 02:53:20.324004 4233 heartbeater.cc:499] Master 127.0.104.190:34577 was elected leader, sending a full tablet report...
I20260202 02:53:20.324386 3888 tablet_service.cc:1552] Processing DeleteTablet for tablet 8d1373239ae14791a720a0a7ca86d09a with delete_type TABLET_DATA_DELETED (replaced by tablet 7b76f6283a604527946408b2a8983ee5) from {username='slave'} at 127.0.0.1:39544
I20260202 02:53:20.326037 4166 tablet_service.cc:1552] Processing DeleteTablet for tablet 8d1373239ae14791a720a0a7ca86d09a with delete_type TABLET_DATA_DELETED (replaced by tablet 7b76f6283a604527946408b2a8983ee5) from {username='slave'} at 127.0.0.1:37328
I20260202 02:53:20.326126 4167 tablet_service.cc:1552] Processing DeleteTablet for tablet 313ff5ec7131498aaaa0c832abea2c56 with delete_type TABLET_DATA_DELETED (replaced by tablet aa58249c51134388b92c95408bd8c923) from {username='slave'} at 127.0.0.1:37328
I20260202 02:53:20.326357 4165 tablet_service.cc:1552] Processing DeleteTablet for tablet 76c092d3e26c4906964e0f852ac24448 with delete_type TABLET_DATA_DELETED (replaced by tablet c58cc255392e4a74b94c1cfae918c70e) from {username='slave'} at 127.0.0.1:37328
I20260202 02:53:20.330786 3889 tablet_service.cc:1552] Processing DeleteTablet for tablet 313ff5ec7131498aaaa0c832abea2c56 with delete_type TABLET_DATA_DELETED (replaced by tablet aa58249c51134388b92c95408bd8c923) from {username='slave'} at 127.0.0.1:39544
I20260202 02:53:20.331429 3889 tablet_service.cc:1552] Processing DeleteTablet for tablet 76c092d3e26c4906964e0f852ac24448 with delete_type TABLET_DATA_DELETED (replaced by tablet c58cc255392e4a74b94c1cfae918c70e) from {username='slave'} at 127.0.0.1:39544
I20260202 02:53:20.332777 4499 tablet_replica.cc:333] T 8d1373239ae14791a720a0a7ca86d09a P 68f130042e0c4b8292b138ff8a322517: stopping tablet replica
I20260202 02:53:20.332973 4499 raft_consensus.cc:2243] T 8d1373239ae14791a720a0a7ca86d09a P 68f130042e0c4b8292b138ff8a322517 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:53:20.333184 4499 raft_consensus.cc:2272] T 8d1373239ae14791a720a0a7ca86d09a P 68f130042e0c4b8292b138ff8a322517 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:53:20.333767 4499 ts_tablet_manager.cc:1916] T 8d1373239ae14791a720a0a7ca86d09a P 68f130042e0c4b8292b138ff8a322517: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:53:20.336012 4498 tablet_replica.cc:333] T 8d1373239ae14791a720a0a7ca86d09a P d9570a47043b4891b2392af17f405e3d: stopping tablet replica
I20260202 02:53:20.336220 4498 raft_consensus.cc:2243] T 8d1373239ae14791a720a0a7ca86d09a P d9570a47043b4891b2392af17f405e3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:53:20.336467 4498 raft_consensus.cc:2272] T 8d1373239ae14791a720a0a7ca86d09a P d9570a47043b4891b2392af17f405e3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:53:20.337072 4498 ts_tablet_manager.cc:1916] T 8d1373239ae14791a720a0a7ca86d09a P d9570a47043b4891b2392af17f405e3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:53:20.344679 4499 ts_tablet_manager.cc:1929] T 8d1373239ae14791a720a0a7ca86d09a P 68f130042e0c4b8292b138ff8a322517: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:53:20.344691 4498 ts_tablet_manager.cc:1929] T 8d1373239ae14791a720a0a7ca86d09a P d9570a47043b4891b2392af17f405e3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:53:20.344816 4499 log.cc:1199] T 8d1373239ae14791a720a0a7ca86d09a P 68f130042e0c4b8292b138ff8a322517: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/8d1373239ae14791a720a0a7ca86d09a
I20260202 02:53:20.344837 4498 log.cc:1199] T 8d1373239ae14791a720a0a7ca86d09a P d9570a47043b4891b2392af17f405e3d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/8d1373239ae14791a720a0a7ca86d09a
I20260202 02:53:20.345204 4499 ts_tablet_manager.cc:1950] T 8d1373239ae14791a720a0a7ca86d09a P 68f130042e0c4b8292b138ff8a322517: Deleting consensus metadata
I20260202 02:53:20.345230 4498 ts_tablet_manager.cc:1950] T 8d1373239ae14791a720a0a7ca86d09a P d9570a47043b4891b2392af17f405e3d: Deleting consensus metadata
I20260202 02:53:20.346472 4498 tablet_replica.cc:333] T 313ff5ec7131498aaaa0c832abea2c56 P d9570a47043b4891b2392af17f405e3d: stopping tablet replica
I20260202 02:53:20.346704 4498 raft_consensus.cc:2243] T 313ff5ec7131498aaaa0c832abea2c56 P d9570a47043b4891b2392af17f405e3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:53:20.346890 4498 raft_consensus.cc:2272] T 313ff5ec7131498aaaa0c832abea2c56 P d9570a47043b4891b2392af17f405e3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:53:20.347224 4499 tablet_replica.cc:333] T 313ff5ec7131498aaaa0c832abea2c56 P 68f130042e0c4b8292b138ff8a322517: stopping tablet replica
I20260202 02:53:20.347429 4499 raft_consensus.cc:2243] T 313ff5ec7131498aaaa0c832abea2c56 P 68f130042e0c4b8292b138ff8a322517 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:53:20.347518 4498 ts_tablet_manager.cc:1916] T 313ff5ec7131498aaaa0c832abea2c56 P d9570a47043b4891b2392af17f405e3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:53:20.347576 4499 raft_consensus.cc:2272] T 313ff5ec7131498aaaa0c832abea2c56 P 68f130042e0c4b8292b138ff8a322517 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:53:20.348067 4499 ts_tablet_manager.cc:1916] T 313ff5ec7131498aaaa0c832abea2c56 P 68f130042e0c4b8292b138ff8a322517: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:53:20.354110 4498 ts_tablet_manager.cc:1929] T 313ff5ec7131498aaaa0c832abea2c56 P d9570a47043b4891b2392af17f405e3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:53:20.354154 4499 ts_tablet_manager.cc:1929] T 313ff5ec7131498aaaa0c832abea2c56 P 68f130042e0c4b8292b138ff8a322517: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260202 02:53:20.354292 4498 log.cc:1199] T 313ff5ec7131498aaaa0c832abea2c56 P d9570a47043b4891b2392af17f405e3d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/313ff5ec7131498aaaa0c832abea2c56
I20260202 02:53:20.354306 4499 log.cc:1199] T 313ff5ec7131498aaaa0c832abea2c56 P 68f130042e0c4b8292b138ff8a322517: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/313ff5ec7131498aaaa0c832abea2c56
I20260202 02:53:20.354647 4498 ts_tablet_manager.cc:1950] T 313ff5ec7131498aaaa0c832abea2c56 P d9570a47043b4891b2392af17f405e3d: Deleting consensus metadata
I20260202 02:53:20.354638 4499 ts_tablet_manager.cc:1950] T 313ff5ec7131498aaaa0c832abea2c56 P 68f130042e0c4b8292b138ff8a322517: Deleting consensus metadata
I20260202 02:53:20.355877 4498 tablet_replica.cc:333] T 76c092d3e26c4906964e0f852ac24448 P d9570a47043b4891b2392af17f405e3d: stopping tablet replica
I20260202 02:53:20.356060 4498 raft_consensus.cc:2243] T 76c092d3e26c4906964e0f852ac24448 P d9570a47043b4891b2392af17f405e3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:53:20.356284 4498 raft_consensus.cc:2272] T 76c092d3e26c4906964e0f852ac24448 P d9570a47043b4891b2392af17f405e3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:53:20.357347 4498 ts_tablet_manager.cc:1916] T 76c092d3e26c4906964e0f852ac24448 P d9570a47043b4891b2392af17f405e3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:53:20.360515 4499 tablet_replica.cc:333] T 76c092d3e26c4906964e0f852ac24448 P 68f130042e0c4b8292b138ff8a322517: stopping tablet replica
I20260202 02:53:20.360715 4499 raft_consensus.cc:2243] T 76c092d3e26c4906964e0f852ac24448 P 68f130042e0c4b8292b138ff8a322517 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:53:20.360934 4499 raft_consensus.cc:2272] T 76c092d3e26c4906964e0f852ac24448 P 68f130042e0c4b8292b138ff8a322517 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:53:20.364106 4498 ts_tablet_manager.cc:1929] T 76c092d3e26c4906964e0f852ac24448 P d9570a47043b4891b2392af17f405e3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:53:20.364260 4498 log.cc:1199] T 76c092d3e26c4906964e0f852ac24448 P d9570a47043b4891b2392af17f405e3d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/76c092d3e26c4906964e0f852ac24448
I20260202 02:53:20.364728 4498 ts_tablet_manager.cc:1950] T 76c092d3e26c4906964e0f852ac24448 P d9570a47043b4891b2392af17f405e3d: Deleting consensus metadata
I20260202 02:53:20.366235 4499 ts_tablet_manager.cc:1916] T 76c092d3e26c4906964e0f852ac24448 P 68f130042e0c4b8292b138ff8a322517: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:53:20.370491 4499 ts_tablet_manager.cc:1929] T 76c092d3e26c4906964e0f852ac24448 P 68f130042e0c4b8292b138ff8a322517: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:53:20.370594 4499 log.cc:1199] T 76c092d3e26c4906964e0f852ac24448 P 68f130042e0c4b8292b138ff8a322517: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/76c092d3e26c4906964e0f852ac24448
I20260202 02:53:20.370862 4499 ts_tablet_manager.cc:1950] T 76c092d3e26c4906964e0f852ac24448 P 68f130042e0c4b8292b138ff8a322517: Deleting consensus metadata
W20260202 02:53:20.517032 3843 connection.cc:537] server connection from 127.0.0.1:39544 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260202 02:53:20.517048 4123 connection.cc:537] client connection to 127.0.104.190:34577 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260202 02:53:20.517316 4121 connection.cc:537] server connection from 127.0.0.1:37328 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260202 02:53:20.517550 4493 negotiation.cc:337] Failed RPC negotiation. Trace:
0202 02:53:20.292423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38038 (local address 127.0.104.130:38525)
0202 02:53:20.307766 (+ 15343us) server_negotiation.cc:207] Beginning negotiation
0202 02:53:20.307773 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0202 02:53:20.310899 (+ 3126us) server_negotiation.cc:408] Connection header received
0202 02:53:20.311836 (+ 937us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0202 02:53:20.311844 (+ 8us) server_negotiation.cc:462] Received NEGOTIATE request from client
0202 02:53:20.312583 (+ 739us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0202 02:53:20.312728 (+ 145us) server_negotiation.cc:227] Negotiated authn=CERTIFICATE
0202 02:53:20.517143 (+204415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38038: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":15198,"thread_start_us":15060,"threads_started":1}
W20260202 02:53:20.517488 4242 connection.cc:537] client connection to 127.0.104.190:34577 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260202 02:53:20.522228 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:53:20.549068 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:53:20.705327 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:53:20.745595 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:53:20.776744 4510 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 666b7e52b5fc4f3397f04960c8b67100)
I20260202 02:53:20.776978 4510 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:20.779038 4510 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:53:20.780661 4380 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:53:20.782608 4380 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:53:20.782851 4380 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:20.782991 4380 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:20.783506 4510 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260202 02:53:21.031390 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 1: this message will repeat every 5th retry.
W20260202 02:53:21.089409 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 1: this message will repeat every 5th retry.
I20260202 02:53:21.426044 4525 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 666b7e52b5fc4f3397f04960c8b67100)
I20260202 02:53:21.426234 4525 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:21.426893 4525 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:53:21.429078 4380 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:53:21.429281 4380 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:21.429406 4380 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:21.429687 4525 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260202 02:53:22.107219 4539 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 666b7e52b5fc4f3397f04960c8b67100)
I20260202 02:53:22.107457 4539 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:22.108270 4539 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:53:22.113605 4380 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:53:22.113873 4380 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:22.113976 4380 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:22.114289 4539 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260202 02:53:22.727798 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskYXxklX/build/asan/bin/kudu
/tmp/dist-test-taskYXxklX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.190:34577
--webserver_interface=127.0.104.190
--webserver_port=33649
--builtin_ntp_servers=127.0.104.148:34531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525
--hive_metastore_uris=thrift://127.0.0.1:46545
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260202 02:53:23.088236 4549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260202 02:53:23.088639 4549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260202 02:53:23.088752 4549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260202 02:53:23.097388 4549 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260202 02:53:23.097500 4549 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260202 02:53:23.097587 4549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260202 02:53:23.097651 4549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260202 02:53:23.097734 4549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260202 02:53:23.097805 4549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260202 02:53:23.108604 4549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:34531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:46545
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.190:34577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.104.190
--webserver_port=33649
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3722ed49c19ce54c7feae2fa944e32b1d4ed8d2d
build type FASTDEBUG
built by None at 02 Feb 2026 02:43:23 UTC on 24a791456cd2
build id 10304
ASAN enabled
I20260202 02:53:23.110304 4549 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260202 02:53:23.112398 4549 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260202 02:53:23.121946 4558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:53:23.122815 4559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:53:23.123284 4561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260202 02:53:23.124457 4549 server_base.cc:1060] running on GCE node
I20260202 02:53:23.125312 4549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260202 02:53:23.126899 4549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260202 02:53:23.128146 4549 hybrid_clock.cc:648] HybridClock initialized: now 1770000803128096 us; error 41 us; skew 500 ppm
I20260202 02:53:23.128607 4549 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260202 02:53:23.131372 4549 webserver.cc:492] Webserver started at http://127.0.104.190:33649/ using document root <none> and password file <none>
I20260202 02:53:23.132222 4549 fs_manager.cc:362] Metadata directory not provided
I20260202 02:53:23.132427 4549 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260202 02:53:23.140681 4549 fs_manager.cc:714] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260202 02:53:23.144385 4567 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260202 02:53:23.146080 4549 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260202 02:53:23.146277 4549 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data,/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/wal
uuid: "666b7e52b5fc4f3397f04960c8b67100"
format_stamp: "Formatted at 2026-02-02 02:53:14 on dist-test-slave-llj2"
I20260202 02:53:23.147068 4549 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260202 02:53:23.167101 4572 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 666b7e52b5fc4f3397f04960c8b67100)
I20260202 02:53:23.167312 4572 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:23.168004 4572 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:53:23.170496 4380 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:53:23.170703 4380 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:23.170811 4380 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:23.171116 4572 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260202 02:53:23.191419 4549 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260202 02:53:23.192448 4549 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260202 02:53:23.192727 4549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260202 02:53:23.214576 4549 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.190:34577
I20260202 02:53:23.214601 4622 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.190:34577 every 8 connection(s)
I20260202 02:53:23.216400 4549 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data/info.pb
I20260202 02:53:23.222131 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskYXxklX/build/asan/bin/kudu as pid 4549
I20260202 02:53:23.224650 4623 sys_catalog.cc:263] Verifying existing consensus state
I20260202 02:53:23.228792 4623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: Bootstrap starting.
I20260202 02:53:23.241961 4623 log.cc:826] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: Log is configured to *not* fsync() on all Append() calls
I20260202 02:53:23.266173 4623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: Bootstrap replayed 1/1 log segments. Stats: ops{read=27 overwritten=0 applied=27 ignored=0} inserts{seen=18 ignored=0} mutations{seen=15 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260202 02:53:23.266780 4623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: Bootstrap complete.
I20260202 02:53:23.272177 4623 raft_consensus.cc:359] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:23.273015 4623 raft_consensus.cc:740] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666b7e52b5fc4f3397f04960c8b67100, State: Initialized, Role: FOLLOWER
I20260202 02:53:23.273715 4623 consensus_queue.cc:260] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 27, Last appended: 4.27, Last appended by leader: 27, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:23.274654 4627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:23.274909 4627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:53:23.275823 4623 sys_catalog.cc:565] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: configured and running, proceeding with master startup.
I20260202 02:53:23.283222 4641 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260202 02:53:23.302855 4641 catalog_manager.cc:1269] Loaded cluster ID: 0d95b889b53247a0813a4fda5550ae30
I20260202 02:53:23.303037 4641 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: success
I20260202 02:53:23.307565 4641 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: acquiring CA information for follower catalog manager: success
I20260202 02:53:23.310848 4641 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260202 02:53:23.312515 4549 master_runner.cc:186] Error getting master registration for 127.0.104.188:42525: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111),
I20260202 02:53:23.316387 4549 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:53:23.352108 3955 heartbeater.cc:344] Connected to a master server at 127.0.104.190:34577
I20260202 02:53:23.352530 3955 heartbeater.cc:499] Master 127.0.104.190:34577 was elected leader, sending a full tablet report...
I20260202 02:53:23.356252 4588 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "d9570a47043b4891b2392af17f405e3d" instance_seqno: 1770000796958858) as {username='slave'} at 127.0.104.129:34287; Asking this server to re-register.
I20260202 02:53:23.357296 3955 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:23.358444 4097 heartbeater.cc:344] Connected to a master server at 127.0.104.190:34577
I20260202 02:53:23.359579 4588 ts_manager.cc:194] Registered new tserver with Master: d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425)
I20260202 02:53:23.360471 4587 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "a34c77c818054f87bc19600949531ad0" instance_seqno: 1770000797481684) as {username='slave'} at 127.0.104.130:43729; Asking this server to re-register.
I20260202 02:53:23.361289 4097 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:23.362995 4587 ts_manager.cc:194] Registered new tserver with Master: a34c77c818054f87bc19600949531ad0 (127.0.104.130:38525)
I20260202 02:53:23.434685 4233 heartbeater.cc:344] Connected to a master server at 127.0.104.190:34577
I20260202 02:53:23.434943 4233 heartbeater.cc:499] Master 127.0.104.190:34577 was elected leader, sending a full tablet report...
I20260202 02:53:23.436651 4587 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "68f130042e0c4b8292b138ff8a322517" instance_seqno: 1770000797970665) as {username='slave'} at 127.0.104.131:56449; Asking this server to re-register.
I20260202 02:53:23.437355 4233 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:23.438884 4587 ts_manager.cc:194] Registered new tserver with Master: 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661)
I20260202 02:53:23.822320 4653 raft_consensus.cc:493] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:23.822589 4653 raft_consensus.cc:515] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:23.824414 4653 leader_election.cc:290] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
I20260202 02:53:23.825129 4402 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "666b7e52b5fc4f3397f04960c8b67100" candidate_term: 5 candidate_status { last_received { term: 4 index: 27 } } ignore_live_leader: false dest_uuid: "3acf71c31a284578b4f8c918eaa50205" is_pre_election: true
I20260202 02:53:23.825393 4402 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666b7e52b5fc4f3397f04960c8b67100 in term 4.
W20260202 02:53:23.825486 4571 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:23.825996 4569 leader_election.cc:304] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205, 666b7e52b5fc4f3397f04960c8b67100; no voters:
I20260202 02:53:23.826396 4653 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260202 02:53:23.826570 4653 raft_consensus.cc:493] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:53:23.826676 4653 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 4 FOLLOWER]: Advancing to term 5
W20260202 02:53:23.827083 4571 leader_election.cc:336] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:23.830359 4653 raft_consensus.cc:515] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:23.831103 4653 leader_election.cc:290] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 5 election: Requested vote from peers 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
I20260202 02:53:23.831657 4402 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "666b7e52b5fc4f3397f04960c8b67100" candidate_term: 5 candidate_status { last_received { term: 4 index: 27 } } ignore_live_leader: false dest_uuid: "3acf71c31a284578b4f8c918eaa50205"
I20260202 02:53:23.831868 4402 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 4 FOLLOWER]: Advancing to term 5
W20260202 02:53:23.832510 4571 leader_election.cc:336] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:23.834020 4402 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666b7e52b5fc4f3397f04960c8b67100 in term 5.
I20260202 02:53:23.834461 4569 leader_election.cc:304] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205, 666b7e52b5fc4f3397f04960c8b67100; no voters: 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:23.834825 4653 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 5 FOLLOWER]: Leader election won for term 5
I20260202 02:53:23.835201 4653 raft_consensus.cc:697] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 5 LEADER]: Becoming Leader. State: Replica: 666b7e52b5fc4f3397f04960c8b67100, State: Running, Role: LEADER
I20260202 02:53:23.835727 4653 consensus_queue.cc:237] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 4.27, Last appended by leader: 27, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:23.839787 4656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 666b7e52b5fc4f3397f04960c8b67100. Latest consensus state: current_term: 5 leader_uuid: "666b7e52b5fc4f3397f04960c8b67100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:23.840034 4656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: This master's current role is: LEADER
I20260202 02:53:23.841146 4658 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260202 02:53:23.844273 4658 catalog_manager.cc:679] Loaded metadata for table default.table_6a23cf9a48824f7e8d243c355f458ce8 [id=0048dd71261d4204ac5548c190957aa9]
I20260202 02:53:23.844898 4658 catalog_manager.cc:679] Loaded metadata for table default.table_8d32aa2872ab405fa4f2e7b1dac33f56 [id=2ab6a7a17803446898f275ae3faa032c]
I20260202 02:53:23.845355 4658 catalog_manager.cc:679] Loaded metadata for table default.table_2c330ab9e5984ba5980e7264050e7a41 [id=3852a850379342c69688c71b95873308]
I20260202 02:53:23.845883 4658 catalog_manager.cc:679] Loaded metadata for table default.table_4e64388f537d44aaa6d541b90e924c65 [id=4f41a1a5a6844fbfb43b6798b2c6f67d]
I20260202 02:53:23.846305 4658 catalog_manager.cc:679] Loaded metadata for table default.table_470455fd011c452b9cceafbc413b1cfe [id=e28252bb319446c39056986e8b98de30]
I20260202 02:53:23.848448 4658 tablet_loader.cc:96] loaded metadata for tablet 7b76f6283a604527946408b2a8983ee5 (table default.table_2c330ab9e5984ba5980e7264050e7a41 [id=3852a850379342c69688c71b95873308])
I20260202 02:53:23.848944 4658 tablet_loader.cc:96] loaded metadata for tablet 906f9148074848da8456b204a587fa8a (table default.table_8d32aa2872ab405fa4f2e7b1dac33f56 [id=2ab6a7a17803446898f275ae3faa032c])
I20260202 02:53:23.849239 4658 tablet_loader.cc:96] loaded metadata for tablet aa58249c51134388b92c95408bd8c923 (table default.table_6a23cf9a48824f7e8d243c355f458ce8 [id=0048dd71261d4204ac5548c190957aa9])
I20260202 02:53:23.849556 4658 tablet_loader.cc:96] loaded metadata for tablet b5a121d55d9d498ba76bb4dcb24d0e20 (table default.table_470455fd011c452b9cceafbc413b1cfe [id=e28252bb319446c39056986e8b98de30])
I20260202 02:53:23.849893 4658 tablet_loader.cc:96] loaded metadata for tablet c58cc255392e4a74b94c1cfae918c70e (table default.table_4e64388f537d44aaa6d541b90e924c65 [id=4f41a1a5a6844fbfb43b6798b2c6f67d])
I20260202 02:53:23.850193 4658 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260202 02:53:23.851215 4658 catalog_manager.cc:1269] Loaded cluster ID: 0d95b889b53247a0813a4fda5550ae30
I20260202 02:53:23.851294 4658 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260202 02:53:23.852769 4658 catalog_manager.cc:1514] Loading token signing keys...
I20260202 02:53:23.853703 4658 catalog_manager.cc:6038] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: Loaded TSK: 0
I20260202 02:53:23.854585 4658 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260202 02:53:23.855486 4658 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260202 02:53:23.856349 4658 catalog_manager.cc:5876] Last processed Hive Metastore notification event ID: -1
I20260202 02:53:23.867446 4402 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 5 FOLLOWER]: Refusing update from remote peer 666b7e52b5fc4f3397f04960c8b67100: Log matching property violated. Preceding OpId in replica: term: 4 index: 27. Preceding OpId from leader: term: 5 index: 29. (index mismatch)
W20260202 02:53:23.868202 4571 consensus_peers.cc:597] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 -> Peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Couldn't send request to peer 879caa0e6c114ab4855b7a756e252c9e. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260202 02:53:23.868242 4656 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20260202 02:53:23.871233 4661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 666b7e52b5fc4f3397f04960c8b67100. Latest consensus state: current_term: 5 leader_uuid: "666b7e52b5fc4f3397f04960c8b67100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:23.871483 4661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:53:23.873387 4653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "666b7e52b5fc4f3397f04960c8b67100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:23.873674 4653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: This master's current role is: LEADER
I20260202 02:53:23.874202 4661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "666b7e52b5fc4f3397f04960c8b67100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:23.874380 4661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:53:23.890986 4166 tablet_service.cc:1505] Processing CreateTablet for tablet c58cc255392e4a74b94c1cfae918c70e (DEFAULT_TABLE table=default.table_4e64388f537d44aaa6d541b90e924c65 [id=4f41a1a5a6844fbfb43b6798b2c6f67d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:23.891184 4167 tablet_service.cc:1505] Processing CreateTablet for tablet aa58249c51134388b92c95408bd8c923 (DEFAULT_TABLE table=default.table_6a23cf9a48824f7e8d243c355f458ce8 [id=0048dd71261d4204ac5548c190957aa9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:23.891214 4165 tablet_service.cc:1505] Processing CreateTablet for tablet 7b76f6283a604527946408b2a8983ee5 (DEFAULT_TABLE table=default.table_2c330ab9e5984ba5980e7264050e7a41 [id=3852a850379342c69688c71b95873308]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:23.891696 4166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c58cc255392e4a74b94c1cfae918c70e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:23.892915 3888 tablet_service.cc:1505] Processing CreateTablet for tablet c58cc255392e4a74b94c1cfae918c70e (DEFAULT_TABLE table=default.table_4e64388f537d44aaa6d541b90e924c65 [id=4f41a1a5a6844fbfb43b6798b2c6f67d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:23.893352 3888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c58cc255392e4a74b94c1cfae918c70e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:23.894250 4030 tablet_service.cc:1505] Processing CreateTablet for tablet c58cc255392e4a74b94c1cfae918c70e (DEFAULT_TABLE table=default.table_4e64388f537d44aaa6d541b90e924c65 [id=4f41a1a5a6844fbfb43b6798b2c6f67d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:23.894610 4029 tablet_service.cc:1505] Processing CreateTablet for tablet aa58249c51134388b92c95408bd8c923 (DEFAULT_TABLE table=default.table_6a23cf9a48824f7e8d243c355f458ce8 [id=0048dd71261d4204ac5548c190957aa9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:23.894734 4030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c58cc255392e4a74b94c1cfae918c70e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:23.895061 4029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa58249c51134388b92c95408bd8c923. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:23.895324 4031 tablet_service.cc:1505] Processing CreateTablet for tablet 7b76f6283a604527946408b2a8983ee5 (DEFAULT_TABLE table=default.table_2c330ab9e5984ba5980e7264050e7a41 [id=3852a850379342c69688c71b95873308]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:23.895720 4031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b76f6283a604527946408b2a8983ee5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:23.896660 3889 tablet_service.cc:1505] Processing CreateTablet for tablet 7b76f6283a604527946408b2a8983ee5 (DEFAULT_TABLE table=default.table_2c330ab9e5984ba5980e7264050e7a41 [id=3852a850379342c69688c71b95873308]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:23.897063 3889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b76f6283a604527946408b2a8983ee5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:23.900780 3887 tablet_service.cc:1505] Processing CreateTablet for tablet aa58249c51134388b92c95408bd8c923 (DEFAULT_TABLE table=default.table_6a23cf9a48824f7e8d243c355f458ce8 [id=0048dd71261d4204ac5548c190957aa9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:23.901203 3887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa58249c51134388b92c95408bd8c923. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:23.904412 4167 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa58249c51134388b92c95408bd8c923. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:23.904740 4165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b76f6283a604527946408b2a8983ee5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:23.904987 4672 tablet_bootstrap.cc:492] T c58cc255392e4a74b94c1cfae918c70e P a34c77c818054f87bc19600949531ad0: Bootstrap starting.
I20260202 02:53:23.906523 4672 tablet_bootstrap.cc:654] T c58cc255392e4a74b94c1cfae918c70e P a34c77c818054f87bc19600949531ad0: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:23.908413 4672 tablet_bootstrap.cc:492] T c58cc255392e4a74b94c1cfae918c70e P a34c77c818054f87bc19600949531ad0: No bootstrap required, opened a new log
I20260202 02:53:23.908589 4672 ts_tablet_manager.cc:1403] T c58cc255392e4a74b94c1cfae918c70e P a34c77c818054f87bc19600949531ad0: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260202 02:53:23.909415 4672 raft_consensus.cc:359] T c58cc255392e4a74b94c1cfae918c70e P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:23.909637 4672 raft_consensus.cc:385] T c58cc255392e4a74b94c1cfae918c70e P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:23.909725 4672 raft_consensus.cc:740] T c58cc255392e4a74b94c1cfae918c70e P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a34c77c818054f87bc19600949531ad0, State: Initialized, Role: FOLLOWER
I20260202 02:53:23.909958 4672 consensus_queue.cc:260] T c58cc255392e4a74b94c1cfae918c70e P a34c77c818054f87bc19600949531ad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:23.911525 4672 ts_tablet_manager.cc:1434] T c58cc255392e4a74b94c1cfae918c70e P a34c77c818054f87bc19600949531ad0: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260202 02:53:23.911850 4672 tablet_bootstrap.cc:492] T aa58249c51134388b92c95408bd8c923 P a34c77c818054f87bc19600949531ad0: Bootstrap starting.
I20260202 02:53:23.913756 4672 tablet_bootstrap.cc:654] T aa58249c51134388b92c95408bd8c923 P a34c77c818054f87bc19600949531ad0: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:23.916307 4097 heartbeater.cc:499] Master 127.0.104.190:34577 was elected leader, sending a full tablet report...
W20260202 02:53:23.918928 4587 catalog_manager.cc:5487] Ignoring report from unknown tablet 313ff5ec7131498aaaa0c832abea2c56
W20260202 02:53:23.919088 4587 catalog_manager.cc:5487] Ignoring report from unknown tablet 76c092d3e26c4906964e0f852ac24448
W20260202 02:53:23.919255 4587 catalog_manager.cc:5487] Ignoring report from unknown tablet 8d1373239ae14791a720a0a7ca86d09a
I20260202 02:53:23.925069 4588 master_service.cc:954] ReplaceTablet: received request to replace tablet 313ff5ec7131498aaaa0c832abea2c56 from {username='slave'} at 127.0.0.1:59224
I20260202 02:53:23.920148 4587 catalog_manager.cc:5654] T 906f9148074848da8456b204a587fa8a P a34c77c818054f87bc19600949531ad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d9570a47043b4891b2392af17f405e3d (127.0.104.129). New cstate: current_term: 1 leader_uuid: "d9570a47043b4891b2392af17f405e3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } }
I20260202 02:53:23.927672 4585 master_service.cc:954] ReplaceTablet: received request to replace tablet 313ff5ec7131498aaaa0c832abea2c56 from {username='slave'} at 127.0.0.1:59224
I20260202 02:53:23.928525 4581 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:59224:
table { table_name: "default.Table_8d32aa2872ab405fa4f2e7b1dac33f56" } new_table_name: "default.Table_042b89c2475b4966b48af7927ba62e21" modify_external_catalogs: true
I20260202 02:53:23.928617 4583 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59224:
name: "default.Table_fb2a5c4107db446aba29a7f07a46c1a8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:23.929508 4585 master_service.cc:954] ReplaceTablet: received request to replace tablet 313ff5ec7131498aaaa0c832abea2c56 from {username='slave'} at 127.0.0.1:59224
I20260202 02:53:23.932847 4588 master_service.cc:954] ReplaceTablet: received request to replace tablet 8d1373239ae14791a720a0a7ca86d09a from {username='slave'} at 127.0.0.1:59224
I20260202 02:53:23.933389 4586 master_service.cc:954] ReplaceTablet: received request to replace tablet 313ff5ec7131498aaaa0c832abea2c56 from {username='slave'} at 127.0.0.1:59224
I20260202 02:53:23.933877 4586 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59224:
name: "default.Table_2a3cc2b06aae483d8e2b84d018d13310"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:23.934108 4579 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:59224:
table { table_name: "default.Table_2c330ab9e5984ba5980e7264050e7a41" } new_table_name: "default.Table_b435fc4b22d0426ebfb76e842d4b43f1" modify_external_catalogs: true
I20260202 02:53:23.935529 4674 tablet_bootstrap.cc:492] T aa58249c51134388b92c95408bd8c923 P 68f130042e0c4b8292b138ff8a322517: Bootstrap starting.
I20260202 02:53:23.937858 4674 tablet_bootstrap.cc:654] T aa58249c51134388b92c95408bd8c923 P 68f130042e0c4b8292b138ff8a322517: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:23.938086 4676 tablet_bootstrap.cc:492] T 7b76f6283a604527946408b2a8983ee5 P d9570a47043b4891b2392af17f405e3d: Bootstrap starting.
I20260202 02:53:23.940441 4676 tablet_bootstrap.cc:654] T 7b76f6283a604527946408b2a8983ee5 P d9570a47043b4891b2392af17f405e3d: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:23.940673 4674 tablet_bootstrap.cc:492] T aa58249c51134388b92c95408bd8c923 P 68f130042e0c4b8292b138ff8a322517: No bootstrap required, opened a new log
I20260202 02:53:23.940478 4585 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59224:
name: "default.Table_4ec28fc3e85f44128ce25471c8cec8b9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:23.940819 4674 ts_tablet_manager.cc:1403] T aa58249c51134388b92c95408bd8c923 P 68f130042e0c4b8292b138ff8a322517: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260202 02:53:23.940997 4582 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:59224:
table { table_name: "default.Table_6a23cf9a48824f7e8d243c355f458ce8" } new_table_name: "default.Table_824602d24d0345759eda1443b7232548" modify_external_catalogs: true
I20260202 02:53:23.941356 4580 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59224:
name: "default.Table_fabda1a38a254c23a907bc93e3777342"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:23.941649 4674 raft_consensus.cc:359] T aa58249c51134388b92c95408bd8c923 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:23.941845 4674 raft_consensus.cc:385] T aa58249c51134388b92c95408bd8c923 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:23.941924 4674 raft_consensus.cc:740] T aa58249c51134388b92c95408bd8c923 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f130042e0c4b8292b138ff8a322517, State: Initialized, Role: FOLLOWER
I20260202 02:53:23.941807 4584 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:59224:
table { table_name: "default.Table_470455fd011c452b9cceafbc413b1cfe" } new_table_name: "default.Table_d4d44071ee424b9fa646dd6fe1c3ecd8" modify_external_catalogs: true
I20260202 02:53:23.942173 4674 consensus_queue.cc:260] T aa58249c51134388b92c95408bd8c923 P 68f130042e0c4b8292b138ff8a322517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:23.945822 4588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59224:
name: "default.Table_851c0addb9fd444eb3b9dd3a3ddd4623"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:23.955672 4587 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59224:
name: "default.Table_2d03ff86a95f4175b2225ecc0d899058"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260202 02:53:23.956825 4640 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 7 received events: NotificationEvent(eventId=1, eventTime=1770000798, eventType=CREATE_TABLE, dbName=default, tableName=table_4e64388f537d44aaa6d541b90e924c65, message={"server":"","servicePrincipal":"","db":"default","table":"table_4e64388f537d44aaa6d541b90e924c65","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_4e64388f537d44aaa6d541b90e924c65\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770000798},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/warehouse/table_4e64388f537d44aaa6d541b90e924c65\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_4e64388f537d44aaa6d541b90e924c65\",\"kudu.table_id\":\"4f41a1a5a6844fbfb43b6798b2c6f67d\",\"transient_lastDdlTime\":\"1770000798\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"0d95b889b53247a0813a4fda5550ae30\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770000798,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1770000798, eventType=CREATE_TABLE, dbName=default, tableName=table_a728b4dd173b4f0c8c6b9cce0bcda94d, message={"server":"","servicePrincipal":"","db":"default","table":"table_a728b4dd173b4f0c8c6b9cce0bcda94d","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_a728b4dd173b4f0c8c6b9cce0bcda94d\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770000798},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/warehouse/table_a728b4dd173b4f0c8c6b9cce0bcda94d\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_a728b4dd173b4f0c8c6b9cce0bcda94d\",\"kudu.table_id\":\"a54f76c7d1ff4769b2d84a5fe325d81c\",\"transient_lastDdlTime\":\"1770000798\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"0d95b889b53247a0813a4fda5550ae30\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770000798,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7f49251fbdcd google::LogMessage::Fail() at ??:0
@ 0x7f49251ffb93 google::LogMessage::SendToLog() at ??:0
I20260202 02:53:23.961236 4672 tablet_bootstrap.cc:492] T aa58249c51134388b92c95408bd8c923 P a34c77c818054f87bc19600949531ad0: No bootstrap required, opened a new log
I20260202 02:53:23.961447 4672 ts_tablet_manager.cc:1403] T aa58249c51134388b92c95408bd8c923 P a34c77c818054f87bc19600949531ad0: Time spent bootstrapping tablet: real 0.050s user 0.010s sys 0.000s
@ 0x7f49251fb7cc google::LogMessage::Flush() at ??:0
I20260202 02:53:23.962155 4676 tablet_bootstrap.cc:492] T 7b76f6283a604527946408b2a8983ee5 P d9570a47043b4891b2392af17f405e3d: No bootstrap required, opened a new log
I20260202 02:53:23.962309 4676 ts_tablet_manager.cc:1403] T 7b76f6283a604527946408b2a8983ee5 P d9570a47043b4891b2392af17f405e3d: Time spent bootstrapping tablet: real 0.024s user 0.018s sys 0.004s
I20260202 02:53:23.962327 4233 heartbeater.cc:499] Master 127.0.104.190:34577 was elected leader, sending a full tablet report...
@ 0x7f49251fba11 google::LogMessage::~LogMessage() at ??:0
I20260202 02:53:23.963364 4672 raft_consensus.cc:359] T aa58249c51134388b92c95408bd8c923 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:23.963572 4672 raft_consensus.cc:385] T aa58249c51134388b92c95408bd8c923 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:23.963639 4672 raft_consensus.cc:740] T aa58249c51134388b92c95408bd8c923 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a34c77c818054f87bc19600949531ad0, State: Initialized, Role: FOLLOWER
I20260202 02:53:23.963915 4676 raft_consensus.cc:359] T 7b76f6283a604527946408b2a8983ee5 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:23.963943 4672 consensus_queue.cc:260] T aa58249c51134388b92c95408bd8c923 P a34c77c818054f87bc19600949531ad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:23.964111 4676 raft_consensus.cc:385] T 7b76f6283a604527946408b2a8983ee5 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:23.964190 4676 raft_consensus.cc:740] T 7b76f6283a604527946408b2a8983ee5 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9570a47043b4891b2392af17f405e3d, State: Initialized, Role: FOLLOWER
I20260202 02:53:23.964430 4676 consensus_queue.cc:260] T 7b76f6283a604527946408b2a8983ee5 P d9570a47043b4891b2392af17f405e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:23.964983 4676 ts_tablet_manager.cc:1434] T 7b76f6283a604527946408b2a8983ee5 P d9570a47043b4891b2392af17f405e3d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260202 02:53:23.965299 4676 tablet_bootstrap.cc:492] T c58cc255392e4a74b94c1cfae918c70e P d9570a47043b4891b2392af17f405e3d: Bootstrap starting.
I20260202 02:53:23.965446 3955 heartbeater.cc:499] Master 127.0.104.190:34577 was elected leader, sending a full tablet report...
I20260202 02:53:23.968039 4672 ts_tablet_manager.cc:1434] T aa58249c51134388b92c95408bd8c923 P a34c77c818054f87bc19600949531ad0: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260202 02:53:23.969103 4672 tablet_bootstrap.cc:492] T 7b76f6283a604527946408b2a8983ee5 P a34c77c818054f87bc19600949531ad0: Bootstrap starting.
I20260202 02:53:23.970791 4672 tablet_bootstrap.cc:654] T 7b76f6283a604527946408b2a8983ee5 P a34c77c818054f87bc19600949531ad0: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:23.972541 4672 tablet_bootstrap.cc:492] T 7b76f6283a604527946408b2a8983ee5 P a34c77c818054f87bc19600949531ad0: No bootstrap required, opened a new log
I20260202 02:53:23.972661 4672 ts_tablet_manager.cc:1403] T 7b76f6283a604527946408b2a8983ee5 P a34c77c818054f87bc19600949531ad0: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260202 02:53:23.973248 4672 raft_consensus.cc:359] T 7b76f6283a604527946408b2a8983ee5 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:23.973405 4672 raft_consensus.cc:385] T 7b76f6283a604527946408b2a8983ee5 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:23.973462 4672 raft_consensus.cc:740] T 7b76f6283a604527946408b2a8983ee5 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a34c77c818054f87bc19600949531ad0, State: Initialized, Role: FOLLOWER
@ 0x7f493bbf27f4 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260202 02:53:23.973673 4672 consensus_queue.cc:260] T 7b76f6283a604527946408b2a8983ee5 P a34c77c818054f87bc19600949531ad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:23.974157 4672 ts_tablet_manager.cc:1434] T 7b76f6283a604527946408b2a8983ee5 P a34c77c818054f87bc19600949531ad0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260202 02:53:23.976796 4676 tablet_bootstrap.cc:654] T c58cc255392e4a74b94c1cfae918c70e P d9570a47043b4891b2392af17f405e3d: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:23.977475 4674 ts_tablet_manager.cc:1434] T aa58249c51134388b92c95408bd8c923 P 68f130042e0c4b8292b138ff8a322517: Time spent starting tablet: real 0.037s user 0.004s sys 0.008s
I20260202 02:53:23.977851 4674 tablet_bootstrap.cc:492] T 7b76f6283a604527946408b2a8983ee5 P 68f130042e0c4b8292b138ff8a322517: Bootstrap starting.
I20260202 02:53:23.981099 4674 tablet_bootstrap.cc:654] T 7b76f6283a604527946408b2a8983ee5 P 68f130042e0c4b8292b138ff8a322517: Neither blocks nor log segments found. Creating new log.
@ 0x7f493bbf10a1 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260202 02:53:23.990815 4676 tablet_bootstrap.cc:492] T c58cc255392e4a74b94c1cfae918c70e P d9570a47043b4891b2392af17f405e3d: No bootstrap required, opened a new log
I20260202 02:53:23.991021 4676 ts_tablet_manager.cc:1403] T c58cc255392e4a74b94c1cfae918c70e P d9570a47043b4891b2392af17f405e3d: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.000s
I20260202 02:53:23.991393 4674 tablet_bootstrap.cc:492] T 7b76f6283a604527946408b2a8983ee5 P 68f130042e0c4b8292b138ff8a322517: No bootstrap required, opened a new log
I20260202 02:53:23.991576 4674 ts_tablet_manager.cc:1403] T 7b76f6283a604527946408b2a8983ee5 P 68f130042e0c4b8292b138ff8a322517: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.000s
I20260202 02:53:23.991897 4676 raft_consensus.cc:359] T c58cc255392e4a74b94c1cfae918c70e P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:23.992120 4676 raft_consensus.cc:385] T c58cc255392e4a74b94c1cfae918c70e P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:23.992204 4676 raft_consensus.cc:740] T c58cc255392e4a74b94c1cfae918c70e P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9570a47043b4891b2392af17f405e3d, State: Initialized, Role: FOLLOWER
I20260202 02:53:23.992378 4674 raft_consensus.cc:359] T 7b76f6283a604527946408b2a8983ee5 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:23.992570 4674 raft_consensus.cc:385] T 7b76f6283a604527946408b2a8983ee5 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:23.992543 4676 consensus_queue.cc:260] T c58cc255392e4a74b94c1cfae918c70e P d9570a47043b4891b2392af17f405e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:23.992658 4674 raft_consensus.cc:740] T 7b76f6283a604527946408b2a8983ee5 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f130042e0c4b8292b138ff8a322517, State: Initialized, Role: FOLLOWER
I20260202 02:53:23.993086 4676 ts_tablet_manager.cc:1434] T c58cc255392e4a74b94c1cfae918c70e P d9570a47043b4891b2392af17f405e3d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260202 02:53:23.993445 4676 tablet_bootstrap.cc:492] T aa58249c51134388b92c95408bd8c923 P d9570a47043b4891b2392af17f405e3d: Bootstrap starting.
I20260202 02:53:23.994367 4674 consensus_queue.cc:260] T 7b76f6283a604527946408b2a8983ee5 P 68f130042e0c4b8292b138ff8a322517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:23.994936 4674 ts_tablet_manager.cc:1434] T 7b76f6283a604527946408b2a8983ee5 P 68f130042e0c4b8292b138ff8a322517: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260202 02:53:23.995246 4674 tablet_bootstrap.cc:492] T c58cc255392e4a74b94c1cfae918c70e P 68f130042e0c4b8292b138ff8a322517: Bootstrap starting.
I20260202 02:53:23.996166 4676 tablet_bootstrap.cc:654] T aa58249c51134388b92c95408bd8c923 P d9570a47043b4891b2392af17f405e3d: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:23.999915 4674 tablet_bootstrap.cc:654] T c58cc255392e4a74b94c1cfae918c70e P 68f130042e0c4b8292b138ff8a322517: Neither blocks nor log segments found. Creating new log.
@ 0x7f493bbf8d1c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260202 02:53:24.001024 4676 tablet_bootstrap.cc:492] T aa58249c51134388b92c95408bd8c923 P d9570a47043b4891b2392af17f405e3d: No bootstrap required, opened a new log
I20260202 02:53:24.001191 4676 ts_tablet_manager.cc:1403] T aa58249c51134388b92c95408bd8c923 P d9570a47043b4891b2392af17f405e3d: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.001s
I20260202 02:53:24.002010 4676 raft_consensus.cc:359] T aa58249c51134388b92c95408bd8c923 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:24.002197 4676 raft_consensus.cc:385] T aa58249c51134388b92c95408bd8c923 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:24.002280 4676 raft_consensus.cc:740] T aa58249c51134388b92c95408bd8c923 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9570a47043b4891b2392af17f405e3d, State: Initialized, Role: FOLLOWER
I20260202 02:53:24.002548 4676 consensus_queue.cc:260] T aa58249c51134388b92c95408bd8c923 P d9570a47043b4891b2392af17f405e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:24.003171 4676 ts_tablet_manager.cc:1434] T aa58249c51134388b92c95408bd8c923 P d9570a47043b4891b2392af17f405e3d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
@ 0x7f493bbf8b99 std::_Function_handler<>::_M_invoke() at ??:0
@ 0xa8970d std::function<>::operator()() at ??:?
I20260202 02:53:24.012585 4674 tablet_bootstrap.cc:492] T c58cc255392e4a74b94c1cfae918c70e P 68f130042e0c4b8292b138ff8a322517: No bootstrap required, opened a new log
I20260202 02:53:24.012727 4674 ts_tablet_manager.cc:1403] T c58cc255392e4a74b94c1cfae918c70e P 68f130042e0c4b8292b138ff8a322517: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.002s
I20260202 02:53:24.013504 4674 raft_consensus.cc:359] T c58cc255392e4a74b94c1cfae918c70e P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:24.013716 4674 raft_consensus.cc:385] T c58cc255392e4a74b94c1cfae918c70e P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:24.013808 4674 raft_consensus.cc:740] T c58cc255392e4a74b94c1cfae918c70e P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f130042e0c4b8292b138ff8a322517, State: Initialized, Role: FOLLOWER
I20260202 02:53:24.014030 4674 consensus_queue.cc:260] T c58cc255392e4a74b94c1cfae918c70e P 68f130042e0c4b8292b138ff8a322517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:24.014595 4674 ts_tablet_manager.cc:1434] T c58cc255392e4a74b94c1cfae918c70e P 68f130042e0c4b8292b138ff8a322517: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
@ 0x7f4927f70f99 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f492b57d6db start_thread at ??:0
@ 0x7f49240ac71f clone at ??:0
I20260202 02:53:24.079039 4659 raft_consensus.cc:493] T aa58249c51134388b92c95408bd8c923 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:24.079267 4659 raft_consensus.cc:515] T aa58249c51134388b92c95408bd8c923 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:24.080068 4659 leader_election.cc:290] T aa58249c51134388b92c95408bd8c923 P d9570a47043b4891b2392af17f405e3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661), a34c77c818054f87bc19600949531ad0 (127.0.104.130:38525)
I20260202 02:53:24.080725 4187 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "aa58249c51134388b92c95408bd8c923" candidate_uuid: "d9570a47043b4891b2392af17f405e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f130042e0c4b8292b138ff8a322517" is_pre_election: true
I20260202 02:53:24.080797 4051 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "aa58249c51134388b92c95408bd8c923" candidate_uuid: "d9570a47043b4891b2392af17f405e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a34c77c818054f87bc19600949531ad0" is_pre_election: true
I20260202 02:53:24.080978 4187 raft_consensus.cc:2468] T aa58249c51134388b92c95408bd8c923 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9570a47043b4891b2392af17f405e3d in term 0.
I20260202 02:53:24.081019 4051 raft_consensus.cc:2468] T aa58249c51134388b92c95408bd8c923 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9570a47043b4891b2392af17f405e3d in term 0.
I20260202 02:53:24.081624 3840 leader_election.cc:304] T aa58249c51134388b92c95408bd8c923 P d9570a47043b4891b2392af17f405e3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 68f130042e0c4b8292b138ff8a322517, a34c77c818054f87bc19600949531ad0, d9570a47043b4891b2392af17f405e3d; no voters:
I20260202 02:53:24.082007 4659 raft_consensus.cc:2804] T aa58249c51134388b92c95408bd8c923 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:53:24.082152 4659 raft_consensus.cc:493] T aa58249c51134388b92c95408bd8c923 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:53:24.082252 4659 raft_consensus.cc:3060] T aa58249c51134388b92c95408bd8c923 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:24.085314 4659 raft_consensus.cc:515] T aa58249c51134388b92c95408bd8c923 P d9570a47043b4891b2392af17f405e3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:24.086059 4659 leader_election.cc:290] T aa58249c51134388b92c95408bd8c923 P d9570a47043b4891b2392af17f405e3d [CANDIDATE]: Term 1 election: Requested vote from peers 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661), a34c77c818054f87bc19600949531ad0 (127.0.104.130:38525)
I20260202 02:53:24.086678 4051 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "aa58249c51134388b92c95408bd8c923" candidate_uuid: "d9570a47043b4891b2392af17f405e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a34c77c818054f87bc19600949531ad0"
I20260202 02:53:24.086705 4187 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "aa58249c51134388b92c95408bd8c923" candidate_uuid: "d9570a47043b4891b2392af17f405e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f130042e0c4b8292b138ff8a322517"
I20260202 02:53:24.086898 4051 raft_consensus.cc:3060] T aa58249c51134388b92c95408bd8c923 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:24.086899 4187 raft_consensus.cc:3060] T aa58249c51134388b92c95408bd8c923 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:24.088860 4051 raft_consensus.cc:2468] T aa58249c51134388b92c95408bd8c923 P a34c77c818054f87bc19600949531ad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9570a47043b4891b2392af17f405e3d in term 1.
I20260202 02:53:24.089351 3841 leader_election.cc:304] T aa58249c51134388b92c95408bd8c923 P d9570a47043b4891b2392af17f405e3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c77c818054f87bc19600949531ad0, d9570a47043b4891b2392af17f405e3d; no voters:
I20260202 02:53:24.089666 4659 raft_consensus.cc:2804] T aa58249c51134388b92c95408bd8c923 P d9570a47043b4891b2392af17f405e3d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:53:24.089808 4187 raft_consensus.cc:2468] T aa58249c51134388b92c95408bd8c923 P 68f130042e0c4b8292b138ff8a322517 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9570a47043b4891b2392af17f405e3d in term 1.
I20260202 02:53:24.090119 4659 raft_consensus.cc:697] T aa58249c51134388b92c95408bd8c923 P d9570a47043b4891b2392af17f405e3d [term 1 LEADER]: Becoming Leader. State: Replica: d9570a47043b4891b2392af17f405e3d, State: Running, Role: LEADER
I20260202 02:53:24.091058 4659 consensus_queue.cc:237] T aa58249c51134388b92c95408bd8c923 P d9570a47043b4891b2392af17f405e3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
W20260202 02:53:24.195271 4242 connection.cc:537] client connection to 127.0.104.190:34577 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260202 02:53:24.195998 4242 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.104.190:34577)
W20260202 02:53:24.196573 4242 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.104.190:34577)
W20260202 02:53:24.196756 4242 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.104.190:34577)
I20260202 02:53:24.341885 4659 raft_consensus.cc:493] T c58cc255392e4a74b94c1cfae918c70e P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:24.342118 4659 raft_consensus.cc:515] T c58cc255392e4a74b94c1cfae918c70e P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:24.342783 4659 leader_election.cc:290] T c58cc255392e4a74b94c1cfae918c70e P d9570a47043b4891b2392af17f405e3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661), a34c77c818054f87bc19600949531ad0 (127.0.104.130:38525)
I20260202 02:53:24.343390 4187 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c58cc255392e4a74b94c1cfae918c70e" candidate_uuid: "d9570a47043b4891b2392af17f405e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f130042e0c4b8292b138ff8a322517" is_pre_election: true
I20260202 02:53:24.343407 4051 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c58cc255392e4a74b94c1cfae918c70e" candidate_uuid: "d9570a47043b4891b2392af17f405e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a34c77c818054f87bc19600949531ad0" is_pre_election: true
I20260202 02:53:24.343621 4051 raft_consensus.cc:2468] T c58cc255392e4a74b94c1cfae918c70e P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9570a47043b4891b2392af17f405e3d in term 0.
I20260202 02:53:24.343616 4187 raft_consensus.cc:2468] T c58cc255392e4a74b94c1cfae918c70e P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9570a47043b4891b2392af17f405e3d in term 0.
I20260202 02:53:24.344060 3841 leader_election.cc:304] T c58cc255392e4a74b94c1cfae918c70e P d9570a47043b4891b2392af17f405e3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 68f130042e0c4b8292b138ff8a322517, a34c77c818054f87bc19600949531ad0, d9570a47043b4891b2392af17f405e3d; no voters:
I20260202 02:53:24.344447 4659 raft_consensus.cc:2804] T c58cc255392e4a74b94c1cfae918c70e P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:53:24.344547 4659 raft_consensus.cc:493] T c58cc255392e4a74b94c1cfae918c70e P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:53:24.344651 4659 raft_consensus.cc:3060] T c58cc255392e4a74b94c1cfae918c70e P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:24.347888 4659 raft_consensus.cc:515] T c58cc255392e4a74b94c1cfae918c70e P d9570a47043b4891b2392af17f405e3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:24.348665 4659 leader_election.cc:290] T c58cc255392e4a74b94c1cfae918c70e P d9570a47043b4891b2392af17f405e3d [CANDIDATE]: Term 1 election: Requested vote from peers 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661), a34c77c818054f87bc19600949531ad0 (127.0.104.130:38525)
I20260202 02:53:24.349306 4051 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c58cc255392e4a74b94c1cfae918c70e" candidate_uuid: "d9570a47043b4891b2392af17f405e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a34c77c818054f87bc19600949531ad0"
I20260202 02:53:24.349280 4187 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c58cc255392e4a74b94c1cfae918c70e" candidate_uuid: "d9570a47043b4891b2392af17f405e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f130042e0c4b8292b138ff8a322517"
I20260202 02:53:24.349546 4051 raft_consensus.cc:3060] T c58cc255392e4a74b94c1cfae918c70e P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:24.349558 4187 raft_consensus.cc:3060] T c58cc255392e4a74b94c1cfae918c70e P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:24.352473 4187 raft_consensus.cc:2468] T c58cc255392e4a74b94c1cfae918c70e P 68f130042e0c4b8292b138ff8a322517 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9570a47043b4891b2392af17f405e3d in term 1.
I20260202 02:53:24.352494 4051 raft_consensus.cc:2468] T c58cc255392e4a74b94c1cfae918c70e P a34c77c818054f87bc19600949531ad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9570a47043b4891b2392af17f405e3d in term 1.
I20260202 02:53:24.353138 3840 leader_election.cc:304] T c58cc255392e4a74b94c1cfae918c70e P d9570a47043b4891b2392af17f405e3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 68f130042e0c4b8292b138ff8a322517, a34c77c818054f87bc19600949531ad0, d9570a47043b4891b2392af17f405e3d; no voters:
I20260202 02:53:24.353503 4659 raft_consensus.cc:2804] T c58cc255392e4a74b94c1cfae918c70e P d9570a47043b4891b2392af17f405e3d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:53:24.353677 4659 raft_consensus.cc:697] T c58cc255392e4a74b94c1cfae918c70e P d9570a47043b4891b2392af17f405e3d [term 1 LEADER]: Becoming Leader. State: Replica: d9570a47043b4891b2392af17f405e3d, State: Running, Role: LEADER
I20260202 02:53:24.354001 4659 consensus_queue.cc:237] T c58cc255392e4a74b94c1cfae918c70e P d9570a47043b4891b2392af17f405e3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:24.425697 4680 raft_consensus.cc:493] T 7b76f6283a604527946408b2a8983ee5 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:24.425908 4680 raft_consensus.cc:515] T 7b76f6283a604527946408b2a8983ee5 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:24.426549 4680 leader_election.cc:290] T 7b76f6283a604527946408b2a8983ee5 P 68f130042e0c4b8292b138ff8a322517 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425), a34c77c818054f87bc19600949531ad0 (127.0.104.130:38525)
I20260202 02:53:24.427143 3909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7b76f6283a604527946408b2a8983ee5" candidate_uuid: "68f130042e0c4b8292b138ff8a322517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9570a47043b4891b2392af17f405e3d" is_pre_election: true
I20260202 02:53:24.427177 4051 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7b76f6283a604527946408b2a8983ee5" candidate_uuid: "68f130042e0c4b8292b138ff8a322517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a34c77c818054f87bc19600949531ad0" is_pre_election: true
I20260202 02:53:24.427372 3909 raft_consensus.cc:2468] T 7b76f6283a604527946408b2a8983ee5 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68f130042e0c4b8292b138ff8a322517 in term 0.
I20260202 02:53:24.427433 4051 raft_consensus.cc:2468] T 7b76f6283a604527946408b2a8983ee5 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68f130042e0c4b8292b138ff8a322517 in term 0.
I20260202 02:53:24.427911 4121 leader_election.cc:304] T 7b76f6283a604527946408b2a8983ee5 P 68f130042e0c4b8292b138ff8a322517 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68f130042e0c4b8292b138ff8a322517, d9570a47043b4891b2392af17f405e3d; no voters:
I20260202 02:53:24.428185 4680 raft_consensus.cc:2804] T 7b76f6283a604527946408b2a8983ee5 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:53:24.428297 4680 raft_consensus.cc:493] T 7b76f6283a604527946408b2a8983ee5 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:53:24.428442 4680 raft_consensus.cc:3060] T 7b76f6283a604527946408b2a8983ee5 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:24.430423 4680 raft_consensus.cc:515] T 7b76f6283a604527946408b2a8983ee5 P 68f130042e0c4b8292b138ff8a322517 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:24.431056 4680 leader_election.cc:290] T 7b76f6283a604527946408b2a8983ee5 P 68f130042e0c4b8292b138ff8a322517 [CANDIDATE]: Term 1 election: Requested vote from peers d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425), a34c77c818054f87bc19600949531ad0 (127.0.104.130:38525)
I20260202 02:53:24.431744 4051 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7b76f6283a604527946408b2a8983ee5" candidate_uuid: "68f130042e0c4b8292b138ff8a322517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a34c77c818054f87bc19600949531ad0"
I20260202 02:53:24.431777 3909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7b76f6283a604527946408b2a8983ee5" candidate_uuid: "68f130042e0c4b8292b138ff8a322517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9570a47043b4891b2392af17f405e3d"
I20260202 02:53:24.431978 3909 raft_consensus.cc:3060] T 7b76f6283a604527946408b2a8983ee5 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:24.432005 4051 raft_consensus.cc:3060] T 7b76f6283a604527946408b2a8983ee5 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:24.434758 4051 raft_consensus.cc:2468] T 7b76f6283a604527946408b2a8983ee5 P a34c77c818054f87bc19600949531ad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68f130042e0c4b8292b138ff8a322517 in term 1.
I20260202 02:53:24.434770 3909 raft_consensus.cc:2468] T 7b76f6283a604527946408b2a8983ee5 P d9570a47043b4891b2392af17f405e3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68f130042e0c4b8292b138ff8a322517 in term 1.
I20260202 02:53:24.435348 4121 leader_election.cc:304] T 7b76f6283a604527946408b2a8983ee5 P 68f130042e0c4b8292b138ff8a322517 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68f130042e0c4b8292b138ff8a322517, d9570a47043b4891b2392af17f405e3d; no voters:
I20260202 02:53:24.435765 4680 raft_consensus.cc:2804] T 7b76f6283a604527946408b2a8983ee5 P 68f130042e0c4b8292b138ff8a322517 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:53:24.436187 4680 raft_consensus.cc:697] T 7b76f6283a604527946408b2a8983ee5 P 68f130042e0c4b8292b138ff8a322517 [term 1 LEADER]: Becoming Leader. State: Replica: 68f130042e0c4b8292b138ff8a322517, State: Running, Role: LEADER
I20260202 02:53:24.436564 4680 consensus_queue.cc:237] T 7b76f6283a604527946408b2a8983ee5 P 68f130042e0c4b8292b138ff8a322517 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:24.454955 4704 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 666b7e52b5fc4f3397f04960c8b67100)
I20260202 02:53:24.455183 4704 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:24.455871 4704 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:53:24.458243 4380 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:53:24.458496 4380 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:24.458724 4380 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:24.458992 4704 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260202 02:53:24.887377 4338 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:53:24.887604 4338 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:53:24.944499 4659 consensus_queue.cc:1048] T aa58249c51134388b92c95408bd8c923 P d9570a47043b4891b2392af17f405e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:53:24.950515 4659 consensus_queue.cc:1048] T aa58249c51134388b92c95408bd8c923 P d9570a47043b4891b2392af17f405e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:53:25.169400 4719 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 666b7e52b5fc4f3397f04960c8b67100)
I20260202 02:53:25.169642 4719 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:25.170320 4719 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:53:25.172585 4380 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:53:25.172812 4380 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:25.172920 4380 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:25.173195 4719 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260202 02:53:25.331048 4659 consensus_queue.cc:1048] T c58cc255392e4a74b94c1cfae918c70e P d9570a47043b4891b2392af17f405e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:53:25.331621 4712 consensus_queue.cc:1048] T 7b76f6283a604527946408b2a8983ee5 P 68f130042e0c4b8292b138ff8a322517 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:53:25.339630 4714 consensus_queue.cc:1048] T c58cc255392e4a74b94c1cfae918c70e P d9570a47043b4891b2392af17f405e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:53:25.342729 4726 consensus_queue.cc:1048] T 7b76f6283a604527946408b2a8983ee5 P 68f130042e0c4b8292b138ff8a322517 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:53:25.573679 418 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskYXxklX/build/asan/bin/kudu pid 4338
I20260202 02:53:25.585767 4423 generic_service.cc:196] Checking for leaks (request via RPC)
I20260202 02:53:25.705652 418 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskYXxklX/build/asan/bin/kudu with pid 4338
I20260202 02:53:26.319589 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskYXxklX/build/asan/bin/kudu
/tmp/dist-test-taskYXxklX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.189:40361
--webserver_interface=127.0.104.189
--webserver_port=35647
--builtin_ntp_servers=127.0.104.148:34531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525
--hive_metastore_uris=thrift://127.0.0.1:46545
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260202 02:53:26.674618 4745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260202 02:53:26.674962 4745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260202 02:53:26.675043 4745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260202 02:53:26.683720 4745 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260202 02:53:26.683821 4745 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260202 02:53:26.683878 4745 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260202 02:53:26.683916 4745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260202 02:53:26.683976 4745 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260202 02:53:26.684031 4745 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260202 02:53:26.694833 4745 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:34531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:46545
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.189:40361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.104.189
--webserver_port=35647
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3722ed49c19ce54c7feae2fa944e32b1d4ed8d2d
build type FASTDEBUG
built by None at 02 Feb 2026 02:43:23 UTC on 24a791456cd2
build id 10304
ASAN enabled
I20260202 02:53:26.696555 4745 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260202 02:53:26.698378 4745 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260202 02:53:26.707378 4751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:53:26.707908 4750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:53:26.708176 4753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260202 02:53:26.708957 4745 server_base.cc:1060] running on GCE node
I20260202 02:53:26.709825 4745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260202 02:53:26.711460 4745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260202 02:53:26.712723 4745 hybrid_clock.cc:648] HybridClock initialized: now 1770000806712686 us; error 42 us; skew 500 ppm
I20260202 02:53:26.713157 4745 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260202 02:53:26.715989 4745 webserver.cc:492] Webserver started at http://127.0.104.189:35647/ using document root <none> and password file <none>
I20260202 02:53:26.717010 4745 fs_manager.cc:362] Metadata directory not provided
I20260202 02:53:26.717134 4745 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260202 02:53:26.723506 4745 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260202 02:53:26.726969 4759 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260202 02:53:26.728777 4745 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260202 02:53:26.729004 4745 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data,/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/wal
uuid: "3acf71c31a284578b4f8c918eaa50205"
format_stamp: "Formatted at 2026-02-02 02:53:15 on dist-test-slave-llj2"
I20260202 02:53:26.729820 4745 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260202 02:53:26.746443 4745 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260202 02:53:26.747356 4745 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260202 02:53:26.747645 4745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260202 02:53:26.768545 4745 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.189:40361
I20260202 02:53:26.768605 4811 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.189:40361 every 8 connection(s)
I20260202 02:53:26.770198 4745 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data/info.pb
I20260202 02:53:26.775731 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskYXxklX/build/asan/bin/kudu as pid 4745
I20260202 02:53:26.778728 4812 sys_catalog.cc:263] Verifying existing consensus state
I20260202 02:53:26.782799 4812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: Bootstrap starting.
I20260202 02:53:26.795406 4812 log.cc:826] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: Log is configured to *not* fsync() on all Append() calls
I20260202 02:53:26.822039 4234 heartbeater.cc:344] Connected to a master server at 127.0.104.189:40361
I20260202 02:53:26.836215 3956 heartbeater.cc:344] Connected to a master server at 127.0.104.189:40361
I20260202 02:53:26.842094 4098 heartbeater.cc:344] Connected to a master server at 127.0.104.189:40361
I20260202 02:53:26.861284 4812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: Bootstrap replayed 1/1 log segments. Stats: ops{read=31 overwritten=0 applied=31 ignored=0} inserts{seen=18 ignored=0} mutations{seen=22 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260202 02:53:26.862042 4812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: Bootstrap complete.
I20260202 02:53:26.870345 4812 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 5 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:26.871447 4812 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3acf71c31a284578b4f8c918eaa50205, State: Initialized, Role: FOLLOWER
I20260202 02:53:26.872641 4812 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 31, Last appended: 5.31, Last appended by leader: 31, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:26.873709 4823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:26.874010 4823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:53:26.875278 4812 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: configured and running, proceeding with master startup.
I20260202 02:53:26.925769 4837 catalog_manager.cc:1269] Loaded cluster ID: 0d95b889b53247a0813a4fda5550ae30
I20260202 02:53:26.925897 4837 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: loading cluster ID for follower catalog manager: success
I20260202 02:53:26.930986 4837 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: acquiring CA information for follower catalog manager: success
I20260202 02:53:26.926348 4745 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:53:26.932539 4745 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:53:26.934301 4837 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260202 02:53:26.950906 4775 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "a34c77c818054f87bc19600949531ad0" instance_seqno: 1770000797481684) as {username='slave'} at 127.0.104.130:54217; Asking this server to re-register.
I20260202 02:53:26.951723 4098 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:26.953706 4775 ts_manager.cc:194] Registered new tserver with Master: a34c77c818054f87bc19600949531ad0 (127.0.104.130:38525)
I20260202 02:53:27.341035 4775 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "d9570a47043b4891b2392af17f405e3d" instance_seqno: 1770000796958858) as {username='slave'} at 127.0.104.129:45891; Asking this server to re-register.
I20260202 02:53:27.342054 3956 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:27.343319 4775 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "68f130042e0c4b8292b138ff8a322517" instance_seqno: 1770000797970665) as {username='slave'} at 127.0.104.131:58471; Asking this server to re-register.
I20260202 02:53:27.343860 4776 ts_manager.cc:194] Registered new tserver with Master: d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425)
I20260202 02:53:27.344017 4234 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:27.345327 4776 ts_manager.cc:194] Registered new tserver with Master: 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661)
I20260202 02:53:27.450606 4844 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:27.450860 4844 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:27.452733 4844 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:53:27.453915 4763 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:53:27.457039 4763 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:53:27.457331 4763 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:27.457433 4763 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:27.457792 4844 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260202 02:53:28.116518 4851 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:28.116701 4851 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:28.117451 4851 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:53:28.119824 4763 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:53:28.120033 4763 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:28.120144 4763 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:28.120476 4851 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260202 02:53:29.004251 4863 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:29.004467 4863 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:29.005143 4863 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:53:29.007809 4763 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:53:29.008041 4763 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:29.008143 4763 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:29.008483 4863 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260202 02:53:29.798393 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskYXxklX/build/asan/bin/kudu
/tmp/dist-test-taskYXxklX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.188:42525
--webserver_interface=127.0.104.188
--webserver_port=42683
--builtin_ntp_servers=127.0.104.148:34531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525
--hive_metastore_uris=thrift://127.0.0.1:46545
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260202 02:53:29.879663 4878 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:29.879871 4878 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:29.880844 4878 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:53:29.883356 4763 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:53:29.883595 4763 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:29.883710 4763 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:29.883980 4878 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260202 02:53:30.167728 4875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260202 02:53:30.168112 4875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260202 02:53:30.168231 4875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260202 02:53:30.177764 4875 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260202 02:53:30.177888 4875 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260202 02:53:30.177966 4875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260202 02:53:30.178028 4875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260202 02:53:30.178114 4875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260202 02:53:30.178190 4875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260202 02:53:30.189975 4875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:34531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:46545
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.188:42525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.104.188
--webserver_port=42683
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3722ed49c19ce54c7feae2fa944e32b1d4ed8d2d
build type FASTDEBUG
built by None at 02 Feb 2026 02:43:23 UTC on 24a791456cd2
build id 10304
ASAN enabled
I20260202 02:53:30.191843 4875 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260202 02:53:30.194113 4875 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260202 02:53:30.203595 4886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:53:30.204033 4885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:53:30.204602 4888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260202 02:53:30.204999 4875 server_base.cc:1060] running on GCE node
I20260202 02:53:30.205951 4875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260202 02:53:30.207685 4875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260202 02:53:30.208926 4875 hybrid_clock.cc:648] HybridClock initialized: now 1770000810208866 us; error 67 us; skew 500 ppm
I20260202 02:53:30.209338 4875 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260202 02:53:30.211899 4875 webserver.cc:492] Webserver started at http://127.0.104.188:42683/ using document root <none> and password file <none>
I20260202 02:53:30.212833 4875 fs_manager.cc:362] Metadata directory not provided
I20260202 02:53:30.212975 4875 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260202 02:53:30.219007 4875 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260202 02:53:30.222368 4894 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260202 02:53:30.224026 4875 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260202 02:53:30.224236 4875 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-2/data,/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-2/wal
uuid: "879caa0e6c114ab4855b7a756e252c9e"
format_stamp: "Formatted at 2026-02-02 02:53:15 on dist-test-slave-llj2"
I20260202 02:53:30.225063 4875 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260202 02:53:30.247475 4875 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260202 02:53:30.248365 4875 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260202 02:53:30.248639 4875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260202 02:53:30.269135 4875 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.188:42525
I20260202 02:53:30.269239 4945 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.188:42525 every 8 connection(s)
I20260202 02:53:30.270977 4875 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-2/data/info.pb
I20260202 02:53:30.272631 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskYXxklX/build/asan/bin/kudu as pid 4875
I20260202 02:53:30.283967 4946 sys_catalog.cc:263] Verifying existing consensus state
I20260202 02:53:30.289454 4946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: Bootstrap starting.
I20260202 02:53:30.293808 4096 heartbeater.cc:344] Connected to a master server at 127.0.104.188:42525
I20260202 02:53:30.301667 4946 log.cc:826] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: Log is configured to *not* fsync() on all Append() calls
I20260202 02:53:30.322687 4946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: Bootstrap replayed 1/1 log segments. Stats: ops{read=21 overwritten=0 applied=21 ignored=0} inserts{seen=15 ignored=0} mutations{seen=10 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260202 02:53:30.323221 4946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: Bootstrap complete.
I20260202 02:53:30.328662 4946 raft_consensus.cc:359] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:30.329490 4946 raft_consensus.cc:740] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 879caa0e6c114ab4855b7a756e252c9e, State: Initialized, Role: FOLLOWER
I20260202 02:53:30.330132 4946 consensus_queue.cc:260] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 21, Last appended: 3.21, Last appended by leader: 21, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:30.331053 4953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:30.331377 4953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:53:30.332217 4946 sys_catalog.cc:565] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [sys.catalog]: configured and running, proceeding with master startup.
I20260202 02:53:30.354920 4967 catalog_manager.cc:1269] Loaded cluster ID: 0d95b889b53247a0813a4fda5550ae30
I20260202 02:53:30.355136 4967 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: loading cluster ID for follower catalog manager: success
W20260202 02:53:30.365826 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 6: this message will repeat every 5th retry.
I20260202 02:53:30.364398 4875 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:53:30.369011 4875 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:53:30.369354 4910 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "a34c77c818054f87bc19600949531ad0" instance_seqno: 1770000797481684) as {username='slave'} at 127.0.104.130:36159; Asking this server to re-register.
I20260202 02:53:30.372536 4096 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:30.375602 4909 ts_manager.cc:194] Registered new tserver with Master: a34c77c818054f87bc19600949531ad0 (127.0.104.130:38525)
I20260202 02:53:30.378226 4967 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: acquiring CA information for follower catalog manager: success
I20260202 02:53:30.380132 4232 heartbeater.cc:344] Connected to a master server at 127.0.104.188:42525
I20260202 02:53:30.382617 4909 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "68f130042e0c4b8292b138ff8a322517" instance_seqno: 1770000797970665) as {username='slave'} at 127.0.104.131:44691; Asking this server to re-register.
I20260202 02:53:30.385282 4232 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:30.386075 3954 heartbeater.cc:344] Connected to a master server at 127.0.104.188:42525
I20260202 02:53:30.387172 4909 ts_manager.cc:194] Registered new tserver with Master: 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661)
I20260202 02:53:30.388011 4910 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "d9570a47043b4891b2392af17f405e3d" instance_seqno: 1770000796958858) as {username='slave'} at 127.0.104.129:39539; Asking this server to re-register.
I20260202 02:53:30.388751 3954 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:30.389955 4967 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260202 02:53:30.390429 4910 ts_manager.cc:194] Registered new tserver with Master: d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425)
W20260202 02:53:30.627890 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 6: this message will repeat every 5th retry.
W20260202 02:53:30.655695 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 6: this message will repeat every 5th retry.
I20260202 02:53:30.710965 4953 raft_consensus.cc:493] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:30.711370 4953 raft_consensus.cc:515] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:30.714751 4953 leader_election.cc:290] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361)
W20260202 02:53:30.715261 4898 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:53:30.717631 4898 leader_election.cc:336] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
I20260202 02:53:30.723841 4786 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "879caa0e6c114ab4855b7a756e252c9e" candidate_term: 4 candidate_status { last_received { term: 3 index: 21 } } ignore_live_leader: false dest_uuid: "3acf71c31a284578b4f8c918eaa50205" is_pre_election: true
I20260202 02:53:30.724402 4786 raft_consensus.cc:2368] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 879caa0e6c114ab4855b7a756e252c9e for earlier term 4. Current term is 5.
I20260202 02:53:30.725478 4896 leader_election.cc:400] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 4 pre-election: Vote denied by peer 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361) with higher term. Message: Invalid argument: T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 879caa0e6c114ab4855b7a756e252c9e for earlier term 4. Current term is 5.
I20260202 02:53:30.725593 4896 leader_election.cc:403] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260202 02:53:30.725950 4953 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 3 FOLLOWER]: Advancing to term 5
I20260202 02:53:30.731909 4953 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361) with higher term. Message: Invalid argument: T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 879caa0e6c114ab4855b7a756e252c9e for earlier term 4. Current term is 5.
W20260202 02:53:30.874291 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 6: this message will repeat every 5th retry.
W20260202 02:53:30.998432 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 6: this message will repeat every 5th retry.
W20260202 02:53:31.154400 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 6: this message will repeat every 5th retry.
I20260202 02:53:31.455915 4985 raft_consensus.cc:493] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:31.456110 4985 raft_consensus.cc:515] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:31.456921 4985 leader_election.cc:290] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361)
I20260202 02:53:31.457515 4786 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "879caa0e6c114ab4855b7a756e252c9e" candidate_term: 6 candidate_status { last_received { term: 3 index: 21 } } ignore_live_leader: false dest_uuid: "3acf71c31a284578b4f8c918eaa50205" is_pre_election: true
I20260202 02:53:31.457794 4786 raft_consensus.cc:2410] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 879caa0e6c114ab4855b7a756e252c9e for term 6 because replica has last-logged OpId of term: 5 index: 31, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 21.
W20260202 02:53:31.458952 4898 leader_election.cc:336] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
I20260202 02:53:31.459084 4898 leader_election.cc:304] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 879caa0e6c114ab4855b7a756e252c9e; no voters: 3acf71c31a284578b4f8c918eaa50205, 666b7e52b5fc4f3397f04960c8b67100
I20260202 02:53:31.459448 4985 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260202 02:53:31.803623 4990 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:31.803834 4990 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:31.804744 4990 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:53:31.808858 4763 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
I20260202 02:53:31.812953 4921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3acf71c31a284578b4f8c918eaa50205" candidate_term: 6 candidate_status { last_received { term: 5 index: 31 } } ignore_live_leader: false dest_uuid: "879caa0e6c114ab4855b7a756e252c9e" is_pre_election: true
I20260202 02:53:31.813457 4921 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3acf71c31a284578b4f8c918eaa50205 in term 5.
I20260202 02:53:31.814210 4763 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205, 879caa0e6c114ab4855b7a756e252c9e; no voters: 666b7e52b5fc4f3397f04960c8b67100
I20260202 02:53:31.814596 4990 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260202 02:53:31.814735 4990 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:53:31.814834 4990 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 5 FOLLOWER]: Advancing to term 6
I20260202 02:53:31.818670 4990 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:31.819361 4990 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 6 election: Requested vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
I20260202 02:53:31.820432 4921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3acf71c31a284578b4f8c918eaa50205" candidate_term: 6 candidate_status { last_received { term: 5 index: 31 } } ignore_live_leader: false dest_uuid: "879caa0e6c114ab4855b7a756e252c9e"
I20260202 02:53:31.820649 4921 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 5 FOLLOWER]: Advancing to term 6
W20260202 02:53:31.820978 4763 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
I20260202 02:53:31.822937 4921 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3acf71c31a284578b4f8c918eaa50205 in term 6.
I20260202 02:53:31.823539 4763 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205, 879caa0e6c114ab4855b7a756e252c9e; no voters: 666b7e52b5fc4f3397f04960c8b67100
I20260202 02:53:31.823920 4990 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 6 FOLLOWER]: Leader election won for term 6
I20260202 02:53:31.824512 4990 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 6 LEADER]: Becoming Leader. State: Replica: 3acf71c31a284578b4f8c918eaa50205, State: Running, Role: LEADER
I20260202 02:53:31.825008 4990 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 5.31, Last appended by leader: 31, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:31.827905 4995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3acf71c31a284578b4f8c918eaa50205. Latest consensus state: current_term: 6 leader_uuid: "3acf71c31a284578b4f8c918eaa50205" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:31.828250 4995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: This master's current role is: LEADER
I20260202 02:53:31.829022 4997 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260202 02:53:31.831811 4997 catalog_manager.cc:679] Loaded metadata for table default.table_6a23cf9a48824f7e8d243c355f458ce8 [id=0048dd71261d4204ac5548c190957aa9]
I20260202 02:53:31.832267 4997 catalog_manager.cc:679] Loaded metadata for table default.table_8d32aa2872ab405fa4f2e7b1dac33f56 [id=2ab6a7a17803446898f275ae3faa032c]
I20260202 02:53:31.832671 4997 catalog_manager.cc:679] Loaded metadata for table default.table_2c330ab9e5984ba5980e7264050e7a41 [id=3852a850379342c69688c71b95873308]
I20260202 02:53:31.833048 4997 catalog_manager.cc:679] Loaded metadata for table default.table_4e64388f537d44aaa6d541b90e924c65 [id=4f41a1a5a6844fbfb43b6798b2c6f67d]
I20260202 02:53:31.833595 4997 catalog_manager.cc:679] Loaded metadata for table default.table_470455fd011c452b9cceafbc413b1cfe [id=e28252bb319446c39056986e8b98de30]
I20260202 02:53:31.835709 4997 tablet_loader.cc:96] loaded metadata for tablet 7b76f6283a604527946408b2a8983ee5 (table default.table_2c330ab9e5984ba5980e7264050e7a41 [id=3852a850379342c69688c71b95873308])
I20260202 02:53:31.836093 4997 tablet_loader.cc:96] loaded metadata for tablet 906f9148074848da8456b204a587fa8a (table default.table_8d32aa2872ab405fa4f2e7b1dac33f56 [id=2ab6a7a17803446898f275ae3faa032c])
I20260202 02:53:31.836416 4997 tablet_loader.cc:96] loaded metadata for tablet aa58249c51134388b92c95408bd8c923 (table default.table_6a23cf9a48824f7e8d243c355f458ce8 [id=0048dd71261d4204ac5548c190957aa9])
I20260202 02:53:31.836709 4997 tablet_loader.cc:96] loaded metadata for tablet b5a121d55d9d498ba76bb4dcb24d0e20 (table default.table_470455fd011c452b9cceafbc413b1cfe [id=e28252bb319446c39056986e8b98de30])
I20260202 02:53:31.836958 4997 tablet_loader.cc:96] loaded metadata for tablet c58cc255392e4a74b94c1cfae918c70e (table default.table_4e64388f537d44aaa6d541b90e924c65 [id=4f41a1a5a6844fbfb43b6798b2c6f67d])
I20260202 02:53:31.837204 4997 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260202 02:53:31.838259 4997 catalog_manager.cc:1269] Loaded cluster ID: 0d95b889b53247a0813a4fda5550ae30
I20260202 02:53:31.838346 4997 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260202 02:53:31.839643 4997 catalog_manager.cc:1514] Loading token signing keys...
I20260202 02:53:31.840652 4997 catalog_manager.cc:6038] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: Loaded TSK: 0
I20260202 02:53:31.841622 4997 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260202 02:53:31.842547 4997 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260202 02:53:31.843389 4997 catalog_manager.cc:5876] Last processed Hive Metastore notification event ID: -1
W20260202 02:53:31.887951 4776 catalog_manager.cc:5487] Ignoring report from unknown tablet 76c092d3e26c4906964e0f852ac24448
F20260202 02:53:31.894367 4836 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 7 received events: NotificationEvent(eventId=1, eventTime=1770000798, eventType=CREATE_TABLE, dbName=default, tableName=table_4e64388f537d44aaa6d541b90e924c65, message={"server":"","servicePrincipal":"","db":"default","table":"table_4e64388f537d44aaa6d541b90e924c65","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_4e64388f537d44aaa6d541b90e924c65\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770000798},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/warehouse/table_4e64388f537d44aaa6d541b90e924c65\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_4e64388f537d44aaa6d541b90e924c65\",\"kudu.table_id\":\"4f41a1a5a6844fbfb43b6798b2c6f67d\",\"transient_lastDdlTime\":\"1770000798\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"0d95b889b53247a0813a4fda5550ae30\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770000798,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1770000798, eventType=CREATE_TABLE, dbName=default, tableName=table_a728b4dd173b4f0c8c6b9cce0bcda94d, message={"server":"","servicePrincipal":"","db":"default","table":"table_a728b4dd173b4f0c8c6b9cce0bcda94d","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_a728b4dd173b4f0c8c6b9cce0bcda94d\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770000798},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/warehouse/table_a728b4dd173b4f0c8c6b9cce0bcda94d\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_a728b4dd173b4f0c8c6b9cce0bcda94d\",\"kudu.table_id\":\"a54f76c7d1ff4769b2d84a5fe325d81c\",\"transient_lastDdlTime\":\"1770000798\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"0d95b889b53247a0813a4fda5550ae30\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770000798,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7f8c2efbcdcd google::LogMessage::Fail() at ??:0
@ 0x7f8c2efc0b93 google::LogMessage::SendToLog() at ??:0
@ 0x7f8c2efbc7cc google::LogMessage::Flush() at ??:0
@ 0x7f8c2efbca11 google::LogMessage::~LogMessage() at ??:0
@ 0x7f8c459b37f4 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f8c459b20a1 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f8c459b9d1c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f8c459b9b99 std::_Function_handler<>::_M_invoke() at ??:0
@ 0xa8970d std::function<>::operator()() at ??:?
@ 0x7f8c31d31f99 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f8c3533e6db start_thread at ??:0
@ 0x7f8c2de6d71f clone at ??:0
I20260202 02:53:31.938710 4098 heartbeater.cc:499] Master 127.0.104.189:40361 was elected leader, sending a full tablet report...
W20260202 02:53:32.100857 4240 connection.cc:537] client connection to 127.0.104.189:40361 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260202 02:53:32.100880 3985 connection.cc:537] client connection to 127.0.104.189:40361 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260202 02:53:32.352250 4234 heartbeater.cc:499] Master 127.0.104.189:40361 was elected leader, sending a full tablet report...
I20260202 02:53:32.365482 3956 heartbeater.cc:499] Master 127.0.104.189:40361 was elected leader, sending a full tablet report...
I20260202 02:53:32.475391 5014 raft_consensus.cc:493] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:32.475622 5014 raft_consensus.cc:515] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:32.476356 5014 leader_election.cc:290] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361)
W20260202 02:53:32.479497 4898 leader_election.cc:336] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:53:32.479699 4896 leader_election.cc:336] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361): Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
I20260202 02:53:32.479806 4896 leader_election.cc:304] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 879caa0e6c114ab4855b7a756e252c9e; no voters: 3acf71c31a284578b4f8c918eaa50205, 666b7e52b5fc4f3397f04960c8b67100
I20260202 02:53:32.480058 5014 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260202 02:53:33.413676 5024 raft_consensus.cc:493] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:33.413879 5024 raft_consensus.cc:515] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:33.414573 5024 leader_election.cc:290] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361)
W20260202 02:53:33.416639 4898 leader_election.cc:336] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:53:33.416863 4896 leader_election.cc:336] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361): Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
I20260202 02:53:33.416978 4896 leader_election.cc:304] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 879caa0e6c114ab4855b7a756e252c9e; no voters: 3acf71c31a284578b4f8c918eaa50205, 666b7e52b5fc4f3397f04960c8b67100
I20260202 02:53:33.417228 5024 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260202 02:53:33.581805 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskYXxklX/build/asan/bin/kudu
/tmp/dist-test-taskYXxklX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.190:34577
--webserver_interface=127.0.104.190
--webserver_port=33649
--builtin_ntp_servers=127.0.104.148:34531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525
--hive_metastore_uris=thrift://127.0.0.1:46545
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260202 02:53:33.936759 5028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260202 02:53:33.937117 5028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260202 02:53:33.937228 5028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260202 02:53:33.945974 5028 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260202 02:53:33.946090 5028 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260202 02:53:33.946167 5028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260202 02:53:33.946233 5028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260202 02:53:33.946316 5028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260202 02:53:33.946385 5028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260202 02:53:33.957105 5028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:34531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:46545
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.190:34577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.104.190
--webserver_port=33649
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3722ed49c19ce54c7feae2fa944e32b1d4ed8d2d
build type FASTDEBUG
built by None at 02 Feb 2026 02:43:23 UTC on 24a791456cd2
build id 10304
ASAN enabled
I20260202 02:53:33.958866 5028 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260202 02:53:33.960886 5028 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260202 02:53:33.975805 5037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:53:33.975860 5038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:53:33.976899 5040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260202 02:53:33.977778 5028 server_base.cc:1060] running on GCE node
I20260202 02:53:33.978684 5028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260202 02:53:33.980314 5028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260202 02:53:33.981606 5028 hybrid_clock.cc:648] HybridClock initialized: now 1770000813981541 us; error 51 us; skew 500 ppm
I20260202 02:53:33.982002 5028 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260202 02:53:33.985258 5028 webserver.cc:492] Webserver started at http://127.0.104.190:33649/ using document root <none> and password file <none>
I20260202 02:53:33.986409 5028 fs_manager.cc:362] Metadata directory not provided
I20260202 02:53:33.986567 5028 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260202 02:53:33.992825 5028 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260202 02:53:33.995959 5047 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260202 02:53:33.997664 5028 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260202 02:53:33.997860 5028 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data,/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/wal
uuid: "666b7e52b5fc4f3397f04960c8b67100"
format_stamp: "Formatted at 2026-02-02 02:53:14 on dist-test-slave-llj2"
I20260202 02:53:33.998615 5028 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260202 02:53:34.020520 5028 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260202 02:53:34.021464 5028 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260202 02:53:34.021749 5028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260202 02:53:34.042115 5028 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.190:34577
I20260202 02:53:34.042146 5098 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.190:34577 every 8 connection(s)
I20260202 02:53:34.043820 5028 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data/info.pb
I20260202 02:53:34.045538 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskYXxklX/build/asan/bin/kudu as pid 5028
I20260202 02:53:34.052361 5099 sys_catalog.cc:263] Verifying existing consensus state
I20260202 02:53:34.056715 5099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: Bootstrap starting.
I20260202 02:53:34.068921 5099 log.cc:826] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: Log is configured to *not* fsync() on all Append() calls
I20260202 02:53:34.097612 5099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: Bootstrap replayed 1/1 log segments. Stats: ops{read=31 overwritten=0 applied=31 ignored=0} inserts{seen=18 ignored=0} mutations{seen=22 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260202 02:53:34.098168 5099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: Bootstrap complete.
I20260202 02:53:34.103551 5099 raft_consensus.cc:359] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 5 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:34.104280 5099 raft_consensus.cc:740] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666b7e52b5fc4f3397f04960c8b67100, State: Initialized, Role: FOLLOWER
I20260202 02:53:34.104996 5099 consensus_queue.cc:260] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 31, Last appended: 5.31, Last appended by leader: 31, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:34.105877 5103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:34.106159 5103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:53:34.107048 5099 sys_catalog.cc:565] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: configured and running, proceeding with master startup.
I20260202 02:53:34.121387 5117 catalog_manager.cc:1269] Loaded cluster ID: 0d95b889b53247a0813a4fda5550ae30
I20260202 02:53:34.121575 5117 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: success
I20260202 02:53:34.126586 5117 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: acquiring CA information for follower catalog manager: success
I20260202 02:53:34.130046 5117 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260202 02:53:34.141361 5028 master_runner.cc:186] Error getting master registration for 127.0.104.189:40361: Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111),
I20260202 02:53:34.147032 5028 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:53:34.173048 4097 heartbeater.cc:344] Connected to a master server at 127.0.104.190:34577
I20260202 02:53:34.176481 5064 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "a34c77c818054f87bc19600949531ad0" instance_seqno: 1770000797481684) as {username='slave'} at 127.0.104.130:32919; Asking this server to re-register.
I20260202 02:53:34.177410 4097 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:34.179443 5064 ts_manager.cc:194] Registered new tserver with Master: a34c77c818054f87bc19600949531ad0 (127.0.104.130:38525)
I20260202 02:53:34.346356 5121 raft_consensus.cc:493] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:34.346576 5121 raft_consensus.cc:515] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:34.347271 5121 leader_election.cc:290] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361)
W20260202 02:53:34.350231 4896 leader_election.cc:336] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361): Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
I20260202 02:53:34.355430 5074 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "879caa0e6c114ab4855b7a756e252c9e" candidate_term: 7 candidate_status { last_received { term: 3 index: 21 } } ignore_live_leader: false dest_uuid: "666b7e52b5fc4f3397f04960c8b67100" is_pre_election: true
I20260202 02:53:34.356191 5074 raft_consensus.cc:2410] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 879caa0e6c114ab4855b7a756e252c9e for term 7 because replica has last-logged OpId of term: 5 index: 31, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 21.
I20260202 02:53:34.356874 4898 leader_election.cc:304] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 879caa0e6c114ab4855b7a756e252c9e; no voters: 3acf71c31a284578b4f8c918eaa50205, 666b7e52b5fc4f3397f04960c8b67100
I20260202 02:53:34.357260 5121 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260202 02:53:34.378625 4233 heartbeater.cc:344] Connected to a master server at 127.0.104.190:34577
I20260202 02:53:34.378887 4233 heartbeater.cc:499] Master 127.0.104.190:34577 was elected leader, sending a full tablet report...
I20260202 02:53:34.381165 5064 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "68f130042e0c4b8292b138ff8a322517" instance_seqno: 1770000797970665) as {username='slave'} at 127.0.104.131:34707; Asking this server to re-register.
I20260202 02:53:34.382150 4233 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:34.384217 5064 ts_manager.cc:194] Registered new tserver with Master: 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661)
I20260202 02:53:34.388828 3955 heartbeater.cc:344] Connected to a master server at 127.0.104.190:34577
I20260202 02:53:34.389068 3955 heartbeater.cc:499] Master 127.0.104.190:34577 was elected leader, sending a full tablet report...
I20260202 02:53:34.391156 5064 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "d9570a47043b4891b2392af17f405e3d" instance_seqno: 1770000796958858) as {username='slave'} at 127.0.104.129:43699; Asking this server to re-register.
I20260202 02:53:34.391882 3955 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:34.393463 5064 ts_manager.cc:194] Registered new tserver with Master: d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425)
I20260202 02:53:34.606043 5103 raft_consensus.cc:493] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:34.606341 5103 raft_consensus.cc:515] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:34.608316 5103 leader_election.cc:290] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:53:34.609262 5049 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
W20260202 02:53:34.611114 5049 leader_election.cc:336] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361): Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
I20260202 02:53:34.614853 4921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "666b7e52b5fc4f3397f04960c8b67100" candidate_term: 6 candidate_status { last_received { term: 5 index: 31 } } ignore_live_leader: false dest_uuid: "879caa0e6c114ab4855b7a756e252c9e" is_pre_election: true
I20260202 02:53:34.615213 4921 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 666b7e52b5fc4f3397f04960c8b67100 in current term 6: Already voted for candidate 3acf71c31a284578b4f8c918eaa50205 in this term.
I20260202 02:53:34.615871 5051 leader_election.cc:304] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666b7e52b5fc4f3397f04960c8b67100; no voters: 3acf71c31a284578b4f8c918eaa50205, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:34.616317 5103 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 5 FOLLOWER]: Advancing to term 6
I20260202 02:53:34.620028 5103 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260202 02:53:35.147493 5133 raft_consensus.cc:493] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:35.147701 5133 raft_consensus.cc:515] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:35.148494 5133 leader_election.cc:290] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
I20260202 02:53:35.149049 4921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "666b7e52b5fc4f3397f04960c8b67100" candidate_term: 7 candidate_status { last_received { term: 5 index: 31 } } ignore_live_leader: false dest_uuid: "879caa0e6c114ab4855b7a756e252c9e" is_pre_election: true
I20260202 02:53:35.149338 4921 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666b7e52b5fc4f3397f04960c8b67100 in term 6.
I20260202 02:53:35.149812 5051 leader_election.cc:304] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e; no voters:
I20260202 02:53:35.150135 5133 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260202 02:53:35.150318 5133 raft_consensus.cc:493] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:53:35.150444 5133 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 6 FOLLOWER]: Advancing to term 7
W20260202 02:53:35.150593 5049 leader_election.cc:336] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361): Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
I20260202 02:53:35.152567 5133 raft_consensus.cc:515] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:35.153306 5133 leader_election.cc:290] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 7 election: Requested vote from peers 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
I20260202 02:53:35.153903 4921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "666b7e52b5fc4f3397f04960c8b67100" candidate_term: 7 candidate_status { last_received { term: 5 index: 31 } } ignore_live_leader: false dest_uuid: "879caa0e6c114ab4855b7a756e252c9e"
I20260202 02:53:35.154104 4921 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 6 FOLLOWER]: Advancing to term 7
W20260202 02:53:35.154722 5049 leader_election.cc:336] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361): Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
I20260202 02:53:35.155989 4921 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666b7e52b5fc4f3397f04960c8b67100 in term 7.
I20260202 02:53:35.156533 5051 leader_election.cc:304] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e; no voters: 3acf71c31a284578b4f8c918eaa50205
I20260202 02:53:35.156983 5133 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 7 FOLLOWER]: Leader election won for term 7
I20260202 02:53:35.158517 5133 raft_consensus.cc:697] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 7 LEADER]: Becoming Leader. State: Replica: 666b7e52b5fc4f3397f04960c8b67100, State: Running, Role: LEADER
I20260202 02:53:35.159250 5133 consensus_queue.cc:237] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 5.31, Last appended by leader: 31, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:35.161917 5136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 666b7e52b5fc4f3397f04960c8b67100. Latest consensus state: current_term: 7 leader_uuid: "666b7e52b5fc4f3397f04960c8b67100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:35.162295 5136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: This master's current role is: LEADER
I20260202 02:53:35.163031 5138 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260202 02:53:35.165490 5138 catalog_manager.cc:679] Loaded metadata for table default.table_6a23cf9a48824f7e8d243c355f458ce8 [id=0048dd71261d4204ac5548c190957aa9]
I20260202 02:53:35.165928 5138 catalog_manager.cc:679] Loaded metadata for table default.table_8d32aa2872ab405fa4f2e7b1dac33f56 [id=2ab6a7a17803446898f275ae3faa032c]
I20260202 02:53:35.166241 5138 catalog_manager.cc:679] Loaded metadata for table default.table_2c330ab9e5984ba5980e7264050e7a41 [id=3852a850379342c69688c71b95873308]
I20260202 02:53:35.166584 5138 catalog_manager.cc:679] Loaded metadata for table default.table_4e64388f537d44aaa6d541b90e924c65 [id=4f41a1a5a6844fbfb43b6798b2c6f67d]
I20260202 02:53:35.166872 5138 catalog_manager.cc:679] Loaded metadata for table default.table_470455fd011c452b9cceafbc413b1cfe [id=e28252bb319446c39056986e8b98de30]
I20260202 02:53:35.168637 5138 tablet_loader.cc:96] loaded metadata for tablet 7b76f6283a604527946408b2a8983ee5 (table default.table_2c330ab9e5984ba5980e7264050e7a41 [id=3852a850379342c69688c71b95873308])
I20260202 02:53:35.168987 5138 tablet_loader.cc:96] loaded metadata for tablet 906f9148074848da8456b204a587fa8a (table default.table_8d32aa2872ab405fa4f2e7b1dac33f56 [id=2ab6a7a17803446898f275ae3faa032c])
I20260202 02:53:35.169272 5138 tablet_loader.cc:96] loaded metadata for tablet aa58249c51134388b92c95408bd8c923 (table default.table_6a23cf9a48824f7e8d243c355f458ce8 [id=0048dd71261d4204ac5548c190957aa9])
I20260202 02:53:35.169551 5138 tablet_loader.cc:96] loaded metadata for tablet b5a121d55d9d498ba76bb4dcb24d0e20 (table default.table_470455fd011c452b9cceafbc413b1cfe [id=e28252bb319446c39056986e8b98de30])
I20260202 02:53:35.169807 5138 tablet_loader.cc:96] loaded metadata for tablet c58cc255392e4a74b94c1cfae918c70e (table default.table_4e64388f537d44aaa6d541b90e924c65 [id=4f41a1a5a6844fbfb43b6798b2c6f67d])
I20260202 02:53:35.170029 5138 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260202 02:53:35.171041 5138 catalog_manager.cc:1269] Loaded cluster ID: 0d95b889b53247a0813a4fda5550ae30
I20260202 02:53:35.171113 5138 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260202 02:53:35.172415 5138 catalog_manager.cc:1514] Loading token signing keys...
I20260202 02:53:35.173308 5138 catalog_manager.cc:6038] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: Loaded TSK: 0
I20260202 02:53:35.174154 5138 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260202 02:53:35.175067 5138 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260202 02:53:35.175922 5138 catalog_manager.cc:5876] Last processed Hive Metastore notification event ID: -1
I20260202 02:53:35.225839 5064 master_service.cc:954] ReplaceTablet: received request to replace tablet 8d1373239ae14791a720a0a7ca86d09a from {username='slave'} at 127.0.0.1:53300
I20260202 02:53:35.225929 5063 master_service.cc:954] ReplaceTablet: received request to replace tablet 76c092d3e26c4906964e0f852ac24448 from {username='slave'} at 127.0.0.1:53300
I20260202 02:53:35.226181 5061 master_service.cc:954] ReplaceTablet: received request to replace tablet 8d1373239ae14791a720a0a7ca86d09a from {username='slave'} at 127.0.0.1:53300
I20260202 02:53:35.226511 5060 master_service.cc:954] ReplaceTablet: received request to replace tablet 7b76f6283a604527946408b2a8983ee5 from {username='slave'} at 127.0.0.1:53300
I20260202 02:53:35.226666 5062 master_service.cc:954] ReplaceTablet: received request to replace tablet 906f9148074848da8456b204a587fa8a from {username='slave'} at 127.0.0.1:53300
I20260202 02:53:35.226945 5059 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:53300:
table { table_name: "default.Table_4e64388f537d44aaa6d541b90e924c65" } new_table_name: "default.Table_4ad4563b34454572a4930ad034763780" modify_external_catalogs: true
I20260202 02:53:35.227326 5061 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53300:
name: "default.Table_2faba70635fa4e55ae56458e06dba641"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:35.228315 5056 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:53300:
table { table_name: "default.Table_470455fd011c452b9cceafbc413b1cfe" } new_table_name: "default.Table_d4d44071ee424b9fa646dd6fe1c3ecd8" modify_external_catalogs: true
I20260202 02:53:35.228513 5058 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53300:
name: "default.Table_dc0bdab2115f4027a86e680d2995f726"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:35.229252 5057 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53300:
name: "default.Table_fb2a5c4107db446aba29a7f07a46c1a8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:35.229966 5055 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:53300:
table { table_name: "default.Table_6a23cf9a48824f7e8d243c355f458ce8" } new_table_name: "default.Table_824602d24d0345759eda1443b7232548" modify_external_catalogs: true
I20260202 02:53:35.230494 5064 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53300:
name: "default.Table_de1272fe656e46328aa237aec594394a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:35.230753 5063 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:53300:
table { table_name: "default.Table_8d32aa2872ab405fa4f2e7b1dac33f56" } new_table_name: "default.Table_042b89c2475b4966b48af7927ba62e21" modify_external_catalogs: true
I20260202 02:53:35.234501 4921 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 7 FOLLOWER]: Refusing update from remote peer 666b7e52b5fc4f3397f04960c8b67100: Log matching property violated. Preceding OpId in replica: term: 3 index: 21. Preceding OpId from leader: term: 7 index: 33. (index mismatch)
I20260202 02:53:35.236008 5136 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 21, Time since last communication: 0.000s
F20260202 02:53:35.236586 5116 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 7 received events: NotificationEvent(eventId=1, eventTime=1770000798, eventType=CREATE_TABLE, dbName=default, tableName=table_4e64388f537d44aaa6d541b90e924c65, message={"server":"","servicePrincipal":"","db":"default","table":"table_4e64388f537d44aaa6d541b90e924c65","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_4e64388f537d44aaa6d541b90e924c65\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770000798},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/warehouse/table_4e64388f537d44aaa6d541b90e924c65\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_4e64388f537d44aaa6d541b90e924c65\",\"kudu.table_id\":\"4f41a1a5a6844fbfb43b6798b2c6f67d\",\"transient_lastDdlTime\":\"1770000798\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"0d95b889b53247a0813a4fda5550ae30\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770000798,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1770000798, eventType=CREATE_TABLE, dbName=default, tableName=table_a728b4dd173b4f0c8c6b9cce0bcda94d, message={"server":"","servicePrincipal":"","db":"default","table":"table_a728b4dd173b4f0c8c6b9cce0bcda94d","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_a728b4dd173b4f0c8c6b9cce0bcda94d\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770000798},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/warehouse/table_a728b4dd173b4f0c8c6b9cce0bcda94d\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_a728b4dd173b4f0c8c6b9cce0bcda94d\",\"kudu.table_id\":\"a54f76c7d1ff4769b2d84a5fe325d81c\",\"transient_lastDdlTime\":\"1770000798\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"0d95b889b53247a0813a4fda5550ae30\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770000798,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
W20260202 02:53:35.238503 5049 consensus_peers.cc:597] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 -> Peer 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361): Couldn't send request to peer 3acf71c31a284578b4f8c918eaa50205. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
@ 0x7f85c1d44dcd google::LogMessage::Fail() at ??:0
@ 0x7f85c1d48b93 google::LogMessage::SendToLog() at ??:0
@ 0x7f85c1d447cc google::LogMessage::Flush() at ??:0
@ 0x7f85c1d44a11 google::LogMessage::~LogMessage() at ??:0
@ 0x7f85d873b7f4 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f85d873a0a1 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260202 02:53:35.254884 5142 mvcc.cc:204] Tried to move back new op lower bound from 7249923292917846016 to 7249923277508771840. Current Snapshot: MvccSnapshot[applied={T|T < 7249923292917846016 or (T in {7249923292917846016})}]
I20260202 02:53:35.256099 5141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 666b7e52b5fc4f3397f04960c8b67100. Latest consensus state: current_term: 7 leader_uuid: "666b7e52b5fc4f3397f04960c8b67100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:35.256414 5141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:53:35.256839 5145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "666b7e52b5fc4f3397f04960c8b67100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:35.257119 5145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:53:35.257078 5149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "666b7e52b5fc4f3397f04960c8b67100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:35.257308 5149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:53:35.258124 5139 mvcc.cc:204] Tried to move back new op lower bound from 7249923339190161408 to 7249923338896924672. Current Snapshot: MvccSnapshot[applied={T|T < 7249923339190161408}]
@ 0x7f85d8741d1c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260202 02:53:35.261734 5149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "666b7e52b5fc4f3397f04960c8b67100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:35.261953 5149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:53:35.263628 5062 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: Sending DeleteTablet for 3 replicas of tablet 906f9148074848da8456b204a587fa8a
I20260202 02:53:35.264057 5060 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: Sending DeleteTablet for 3 replicas of tablet 7b76f6283a604527946408b2a8983ee5
@ 0x7f85d8741b99 std::_Function_handler<>::_M_invoke() at ??:0
I20260202 02:53:35.265689 5062 catalog_manager.cc:6724] ReplaceTablet: tablet 906f9148074848da8456b204a587fa8a deleted and replaced by tablet bb2b2a691a544f8a8997d0c144acb2e9
@ 0xa8970d std::function<>::operator()() at ??:?
I20260202 02:53:35.268308 5060 catalog_manager.cc:6724] ReplaceTablet: tablet 7b76f6283a604527946408b2a8983ee5 deleted and replaced by tablet 4f3dd16be700418b80728adc2ae0cd85
@ 0x7f85c4ab9f99 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f85c80c66db start_thread at ??:0
@ 0x7f85c0bf571f clone at ??:0
W20260202 02:53:35.459255 4898 connection.cc:537] client connection to 127.0.104.190:34577 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260202 02:53:35.459785 4875 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107),
I20260202 02:53:35.460022 4875 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:53:35.459990 4242 connection.cc:537] client connection to 127.0.104.190:34577 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260202 02:53:35.460233 4123 connection.cc:537] client connection to 127.0.104.190:34577 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260202 02:53:35.460253 5158 negotiation.cc:337] Failed RPC negotiation. Trace:
0202 02:53:35.267322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60032 (local address 127.0.104.129:44425)
0202 02:53:35.282111 (+ 14789us) server_negotiation.cc:207] Beginning negotiation
0202 02:53:35.282118 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0202 02:53:35.282149 (+ 31us) server_negotiation.cc:408] Connection header received
0202 02:53:35.282242 (+ 93us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0202 02:53:35.282249 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0202 02:53:35.282357 (+ 108us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0202 02:53:35.282513 (+ 156us) server_negotiation.cc:227] Negotiated authn=CERTIFICATE
0202 02:53:35.459233 (+176720us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60032: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":14634,"thread_start_us":14538,"threads_started":1}
W20260202 02:53:35.460695 5159 negotiation.cc:337] Failed RPC negotiation. Trace:
0202 02:53:35.267701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43232 (local address 127.0.104.131:46661)
0202 02:53:35.282165 (+ 14464us) server_negotiation.cc:207] Beginning negotiation
0202 02:53:35.282171 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0202 02:53:35.459194 (+177023us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43232: BlockingRecv error: recv got EOF from 127.0.0.1:43232 (error 108)
Metrics: {"server-negotiator.queue_time_us":13369,"thread_start_us":13212,"threads_started":1}
W20260202 02:53:35.461099 4242 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.104.190:34577)
W20260202 02:53:35.461601 4242 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.104.190:34577)
W20260202 02:53:35.460578 3843 connection.cc:537] client connection to 127.0.104.190:34577 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260202 02:53:35.462049 4242 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.104.190:34577)
W20260202 02:53:35.462458 5157 negotiation.cc:337] Failed RPC negotiation. Trace:
0202 02:53:35.267134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55560 (local address 127.0.104.130:38525)
0202 02:53:35.282557 (+ 15423us) server_negotiation.cc:207] Beginning negotiation
0202 02:53:35.282564 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0202 02:53:35.282584 (+ 20us) server_negotiation.cc:408] Connection header received
0202 02:53:35.282654 (+ 70us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0202 02:53:35.282660 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0202 02:53:35.282735 (+ 75us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0202 02:53:35.282831 (+ 96us) server_negotiation.cc:227] Negotiated authn=CERTIFICATE
0202 02:53:35.460871 (+178040us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55560: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":15302,"thread_start_us":15194,"threads_started":1}
I20260202 02:53:35.759454 5149 raft_consensus.cc:493] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 666b7e52b5fc4f3397f04960c8b67100)
I20260202 02:53:35.759644 5149 raft_consensus.cc:515] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:35.760314 5149 leader_election.cc:290] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361)
W20260202 02:53:35.761422 4898 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111) [suppressed 6 similar messages]
W20260202 02:53:35.762810 4896 leader_election.cc:336] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361): Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
W20260202 02:53:35.762974 4898 leader_election.cc:336] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
I20260202 02:53:35.763126 4898 leader_election.cc:304] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 879caa0e6c114ab4855b7a756e252c9e; no voters: 3acf71c31a284578b4f8c918eaa50205, 666b7e52b5fc4f3397f04960c8b67100
I20260202 02:53:35.763358 5149 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260202 02:53:36.508423 5178 raft_consensus.cc:493] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 666b7e52b5fc4f3397f04960c8b67100)
I20260202 02:53:36.508646 5178 raft_consensus.cc:515] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:36.509284 5178 leader_election.cc:290] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361)
W20260202 02:53:36.511346 4896 leader_election.cc:336] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361): Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
W20260202 02:53:36.511492 4898 leader_election.cc:336] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
I20260202 02:53:36.511631 4898 leader_election.cc:304] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 879caa0e6c114ab4855b7a756e252c9e; no voters: 3acf71c31a284578b4f8c918eaa50205, 666b7e52b5fc4f3397f04960c8b67100
I20260202 02:53:36.511889 5178 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260202 02:53:37.110469 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskYXxklX/build/asan/bin/kudu
/tmp/dist-test-taskYXxklX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.190:34577
--webserver_interface=127.0.104.190
--webserver_port=33649
--builtin_ntp_servers=127.0.104.148:34531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525
--hive_metastore_uris=thrift://127.0.0.1:46545
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260202 02:53:37.484970 5183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260202 02:53:37.485316 5183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260202 02:53:37.485430 5183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260202 02:53:37.496968 5183 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260202 02:53:37.497102 5183 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260202 02:53:37.497196 5183 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260202 02:53:37.497259 5183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260202 02:53:37.497339 5183 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260202 02:53:37.497409 5183 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260202 02:53:37.511305 5183 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:34531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:46545
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.190:34577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.104.190
--webserver_port=33649
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3722ed49c19ce54c7feae2fa944e32b1d4ed8d2d
build type FASTDEBUG
built by None at 02 Feb 2026 02:43:23 UTC on 24a791456cd2
build id 10304
ASAN enabled
I20260202 02:53:37.513067 5183 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260202 02:53:37.515096 5183 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260202 02:53:37.525204 5193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:53:37.525362 5191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:53:37.525216 5190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260202 02:53:37.525928 5183 server_base.cc:1060] running on GCE node
I20260202 02:53:37.526767 5183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260202 02:53:37.528430 5183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260202 02:53:37.529685 5183 hybrid_clock.cc:648] HybridClock initialized: now 1770000817529632 us; error 49 us; skew 500 ppm
I20260202 02:53:37.530048 5183 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260202 02:53:37.532599 5183 webserver.cc:492] Webserver started at http://127.0.104.190:33649/ using document root <none> and password file <none>
I20260202 02:53:37.533401 5183 fs_manager.cc:362] Metadata directory not provided
I20260202 02:53:37.533504 5183 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260202 02:53:37.539682 5183 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.000s sys 0.006s
I20260202 02:53:37.543352 5199 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260202 02:53:37.545051 5183 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260202 02:53:37.545248 5183 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data,/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/wal
uuid: "666b7e52b5fc4f3397f04960c8b67100"
format_stamp: "Formatted at 2026-02-02 02:53:14 on dist-test-slave-llj2"
I20260202 02:53:37.546032 5183 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260202 02:53:37.566475 5183 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260202 02:53:37.567381 5183 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260202 02:53:37.567683 5183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260202 02:53:37.589848 5183 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.190:34577
I20260202 02:53:37.589879 5250 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.190:34577 every 8 connection(s)
I20260202 02:53:37.591603 5183 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data/info.pb
I20260202 02:53:37.595145 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskYXxklX/build/asan/bin/kudu as pid 5183
I20260202 02:53:37.600224 5251 sys_catalog.cc:263] Verifying existing consensus state
I20260202 02:53:37.605188 5251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: Bootstrap starting.
I20260202 02:53:37.617983 5254 raft_consensus.cc:493] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 666b7e52b5fc4f3397f04960c8b67100)
I20260202 02:53:37.618232 5254 raft_consensus.cc:515] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:37.618484 5251 log.cc:826] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: Log is configured to *not* fsync() on all Append() calls
I20260202 02:53:37.619616 5254 leader_election.cc:290] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361)
W20260202 02:53:37.623677 4896 leader_election.cc:336] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361): Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
I20260202 02:53:37.654980 5251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: Bootstrap replayed 1/1 log segments. Stats: ops{read=34 overwritten=0 applied=34 ignored=0} inserts{seen=20 ignored=0} mutations{seen=24 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260202 02:53:37.655686 5251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: Bootstrap complete.
I20260202 02:53:37.665037 5251 raft_consensus.cc:359] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 7 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:37.666079 5251 raft_consensus.cc:740] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 7 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666b7e52b5fc4f3397f04960c8b67100, State: Initialized, Role: FOLLOWER
I20260202 02:53:37.666832 5251 consensus_queue.cc:260] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 34, Last appended: 7.34, Last appended by leader: 34, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:37.667761 5260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 7 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:37.668205 5260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:53:37.669008 5251 sys_catalog.cc:565] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: configured and running, proceeding with master startup.
I20260202 02:53:37.669044 5226 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "879caa0e6c114ab4855b7a756e252c9e" candidate_term: 8 candidate_status { last_received { term: 7 index: 34 } } ignore_live_leader: false dest_uuid: "666b7e52b5fc4f3397f04960c8b67100" is_pre_election: true
I20260202 02:53:37.669661 5226 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 879caa0e6c114ab4855b7a756e252c9e in term 7.
I20260202 02:53:37.670362 4898 leader_election.cc:304] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e; no voters: 3acf71c31a284578b4f8c918eaa50205
I20260202 02:53:37.670756 5254 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260202 02:53:37.670888 5254 raft_consensus.cc:493] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 666b7e52b5fc4f3397f04960c8b67100)
I20260202 02:53:37.670977 5254 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 7 FOLLOWER]: Advancing to term 8
I20260202 02:53:37.673484 5254 raft_consensus.cc:515] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:37.674227 5254 leader_election.cc:290] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 8 election: Requested vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361)
I20260202 02:53:37.674821 5226 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "879caa0e6c114ab4855b7a756e252c9e" candidate_term: 8 candidate_status { last_received { term: 7 index: 34 } } ignore_live_leader: false dest_uuid: "666b7e52b5fc4f3397f04960c8b67100"
I20260202 02:53:37.675029 5226 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 7 FOLLOWER]: Advancing to term 8
W20260202 02:53:37.676196 4896 leader_election.cc:336] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361): Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
I20260202 02:53:37.680895 5226 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 879caa0e6c114ab4855b7a756e252c9e in term 8.
I20260202 02:53:37.681587 4898 leader_election.cc:304] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e; no voters: 3acf71c31a284578b4f8c918eaa50205
I20260202 02:53:37.684450 5254 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 8 FOLLOWER]: Leader election won for term 8
I20260202 02:53:37.700655 5254 raft_consensus.cc:697] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [term 8 LEADER]: Becoming Leader. State: Replica: 879caa0e6c114ab4855b7a756e252c9e, State: Running, Role: LEADER
I20260202 02:53:37.701309 5254 consensus_queue.cc:237] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 7.34, Last appended by leader: 34, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:37.698035 5183 master_runner.cc:186] Error getting master registration for 127.0.104.189:40361: Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111),
I20260202 02:53:37.701905 5183 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:53:37.704838 5278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 879caa0e6c114ab4855b7a756e252c9e. Latest consensus state: current_term: 8 leader_uuid: "879caa0e6c114ab4855b7a756e252c9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:37.705307 5278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [sys.catalog]: This master's current role is: LEADER
I20260202 02:53:37.706538 5280 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260202 02:53:37.706550 5274 catalog_manager.cc:1269] Loaded cluster ID: 0d95b889b53247a0813a4fda5550ae30
I20260202 02:53:37.706641 5274 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: success
I20260202 02:53:37.709003 5280 catalog_manager.cc:679] Loaded metadata for table default.table_6a23cf9a48824f7e8d243c355f458ce8 [id=0048dd71261d4204ac5548c190957aa9]
I20260202 02:53:37.709474 5280 catalog_manager.cc:679] Loaded metadata for table default.table_8d32aa2872ab405fa4f2e7b1dac33f56 [id=2ab6a7a17803446898f275ae3faa032c]
I20260202 02:53:37.709852 5280 catalog_manager.cc:679] Loaded metadata for table default.table_2c330ab9e5984ba5980e7264050e7a41 [id=3852a850379342c69688c71b95873308]
I20260202 02:53:37.710246 5280 catalog_manager.cc:679] Loaded metadata for table default.table_4e64388f537d44aaa6d541b90e924c65 [id=4f41a1a5a6844fbfb43b6798b2c6f67d]
I20260202 02:53:37.710588 5280 catalog_manager.cc:679] Loaded metadata for table default.table_470455fd011c452b9cceafbc413b1cfe [id=e28252bb319446c39056986e8b98de30]
I20260202 02:53:37.711015 5274 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: acquiring CA information for follower catalog manager: success
I20260202 02:53:37.712544 5280 tablet_loader.cc:96] loaded metadata for tablet 4f3dd16be700418b80728adc2ae0cd85 (table default.table_2c330ab9e5984ba5980e7264050e7a41 [id=3852a850379342c69688c71b95873308])
I20260202 02:53:37.713944 5280 tablet_loader.cc:96] loaded metadata for tablet aa58249c51134388b92c95408bd8c923 (table default.table_6a23cf9a48824f7e8d243c355f458ce8 [id=0048dd71261d4204ac5548c190957aa9])
I20260202 02:53:37.714025 5274 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260202 02:53:37.714330 5280 tablet_loader.cc:96] loaded metadata for tablet b5a121d55d9d498ba76bb4dcb24d0e20 (table default.table_470455fd011c452b9cceafbc413b1cfe [id=e28252bb319446c39056986e8b98de30])
I20260202 02:53:37.714592 5280 tablet_loader.cc:96] loaded metadata for tablet bb2b2a691a544f8a8997d0c144acb2e9 (table default.table_8d32aa2872ab405fa4f2e7b1dac33f56 [id=2ab6a7a17803446898f275ae3faa032c])
I20260202 02:53:37.714993 5280 tablet_loader.cc:96] loaded metadata for tablet c58cc255392e4a74b94c1cfae918c70e (table default.table_4e64388f537d44aaa6d541b90e924c65 [id=4f41a1a5a6844fbfb43b6798b2c6f67d])
I20260202 02:53:37.715301 5280 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260202 02:53:37.716486 5280 catalog_manager.cc:1269] Loaded cluster ID: 0d95b889b53247a0813a4fda5550ae30
I20260202 02:53:37.716572 5280 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260202 02:53:37.718043 5280 catalog_manager.cc:1514] Loading token signing keys...
I20260202 02:53:37.719089 5280 catalog_manager.cc:6038] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: Loaded TSK: 0
I20260202 02:53:37.720003 5280 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260202 02:53:37.721058 5280 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260202 02:53:37.722012 5280 catalog_manager.cc:5876] Last processed Hive Metastore notification event ID: -1
I20260202 02:53:37.732810 5226 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Refusing update from remote peer 879caa0e6c114ab4855b7a756e252c9e: Log matching property violated. Preceding OpId in replica: term: 7 index: 34. Preceding OpId from leader: term: 8 index: 36. (index mismatch)
W20260202 02:53:37.733286 4896 consensus_peers.cc:597] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e -> Peer 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361): Couldn't send request to peer 3acf71c31a284578b4f8c918eaa50205. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260202 02:53:37.733795 5254 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20260202 02:53:37.739162 5260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 879caa0e6c114ab4855b7a756e252c9e. Latest consensus state: current_term: 8 leader_uuid: "879caa0e6c114ab4855b7a756e252c9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:37.739486 5260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:53:37.741650 5278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 8 leader_uuid: "879caa0e6c114ab4855b7a756e252c9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:37.741869 5278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e [sys.catalog]: This master's current role is: LEADER
I20260202 02:53:37.746075 5260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 8 leader_uuid: "879caa0e6c114ab4855b7a756e252c9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:37.746325 5260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:53:37.770151 4097 heartbeater.cc:344] Connected to a master server at 127.0.104.190:34577
W20260202 02:53:37.770218 4909 catalog_manager.cc:5487] Ignoring report from unknown tablet 313ff5ec7131498aaaa0c832abea2c56
I20260202 02:53:37.774399 5215 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "a34c77c818054f87bc19600949531ad0" instance_seqno: 1770000797481684) as {username='slave'} at 127.0.104.130:48249; Asking this server to re-register.
I20260202 02:53:37.775287 4165 tablet_service.cc:1505] Processing CreateTablet for tablet bb2b2a691a544f8a8997d0c144acb2e9 (DEFAULT_TABLE table=default.table_8d32aa2872ab405fa4f2e7b1dac33f56 [id=2ab6a7a17803446898f275ae3faa032c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:37.775287 4167 tablet_service.cc:1505] Processing CreateTablet for tablet 4f3dd16be700418b80728adc2ae0cd85 (DEFAULT_TABLE table=default.table_2c330ab9e5984ba5980e7264050e7a41 [id=3852a850379342c69688c71b95873308]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:37.775810 4167 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f3dd16be700418b80728adc2ae0cd85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:37.780246 4165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb2b2a691a544f8a8997d0c144acb2e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:37.788955 4097 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:37.791333 5216 ts_manager.cc:194] Registered new tserver with Master: a34c77c818054f87bc19600949531ad0 (127.0.104.130:38525)
I20260202 02:53:37.793056 4909 master_service.cc:954] ReplaceTablet: received request to replace tablet 7b76f6283a604527946408b2a8983ee5 from {username='slave'} at 127.0.0.1:42196
I20260202 02:53:37.795823 4903 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42196:
name: "default.Table_2faba70635fa4e55ae56458e06dba641"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:37.796720 4910 master_service.cc:954] ReplaceTablet: received request to replace tablet c58cc255392e4a74b94c1cfae918c70e from {username='slave'} at 127.0.0.1:42196
I20260202 02:53:37.799315 4911 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42196:
name: "default.Table_a728b4dd173b4f0c8c6b9cce0bcda94d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:37.800130 4908 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42196:
name: "default.Table_2d03ff86a95f4175b2225ecc0d899058"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:37.800947 4907 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42196:
name: "default.Table_2a3cc2b06aae483d8e2b84d018d13310"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:37.801687 4906 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42196:
name: "default.Table_851c0addb9fd444eb3b9dd3a3ddd4623"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:37.802745 4905 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:42196:
table { table_name: "default.Table_4e64388f537d44aaa6d541b90e924c65" } new_table_name: "default.Table_4ad4563b34454572a4930ad034763780" modify_external_catalogs: true
I20260202 02:53:37.803475 4904 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42196:
name: "default.Table_de1272fe656e46328aa237aec594394a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:37.804185 4902 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42196:
name: "default.Table_dc0bdab2115f4027a86e680d2995f726"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260202 02:53:37.815831 4966 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 7 received events: NotificationEvent(eventId=1, eventTime=1770000798, eventType=CREATE_TABLE, dbName=default, tableName=table_4e64388f537d44aaa6d541b90e924c65, message={"server":"","servicePrincipal":"","db":"default","table":"table_4e64388f537d44aaa6d541b90e924c65","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_4e64388f537d44aaa6d541b90e924c65\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770000798},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/warehouse/table_4e64388f537d44aaa6d541b90e924c65\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_4e64388f537d44aaa6d541b90e924c65\",\"kudu.table_id\":\"4f41a1a5a6844fbfb43b6798b2c6f67d\",\"transient_lastDdlTime\":\"1770000798\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"0d95b889b53247a0813a4fda5550ae30\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770000798,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1770000798, eventType=CREATE_TABLE, dbName=default, tableName=table_a728b4dd173b4f0c8c6b9cce0bcda94d, message={"server":"","servicePrincipal":"","db":"default","table":"table_a728b4dd173b4f0c8c6b9cce0bcda94d","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_a728b4dd173b4f0c8c6b9cce0bcda94d\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770000798},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/warehouse/table_a728b4dd173b4f0c8c6b9cce0bcda94d\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_a728b4dd173b4f0c8c6b9cce0bcda94d\",\"kudu.table_id\":\"a54f76c7d1ff4769b2d84a5fe325d81c\",\"transient_lastDdlTime\":\"1770000798\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"0d95b889b53247a0813a4fda5550ae30\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770000798,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
I20260202 02:53:37.817711 4910 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 879caa0e6c114ab4855b7a756e252c9e: Sending DeleteTablet for 3 replicas of tablet c58cc255392e4a74b94c1cfae918c70e
I20260202 02:53:37.819574 4909 master_service.cc:954] ReplaceTablet: received request to replace tablet 313ff5ec7131498aaaa0c832abea2c56 from {username='slave'} at 127.0.0.1:42196
I20260202 02:53:37.820732 3889 tablet_service.cc:1505] Processing CreateTablet for tablet 4f3dd16be700418b80728adc2ae0cd85 (DEFAULT_TABLE table=default.table_2c330ab9e5984ba5980e7264050e7a41 [id=3852a850379342c69688c71b95873308]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:37.821174 3887 tablet_service.cc:1505] Processing CreateTablet for tablet bb2b2a691a544f8a8997d0c144acb2e9 (DEFAULT_TABLE table=default.table_8d32aa2872ab405fa4f2e7b1dac33f56 [id=2ab6a7a17803446898f275ae3faa032c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:37.821309 3889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f3dd16be700418b80728adc2ae0cd85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:37.821556 3887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb2b2a691a544f8a8997d0c144acb2e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:37.827575 4167 tablet_service.cc:1552] Processing DeleteTablet for tablet c58cc255392e4a74b94c1cfae918c70e with delete_type TABLET_DATA_DELETED (replaced by tablet 5313e773f68d4ad2ad3e36880668051f) from {username='slave'} at 127.0.0.1:52846
I20260202 02:53:37.827857 3888 tablet_service.cc:1552] Processing DeleteTablet for tablet c58cc255392e4a74b94c1cfae918c70e with delete_type TABLET_DATA_DELETED (replaced by tablet 5313e773f68d4ad2ad3e36880668051f) from {username='slave'} at 127.0.0.1:33434
I20260202 02:53:37.835173 5294 tablet_replica.cc:333] T c58cc255392e4a74b94c1cfae918c70e P d9570a47043b4891b2392af17f405e3d: stopping tablet replica
I20260202 02:53:37.836149 5294 raft_consensus.cc:2243] T c58cc255392e4a74b94c1cfae918c70e P d9570a47043b4891b2392af17f405e3d [term 1 LEADER]: Raft consensus shutting down.
I20260202 02:53:37.837213 5294 raft_consensus.cc:2272] T c58cc255392e4a74b94c1cfae918c70e P d9570a47043b4891b2392af17f405e3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:53:37.840065 4910 catalog_manager.cc:6724] ReplaceTablet: tablet c58cc255392e4a74b94c1cfae918c70e deleted and replaced by tablet 5313e773f68d4ad2ad3e36880668051f
I20260202 02:53:37.840273 4166 tablet_service.cc:1505] Processing CreateTablet for tablet 5313e773f68d4ad2ad3e36880668051f (DEFAULT_TABLE table=default.table_4e64388f537d44aaa6d541b90e924c65 [id=4f41a1a5a6844fbfb43b6798b2c6f67d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:37.840824 4166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5313e773f68d4ad2ad3e36880668051f. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f4ddf89edcd google::LogMessage::Fail() at ??:0
I20260202 02:53:37.842837 4909 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42196:
name: "default.Table_fb2a5c4107db446aba29a7f07a46c1a8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:37.843263 4910 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:42196:
table { table_name: "default.Table_470455fd011c452b9cceafbc413b1cfe" } new_table_name: "default.Table_d4d44071ee424b9fa646dd6fe1c3ecd8" modify_external_catalogs: true
@ 0x7f4ddf8a2b93 google::LogMessage::SendToLog() at ??:0
@ 0x7f4ddf89e7cc google::LogMessage::Flush() at ??:0
@ 0x7f4ddf89ea11 google::LogMessage::~LogMessage() at ??:0
I20260202 02:53:37.840616 3886 tablet_service.cc:1505] Processing CreateTablet for tablet 5313e773f68d4ad2ad3e36880668051f (DEFAULT_TABLE table=default.table_4e64388f537d44aaa6d541b90e924c65 [id=4f41a1a5a6844fbfb43b6798b2c6f67d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:37.849081 3886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5313e773f68d4ad2ad3e36880668051f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:37.849193 5294 ts_tablet_manager.cc:1916] T c58cc255392e4a74b94c1cfae918c70e P d9570a47043b4891b2392af17f405e3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:53:37.853559 5293 tablet_replica.cc:333] T c58cc255392e4a74b94c1cfae918c70e P 68f130042e0c4b8292b138ff8a322517: stopping tablet replica
I20260202 02:53:37.854106 4030 tablet_service.cc:1505] Processing CreateTablet for tablet 4f3dd16be700418b80728adc2ae0cd85 (DEFAULT_TABLE table=default.table_2c330ab9e5984ba5980e7264050e7a41 [id=3852a850379342c69688c71b95873308]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:37.854426 4029 tablet_service.cc:1552] Processing DeleteTablet for tablet c58cc255392e4a74b94c1cfae918c70e with delete_type TABLET_DATA_DELETED (replaced by tablet 5313e773f68d4ad2ad3e36880668051f) from {username='slave'} at 127.0.0.1:38554
I20260202 02:53:37.854620 4030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f3dd16be700418b80728adc2ae0cd85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:37.856863 4028 tablet_service.cc:1505] Processing CreateTablet for tablet 5313e773f68d4ad2ad3e36880668051f (DEFAULT_TABLE table=default.table_4e64388f537d44aaa6d541b90e924c65 [id=4f41a1a5a6844fbfb43b6798b2c6f67d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:37.857460 4028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5313e773f68d4ad2ad3e36880668051f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:37.854106 4031 tablet_service.cc:1505] Processing CreateTablet for tablet bb2b2a691a544f8a8997d0c144acb2e9 (DEFAULT_TABLE table=default.table_8d32aa2872ab405fa4f2e7b1dac33f56 [id=2ab6a7a17803446898f275ae3faa032c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:37.860925 4031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb2b2a691a544f8a8997d0c144acb2e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:37.867375 5291 tablet_bootstrap.cc:492] T bb2b2a691a544f8a8997d0c144acb2e9 P 68f130042e0c4b8292b138ff8a322517: Bootstrap starting.
@ 0x7f4df62957f4 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260202 02:53:37.874655 5291 tablet_bootstrap.cc:654] T bb2b2a691a544f8a8997d0c144acb2e9 P 68f130042e0c4b8292b138ff8a322517: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:37.877056 5301 tablet_bootstrap.cc:492] T bb2b2a691a544f8a8997d0c144acb2e9 P a34c77c818054f87bc19600949531ad0: Bootstrap starting.
I20260202 02:53:37.877784 5293 raft_consensus.cc:2243] T c58cc255392e4a74b94c1cfae918c70e P 68f130042e0c4b8292b138ff8a322517 [term 1 FOLLOWER]: Raft consensus shutting down.
@ 0x7f4df62940a1 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260202 02:53:37.886114 5301 tablet_bootstrap.cc:654] T bb2b2a691a544f8a8997d0c144acb2e9 P a34c77c818054f87bc19600949531ad0: Neither blocks nor log segments found. Creating new log.
@ 0x7f4df629bd1c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260202 02:53:37.887513 5296 tablet_replica.cc:333] T c58cc255392e4a74b94c1cfae918c70e P a34c77c818054f87bc19600949531ad0: stopping tablet replica
I20260202 02:53:37.887725 5296 raft_consensus.cc:2243] T c58cc255392e4a74b94c1cfae918c70e P a34c77c818054f87bc19600949531ad0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:53:37.887974 5296 raft_consensus.cc:2272] T c58cc255392e4a74b94c1cfae918c70e P a34c77c818054f87bc19600949531ad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:53:37.888700 5296 ts_tablet_manager.cc:1916] T c58cc255392e4a74b94c1cfae918c70e P a34c77c818054f87bc19600949531ad0: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7f4df629bb99 std::_Function_handler<>::_M_invoke() at ??:0
@ 0xa8970d std::function<>::operator()() at ??:?
I20260202 02:53:37.892980 5293 raft_consensus.cc:2272] T c58cc255392e4a74b94c1cfae918c70e P 68f130042e0c4b8292b138ff8a322517 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:53:37.893815 5293 ts_tablet_manager.cc:1916] T c58cc255392e4a74b94c1cfae918c70e P 68f130042e0c4b8292b138ff8a322517: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:53:37.895697 5291 tablet_bootstrap.cc:492] T bb2b2a691a544f8a8997d0c144acb2e9 P 68f130042e0c4b8292b138ff8a322517: No bootstrap required, opened a new log
I20260202 02:53:37.895900 5291 ts_tablet_manager.cc:1403] T bb2b2a691a544f8a8997d0c144acb2e9 P 68f130042e0c4b8292b138ff8a322517: Time spent bootstrapping tablet: real 0.029s user 0.004s sys 0.008s
@ 0x7f4de2613f99 kudu::Thread::SuperviseThread() at ??:0
I20260202 02:53:37.897039 5291 raft_consensus.cc:359] T bb2b2a691a544f8a8997d0c144acb2e9 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:37.897260 5291 raft_consensus.cc:385] T bb2b2a691a544f8a8997d0c144acb2e9 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:37.897356 5291 raft_consensus.cc:740] T bb2b2a691a544f8a8997d0c144acb2e9 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f130042e0c4b8292b138ff8a322517, State: Initialized, Role: FOLLOWER
I20260202 02:53:37.898175 5291 consensus_queue.cc:260] T bb2b2a691a544f8a8997d0c144acb2e9 P 68f130042e0c4b8292b138ff8a322517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
@ 0x7f4de5c206db start_thread at ??:0
@ 0x7f4dde74f71f clone at ??:0
I20260202 02:53:37.901594 5301 tablet_bootstrap.cc:492] T bb2b2a691a544f8a8997d0c144acb2e9 P a34c77c818054f87bc19600949531ad0: No bootstrap required, opened a new log
I20260202 02:53:37.901758 5301 ts_tablet_manager.cc:1403] T bb2b2a691a544f8a8997d0c144acb2e9 P a34c77c818054f87bc19600949531ad0: Time spent bootstrapping tablet: real 0.026s user 0.004s sys 0.000s
I20260202 02:53:37.902669 5301 raft_consensus.cc:359] T bb2b2a691a544f8a8997d0c144acb2e9 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:37.902866 5301 raft_consensus.cc:385] T bb2b2a691a544f8a8997d0c144acb2e9 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:37.902949 5301 raft_consensus.cc:740] T bb2b2a691a544f8a8997d0c144acb2e9 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a34c77c818054f87bc19600949531ad0, State: Initialized, Role: FOLLOWER
I20260202 02:53:37.903250 5301 consensus_queue.cc:260] T bb2b2a691a544f8a8997d0c144acb2e9 P a34c77c818054f87bc19600949531ad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:37.903770 4096 heartbeater.cc:499] Master 127.0.104.188:42525 was elected leader, sending a full tablet report...
I20260202 02:53:37.905952 5301 ts_tablet_manager.cc:1434] T bb2b2a691a544f8a8997d0c144acb2e9 P a34c77c818054f87bc19600949531ad0: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260202 02:53:37.906453 5301 tablet_bootstrap.cc:492] T 4f3dd16be700418b80728adc2ae0cd85 P a34c77c818054f87bc19600949531ad0: Bootstrap starting.
I20260202 02:53:37.908912 5292 tablet_bootstrap.cc:492] T bb2b2a691a544f8a8997d0c144acb2e9 P d9570a47043b4891b2392af17f405e3d: Bootstrap starting.
I20260202 02:53:37.909737 5291 ts_tablet_manager.cc:1434] T bb2b2a691a544f8a8997d0c144acb2e9 P 68f130042e0c4b8292b138ff8a322517: Time spent starting tablet: real 0.014s user 0.003s sys 0.000s
I20260202 02:53:37.910094 5291 tablet_bootstrap.cc:492] T 4f3dd16be700418b80728adc2ae0cd85 P 68f130042e0c4b8292b138ff8a322517: Bootstrap starting.
I20260202 02:53:37.910935 5292 tablet_bootstrap.cc:654] T bb2b2a691a544f8a8997d0c144acb2e9 P d9570a47043b4891b2392af17f405e3d: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:37.911957 5291 tablet_bootstrap.cc:654] T 4f3dd16be700418b80728adc2ae0cd85 P 68f130042e0c4b8292b138ff8a322517: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:37.913488 4233 heartbeater.cc:344] Connected to a master server at 127.0.104.190:34577
I20260202 02:53:37.915196 5216 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "68f130042e0c4b8292b138ff8a322517" instance_seqno: 1770000797970665) as {username='slave'} at 127.0.104.131:50253; Asking this server to re-register.
I20260202 02:53:37.916105 4233 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:37.918365 5216 ts_manager.cc:194] Registered new tserver with Master: 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661)
I20260202 02:53:37.919370 5291 tablet_bootstrap.cc:492] T 4f3dd16be700418b80728adc2ae0cd85 P 68f130042e0c4b8292b138ff8a322517: No bootstrap required, opened a new log
I20260202 02:53:37.919559 5291 ts_tablet_manager.cc:1403] T 4f3dd16be700418b80728adc2ae0cd85 P 68f130042e0c4b8292b138ff8a322517: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260202 02:53:37.920647 5291 raft_consensus.cc:359] T 4f3dd16be700418b80728adc2ae0cd85 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:37.920881 5291 raft_consensus.cc:385] T 4f3dd16be700418b80728adc2ae0cd85 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:37.920964 5291 raft_consensus.cc:740] T 4f3dd16be700418b80728adc2ae0cd85 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f130042e0c4b8292b138ff8a322517, State: Initialized, Role: FOLLOWER
I20260202 02:53:37.921329 5291 consensus_queue.cc:260] T 4f3dd16be700418b80728adc2ae0cd85 P 68f130042e0c4b8292b138ff8a322517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:37.923027 5291 ts_tablet_manager.cc:1434] T 4f3dd16be700418b80728adc2ae0cd85 P 68f130042e0c4b8292b138ff8a322517: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260202 02:53:37.923379 5291 tablet_bootstrap.cc:492] T 5313e773f68d4ad2ad3e36880668051f P 68f130042e0c4b8292b138ff8a322517: Bootstrap starting.
I20260202 02:53:37.908288 5301 tablet_bootstrap.cc:654] T 4f3dd16be700418b80728adc2ae0cd85 P a34c77c818054f87bc19600949531ad0: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:37.925244 5291 tablet_bootstrap.cc:654] T 5313e773f68d4ad2ad3e36880668051f P 68f130042e0c4b8292b138ff8a322517: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:37.925662 5294 ts_tablet_manager.cc:1929] T c58cc255392e4a74b94c1cfae918c70e P d9570a47043b4891b2392af17f405e3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:53:37.925807 5294 log.cc:1199] T c58cc255392e4a74b94c1cfae918c70e P d9570a47043b4891b2392af17f405e3d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/c58cc255392e4a74b94c1cfae918c70e
I20260202 02:53:37.926267 5294 ts_tablet_manager.cc:1950] T c58cc255392e4a74b94c1cfae918c70e P d9570a47043b4891b2392af17f405e3d: Deleting consensus metadata
I20260202 02:53:37.928370 5292 tablet_bootstrap.cc:492] T bb2b2a691a544f8a8997d0c144acb2e9 P d9570a47043b4891b2392af17f405e3d: No bootstrap required, opened a new log
I20260202 02:53:37.928540 5292 ts_tablet_manager.cc:1403] T bb2b2a691a544f8a8997d0c144acb2e9 P d9570a47043b4891b2392af17f405e3d: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.000s
I20260202 02:53:37.929527 5292 raft_consensus.cc:359] T bb2b2a691a544f8a8997d0c144acb2e9 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:37.929720 5292 raft_consensus.cc:385] T bb2b2a691a544f8a8997d0c144acb2e9 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:37.929809 5292 raft_consensus.cc:740] T bb2b2a691a544f8a8997d0c144acb2e9 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9570a47043b4891b2392af17f405e3d, State: Initialized, Role: FOLLOWER
I20260202 02:53:37.931553 5293 ts_tablet_manager.cc:1929] T c58cc255392e4a74b94c1cfae918c70e P 68f130042e0c4b8292b138ff8a322517: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:53:37.931671 5293 log.cc:1199] T c58cc255392e4a74b94c1cfae918c70e P 68f130042e0c4b8292b138ff8a322517: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/c58cc255392e4a74b94c1cfae918c70e
I20260202 02:53:37.931979 5293 ts_tablet_manager.cc:1950] T c58cc255392e4a74b94c1cfae918c70e P 68f130042e0c4b8292b138ff8a322517: Deleting consensus metadata
I20260202 02:53:37.932075 5291 tablet_bootstrap.cc:492] T 5313e773f68d4ad2ad3e36880668051f P 68f130042e0c4b8292b138ff8a322517: No bootstrap required, opened a new log
I20260202 02:53:37.932240 5291 ts_tablet_manager.cc:1403] T 5313e773f68d4ad2ad3e36880668051f P 68f130042e0c4b8292b138ff8a322517: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.005s
I20260202 02:53:37.933121 5291 raft_consensus.cc:359] T 5313e773f68d4ad2ad3e36880668051f P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:37.933329 5291 raft_consensus.cc:385] T 5313e773f68d4ad2ad3e36880668051f P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:37.933442 5291 raft_consensus.cc:740] T 5313e773f68d4ad2ad3e36880668051f P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f130042e0c4b8292b138ff8a322517, State: Initialized, Role: FOLLOWER
I20260202 02:53:37.933676 5291 consensus_queue.cc:260] T 5313e773f68d4ad2ad3e36880668051f P 68f130042e0c4b8292b138ff8a322517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:37.933935 5296 ts_tablet_manager.cc:1929] T c58cc255392e4a74b94c1cfae918c70e P a34c77c818054f87bc19600949531ad0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:53:37.934069 5296 log.cc:1199] T c58cc255392e4a74b94c1cfae918c70e P a34c77c818054f87bc19600949531ad0: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/c58cc255392e4a74b94c1cfae918c70e
I20260202 02:53:37.934587 5296 ts_tablet_manager.cc:1950] T c58cc255392e4a74b94c1cfae918c70e P a34c77c818054f87bc19600949531ad0: Deleting consensus metadata
I20260202 02:53:37.935463 5291 ts_tablet_manager.cc:1434] T 5313e773f68d4ad2ad3e36880668051f P 68f130042e0c4b8292b138ff8a322517: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260202 02:53:37.935806 5292 consensus_queue.cc:260] T bb2b2a691a544f8a8997d0c144acb2e9 P d9570a47043b4891b2392af17f405e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:37.943606 5301 tablet_bootstrap.cc:492] T 4f3dd16be700418b80728adc2ae0cd85 P a34c77c818054f87bc19600949531ad0: No bootstrap required, opened a new log
I20260202 02:53:37.943820 5301 ts_tablet_manager.cc:1403] T 4f3dd16be700418b80728adc2ae0cd85 P a34c77c818054f87bc19600949531ad0: Time spent bootstrapping tablet: real 0.037s user 0.003s sys 0.000s
I20260202 02:53:37.944772 5301 raft_consensus.cc:359] T 4f3dd16be700418b80728adc2ae0cd85 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:37.945011 5301 raft_consensus.cc:385] T 4f3dd16be700418b80728adc2ae0cd85 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:37.945086 5301 raft_consensus.cc:740] T 4f3dd16be700418b80728adc2ae0cd85 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a34c77c818054f87bc19600949531ad0, State: Initialized, Role: FOLLOWER
I20260202 02:53:37.945358 5301 consensus_queue.cc:260] T 4f3dd16be700418b80728adc2ae0cd85 P a34c77c818054f87bc19600949531ad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:37.946324 3955 heartbeater.cc:344] Connected to a master server at 127.0.104.190:34577
I20260202 02:53:37.946658 5301 ts_tablet_manager.cc:1434] T 4f3dd16be700418b80728adc2ae0cd85 P a34c77c818054f87bc19600949531ad0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260202 02:53:37.946990 5301 tablet_bootstrap.cc:492] T 5313e773f68d4ad2ad3e36880668051f P a34c77c818054f87bc19600949531ad0: Bootstrap starting.
I20260202 02:53:37.947866 5215 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "d9570a47043b4891b2392af17f405e3d" instance_seqno: 1770000796958858) as {username='slave'} at 127.0.104.129:45825; Asking this server to re-register.
I20260202 02:53:37.948534 3955 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:37.949749 5292 ts_tablet_manager.cc:1434] T bb2b2a691a544f8a8997d0c144acb2e9 P d9570a47043b4891b2392af17f405e3d: Time spent starting tablet: real 0.021s user 0.002s sys 0.000s
I20260202 02:53:37.950124 5292 tablet_bootstrap.cc:492] T 4f3dd16be700418b80728adc2ae0cd85 P d9570a47043b4891b2392af17f405e3d: Bootstrap starting.
I20260202 02:53:37.950246 5215 ts_manager.cc:194] Registered new tserver with Master: d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425)
I20260202 02:53:37.951961 5292 tablet_bootstrap.cc:654] T 4f3dd16be700418b80728adc2ae0cd85 P d9570a47043b4891b2392af17f405e3d: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:37.952821 5301 tablet_bootstrap.cc:654] T 5313e773f68d4ad2ad3e36880668051f P a34c77c818054f87bc19600949531ad0: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:37.961812 5292 tablet_bootstrap.cc:492] T 4f3dd16be700418b80728adc2ae0cd85 P d9570a47043b4891b2392af17f405e3d: No bootstrap required, opened a new log
I20260202 02:53:37.962013 5292 ts_tablet_manager.cc:1403] T 4f3dd16be700418b80728adc2ae0cd85 P d9570a47043b4891b2392af17f405e3d: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.003s
I20260202 02:53:37.962905 5292 raft_consensus.cc:359] T 4f3dd16be700418b80728adc2ae0cd85 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:37.963135 5292 raft_consensus.cc:385] T 4f3dd16be700418b80728adc2ae0cd85 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:37.963228 5292 raft_consensus.cc:740] T 4f3dd16be700418b80728adc2ae0cd85 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9570a47043b4891b2392af17f405e3d, State: Initialized, Role: FOLLOWER
I20260202 02:53:37.963565 5292 consensus_queue.cc:260] T 4f3dd16be700418b80728adc2ae0cd85 P d9570a47043b4891b2392af17f405e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:37.964143 5292 ts_tablet_manager.cc:1434] T 4f3dd16be700418b80728adc2ae0cd85 P d9570a47043b4891b2392af17f405e3d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260202 02:53:37.964926 5292 tablet_bootstrap.cc:492] T 5313e773f68d4ad2ad3e36880668051f P d9570a47043b4891b2392af17f405e3d: Bootstrap starting.
I20260202 02:53:37.967490 5292 tablet_bootstrap.cc:654] T 5313e773f68d4ad2ad3e36880668051f P d9570a47043b4891b2392af17f405e3d: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:37.969756 5301 tablet_bootstrap.cc:492] T 5313e773f68d4ad2ad3e36880668051f P a34c77c818054f87bc19600949531ad0: No bootstrap required, opened a new log
I20260202 02:53:37.969913 5301 ts_tablet_manager.cc:1403] T 5313e773f68d4ad2ad3e36880668051f P a34c77c818054f87bc19600949531ad0: Time spent bootstrapping tablet: real 0.023s user 0.007s sys 0.009s
I20260202 02:53:37.970721 5301 raft_consensus.cc:359] T 5313e773f68d4ad2ad3e36880668051f P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:37.970897 5301 raft_consensus.cc:385] T 5313e773f68d4ad2ad3e36880668051f P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:37.970970 5301 raft_consensus.cc:740] T 5313e773f68d4ad2ad3e36880668051f P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a34c77c818054f87bc19600949531ad0, State: Initialized, Role: FOLLOWER
I20260202 02:53:37.971199 5301 consensus_queue.cc:260] T 5313e773f68d4ad2ad3e36880668051f P a34c77c818054f87bc19600949531ad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:37.974031 5301 ts_tablet_manager.cc:1434] T 5313e773f68d4ad2ad3e36880668051f P a34c77c818054f87bc19600949531ad0: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260202 02:53:37.999264 4515 raft_consensus.cc:493] T bb2b2a691a544f8a8997d0c144acb2e9 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:37.999460 4515 raft_consensus.cc:515] T bb2b2a691a544f8a8997d0c144acb2e9 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:38.000387 4515 leader_election.cc:290] T bb2b2a691a544f8a8997d0c144acb2e9 P a34c77c818054f87bc19600949531ad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661), d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425)
I20260202 02:53:38.001183 3909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bb2b2a691a544f8a8997d0c144acb2e9" candidate_uuid: "a34c77c818054f87bc19600949531ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9570a47043b4891b2392af17f405e3d" is_pre_election: true
I20260202 02:53:38.001458 3909 raft_consensus.cc:2468] T bb2b2a691a544f8a8997d0c144acb2e9 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c77c818054f87bc19600949531ad0 in term 0.
I20260202 02:53:38.002421 4187 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bb2b2a691a544f8a8997d0c144acb2e9" candidate_uuid: "a34c77c818054f87bc19600949531ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f130042e0c4b8292b138ff8a322517" is_pre_election: true
I20260202 02:53:38.002475 3985 leader_election.cc:304] T bb2b2a691a544f8a8997d0c144acb2e9 P a34c77c818054f87bc19600949531ad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c77c818054f87bc19600949531ad0, d9570a47043b4891b2392af17f405e3d; no voters:
I20260202 02:53:38.002686 4187 raft_consensus.cc:2468] T bb2b2a691a544f8a8997d0c144acb2e9 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c77c818054f87bc19600949531ad0 in term 0.
I20260202 02:53:38.002832 4515 raft_consensus.cc:2804] T bb2b2a691a544f8a8997d0c144acb2e9 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:53:38.002933 4515 raft_consensus.cc:493] T bb2b2a691a544f8a8997d0c144acb2e9 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:53:38.003006 4515 raft_consensus.cc:3060] T bb2b2a691a544f8a8997d0c144acb2e9 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:38.005749 5022 raft_consensus.cc:493] T 4f3dd16be700418b80728adc2ae0cd85 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:38.005965 5022 raft_consensus.cc:515] T 4f3dd16be700418b80728adc2ae0cd85 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:38.005972 4515 raft_consensus.cc:515] T bb2b2a691a544f8a8997d0c144acb2e9 P a34c77c818054f87bc19600949531ad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:38.007193 4187 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bb2b2a691a544f8a8997d0c144acb2e9" candidate_uuid: "a34c77c818054f87bc19600949531ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f130042e0c4b8292b138ff8a322517"
I20260202 02:53:38.007247 5022 leader_election.cc:290] T 4f3dd16be700418b80728adc2ae0cd85 P d9570a47043b4891b2392af17f405e3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661), a34c77c818054f87bc19600949531ad0 (127.0.104.130:38525)
I20260202 02:53:38.007411 4187 raft_consensus.cc:3060] T bb2b2a691a544f8a8997d0c144acb2e9 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:38.008724 4186 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f3dd16be700418b80728adc2ae0cd85" candidate_uuid: "d9570a47043b4891b2392af17f405e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f130042e0c4b8292b138ff8a322517" is_pre_election: true
I20260202 02:53:38.009110 4186 raft_consensus.cc:2468] T 4f3dd16be700418b80728adc2ae0cd85 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9570a47043b4891b2392af17f405e3d in term 0.
I20260202 02:53:38.009771 3840 leader_election.cc:304] T 4f3dd16be700418b80728adc2ae0cd85 P d9570a47043b4891b2392af17f405e3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68f130042e0c4b8292b138ff8a322517, d9570a47043b4891b2392af17f405e3d; no voters:
I20260202 02:53:38.010175 5022 raft_consensus.cc:2804] T 4f3dd16be700418b80728adc2ae0cd85 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:53:38.010293 5022 raft_consensus.cc:493] T 4f3dd16be700418b80728adc2ae0cd85 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:53:38.010380 5022 raft_consensus.cc:3060] T 4f3dd16be700418b80728adc2ae0cd85 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:38.010445 4187 raft_consensus.cc:2468] T bb2b2a691a544f8a8997d0c144acb2e9 P 68f130042e0c4b8292b138ff8a322517 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c77c818054f87bc19600949531ad0 in term 1.
I20260202 02:53:38.011005 4051 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f3dd16be700418b80728adc2ae0cd85" candidate_uuid: "d9570a47043b4891b2392af17f405e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a34c77c818054f87bc19600949531ad0" is_pre_election: true
I20260202 02:53:38.011242 4051 raft_consensus.cc:2468] T 4f3dd16be700418b80728adc2ae0cd85 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9570a47043b4891b2392af17f405e3d in term 0.
I20260202 02:53:38.011662 3984 leader_election.cc:304] T bb2b2a691a544f8a8997d0c144acb2e9 P a34c77c818054f87bc19600949531ad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68f130042e0c4b8292b138ff8a322517, a34c77c818054f87bc19600949531ad0; no voters:
I20260202 02:53:38.011983 4544 raft_consensus.cc:2804] T bb2b2a691a544f8a8997d0c144acb2e9 P a34c77c818054f87bc19600949531ad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:53:38.012127 4544 raft_consensus.cc:697] T bb2b2a691a544f8a8997d0c144acb2e9 P a34c77c818054f87bc19600949531ad0 [term 1 LEADER]: Becoming Leader. State: Replica: a34c77c818054f87bc19600949531ad0, State: Running, Role: LEADER
I20260202 02:53:38.012483 4544 consensus_queue.cc:237] T bb2b2a691a544f8a8997d0c144acb2e9 P a34c77c818054f87bc19600949531ad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:38.015275 5022 raft_consensus.cc:515] T 4f3dd16be700418b80728adc2ae0cd85 P d9570a47043b4891b2392af17f405e3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:38.015944 4515 leader_election.cc:290] T bb2b2a691a544f8a8997d0c144acb2e9 P a34c77c818054f87bc19600949531ad0 [CANDIDATE]: Term 1 election: Requested vote from peers 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661), d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425)
I20260202 02:53:38.016561 3909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bb2b2a691a544f8a8997d0c144acb2e9" candidate_uuid: "a34c77c818054f87bc19600949531ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9570a47043b4891b2392af17f405e3d"
I20260202 02:53:38.016788 3909 raft_consensus.cc:3060] T bb2b2a691a544f8a8997d0c144acb2e9 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:38.018106 5292 tablet_bootstrap.cc:492] T 5313e773f68d4ad2ad3e36880668051f P d9570a47043b4891b2392af17f405e3d: No bootstrap required, opened a new log
I20260202 02:53:38.018477 5292 ts_tablet_manager.cc:1403] T 5313e773f68d4ad2ad3e36880668051f P d9570a47043b4891b2392af17f405e3d: Time spent bootstrapping tablet: real 0.054s user 0.020s sys 0.025s
I20260202 02:53:38.019295 5292 raft_consensus.cc:359] T 5313e773f68d4ad2ad3e36880668051f P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:38.019498 5292 raft_consensus.cc:385] T 5313e773f68d4ad2ad3e36880668051f P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:38.019577 3909 raft_consensus.cc:2468] T bb2b2a691a544f8a8997d0c144acb2e9 P d9570a47043b4891b2392af17f405e3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c77c818054f87bc19600949531ad0 in term 1.
I20260202 02:53:38.019598 5292 raft_consensus.cc:740] T 5313e773f68d4ad2ad3e36880668051f P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9570a47043b4891b2392af17f405e3d, State: Initialized, Role: FOLLOWER
I20260202 02:53:38.019876 5292 consensus_queue.cc:260] T 5313e773f68d4ad2ad3e36880668051f P d9570a47043b4891b2392af17f405e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:38.021380 4187 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f3dd16be700418b80728adc2ae0cd85" candidate_uuid: "d9570a47043b4891b2392af17f405e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f130042e0c4b8292b138ff8a322517"
I20260202 02:53:38.021618 4187 raft_consensus.cc:3060] T 4f3dd16be700418b80728adc2ae0cd85 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:38.021930 5022 leader_election.cc:290] T 4f3dd16be700418b80728adc2ae0cd85 P d9570a47043b4891b2392af17f405e3d [CANDIDATE]: Term 1 election: Requested vote from peers 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661), a34c77c818054f87bc19600949531ad0 (127.0.104.130:38525)
I20260202 02:53:38.021974 4051 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f3dd16be700418b80728adc2ae0cd85" candidate_uuid: "d9570a47043b4891b2392af17f405e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a34c77c818054f87bc19600949531ad0"
I20260202 02:53:38.022181 4051 raft_consensus.cc:3060] T 4f3dd16be700418b80728adc2ae0cd85 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:38.025075 4051 raft_consensus.cc:2468] T 4f3dd16be700418b80728adc2ae0cd85 P a34c77c818054f87bc19600949531ad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9570a47043b4891b2392af17f405e3d in term 1.
I20260202 02:53:38.025662 3841 leader_election.cc:304] T 4f3dd16be700418b80728adc2ae0cd85 P d9570a47043b4891b2392af17f405e3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c77c818054f87bc19600949531ad0, d9570a47043b4891b2392af17f405e3d; no voters:
I20260202 02:53:38.026082 5022 raft_consensus.cc:2804] T 4f3dd16be700418b80728adc2ae0cd85 P d9570a47043b4891b2392af17f405e3d [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:53:38.026257 5022 raft_consensus.cc:697] T 4f3dd16be700418b80728adc2ae0cd85 P d9570a47043b4891b2392af17f405e3d [term 1 LEADER]: Becoming Leader. State: Replica: d9570a47043b4891b2392af17f405e3d, State: Running, Role: LEADER
I20260202 02:53:38.026609 5022 consensus_queue.cc:237] T 4f3dd16be700418b80728adc2ae0cd85 P d9570a47043b4891b2392af17f405e3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:38.028028 4187 raft_consensus.cc:2468] T 4f3dd16be700418b80728adc2ae0cd85 P 68f130042e0c4b8292b138ff8a322517 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9570a47043b4891b2392af17f405e3d in term 1.
I20260202 02:53:38.041002 5292 ts_tablet_manager.cc:1434] T 5313e773f68d4ad2ad3e36880668051f P d9570a47043b4891b2392af17f405e3d: Time spent starting tablet: real 0.022s user 0.001s sys 0.007s
W20260202 02:53:38.174167 4120 connection.cc:537] server connection from 127.0.0.1:52846 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260202 02:53:38.174203 3987 connection.cc:537] server connection from 127.0.0.1:38554 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260202 02:53:38.174556 3843 connection.cc:537] client connection to 127.0.104.188:42525 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260202 02:53:38.174726 3842 connection.cc:537] server connection from 127.0.0.1:33434 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260202 02:53:38.176370 4242 connection.cc:537] client connection to 127.0.104.188:42525 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260202 02:53:38.176944 4242 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.104.188:42525)
W20260202 02:53:38.177515 4242 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.104.188:42525)
W20260202 02:53:38.177736 4242 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.104.188:42525)
W20260202 02:53:38.178045 5313 negotiation.cc:337] Failed RPC negotiation. Trace:
0202 02:53:38.175609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.104.188:42525 (local address 127.0.104.129:58725)
0202 02:53:38.175875 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0202 02:53:38.175897 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0202 02:53:38.176034 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0202 02:53:38.177728 (+ 1694us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":151}
W20260202 02:53:38.178181 5175 negotiation.cc:337] Failed RPC negotiation. Trace:
0202 02:53:38.176268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.104.188:42525 (local address 127.0.104.131:35435)
0202 02:53:38.176510 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0202 02:53:38.176531 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0202 02:53:38.176657 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0202 02:53:38.177940 (+ 1283us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":92}
W20260202 02:53:38.178746 3987 connection.cc:537] client connection to 127.0.104.188:42525 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260202 02:53:38.338464 5324 raft_consensus.cc:493] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 879caa0e6c114ab4855b7a756e252c9e)
I20260202 02:53:38.338653 5324 raft_consensus.cc:515] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:38.340621 5324 leader_election.cc:290] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:53:38.341575 5201 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
W20260202 02:53:38.345880 5201 leader_election.cc:336] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361): Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
W20260202 02:53:38.346232 5203 leader_election.cc:336] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:38.346874 5203 leader_election.cc:304] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666b7e52b5fc4f3397f04960c8b67100; no voters: 3acf71c31a284578b4f8c918eaa50205, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:38.347321 5324 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260202 02:53:38.778858 4515 raft_consensus.cc:493] T 5313e773f68d4ad2ad3e36880668051f P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:38.779057 4515 raft_consensus.cc:515] T 5313e773f68d4ad2ad3e36880668051f P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:38.779762 4515 leader_election.cc:290] T 5313e773f68d4ad2ad3e36880668051f P a34c77c818054f87bc19600949531ad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661), d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425)
I20260202 02:53:38.780290 4187 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5313e773f68d4ad2ad3e36880668051f" candidate_uuid: "a34c77c818054f87bc19600949531ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f130042e0c4b8292b138ff8a322517" is_pre_election: true
I20260202 02:53:38.780431 3909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5313e773f68d4ad2ad3e36880668051f" candidate_uuid: "a34c77c818054f87bc19600949531ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9570a47043b4891b2392af17f405e3d" is_pre_election: true
I20260202 02:53:38.780550 4187 raft_consensus.cc:2468] T 5313e773f68d4ad2ad3e36880668051f P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c77c818054f87bc19600949531ad0 in term 0.
I20260202 02:53:38.780670 3909 raft_consensus.cc:2468] T 5313e773f68d4ad2ad3e36880668051f P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c77c818054f87bc19600949531ad0 in term 0.
I20260202 02:53:38.780998 3984 leader_election.cc:304] T 5313e773f68d4ad2ad3e36880668051f P a34c77c818054f87bc19600949531ad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68f130042e0c4b8292b138ff8a322517, a34c77c818054f87bc19600949531ad0; no voters:
I20260202 02:53:38.781325 4515 raft_consensus.cc:2804] T 5313e773f68d4ad2ad3e36880668051f P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:53:38.781443 4515 raft_consensus.cc:493] T 5313e773f68d4ad2ad3e36880668051f P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:53:38.781533 4515 raft_consensus.cc:3060] T 5313e773f68d4ad2ad3e36880668051f P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:38.783461 4515 raft_consensus.cc:515] T 5313e773f68d4ad2ad3e36880668051f P a34c77c818054f87bc19600949531ad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:38.784031 4515 leader_election.cc:290] T 5313e773f68d4ad2ad3e36880668051f P a34c77c818054f87bc19600949531ad0 [CANDIDATE]: Term 1 election: Requested vote from peers 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661), d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425)
I20260202 02:53:38.784621 4187 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5313e773f68d4ad2ad3e36880668051f" candidate_uuid: "a34c77c818054f87bc19600949531ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f130042e0c4b8292b138ff8a322517"
I20260202 02:53:38.784621 3909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5313e773f68d4ad2ad3e36880668051f" candidate_uuid: "a34c77c818054f87bc19600949531ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9570a47043b4891b2392af17f405e3d"
I20260202 02:53:38.784822 3909 raft_consensus.cc:3060] T 5313e773f68d4ad2ad3e36880668051f P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:38.784822 4187 raft_consensus.cc:3060] T 5313e773f68d4ad2ad3e36880668051f P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:38.787477 4187 raft_consensus.cc:2468] T 5313e773f68d4ad2ad3e36880668051f P 68f130042e0c4b8292b138ff8a322517 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c77c818054f87bc19600949531ad0 in term 1.
I20260202 02:53:38.787475 3909 raft_consensus.cc:2468] T 5313e773f68d4ad2ad3e36880668051f P d9570a47043b4891b2392af17f405e3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c77c818054f87bc19600949531ad0 in term 1.
I20260202 02:53:38.787971 3985 leader_election.cc:304] T 5313e773f68d4ad2ad3e36880668051f P a34c77c818054f87bc19600949531ad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c77c818054f87bc19600949531ad0, d9570a47043b4891b2392af17f405e3d; no voters:
I20260202 02:53:38.788362 4515 raft_consensus.cc:2804] T 5313e773f68d4ad2ad3e36880668051f P a34c77c818054f87bc19600949531ad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:53:38.788530 4515 raft_consensus.cc:697] T 5313e773f68d4ad2ad3e36880668051f P a34c77c818054f87bc19600949531ad0 [term 1 LEADER]: Becoming Leader. State: Replica: a34c77c818054f87bc19600949531ad0, State: Running, Role: LEADER
I20260202 02:53:38.788847 4515 consensus_queue.cc:237] T 5313e773f68d4ad2ad3e36880668051f P a34c77c818054f87bc19600949531ad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:38.908452 5330 raft_consensus.cc:493] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 879caa0e6c114ab4855b7a756e252c9e)
I20260202 02:53:38.908654 5330 raft_consensus.cc:515] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:38.909387 5330 leader_election.cc:290] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:53:38.911370 5201 leader_election.cc:336] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361): Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
W20260202 02:53:38.911550 5203 leader_election.cc:336] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:38.911661 5203 leader_election.cc:304] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666b7e52b5fc4f3397f04960c8b67100; no voters: 3acf71c31a284578b4f8c918eaa50205, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:38.911928 5330 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260202 02:53:39.013382 5022 consensus_queue.cc:1048] T 4f3dd16be700418b80728adc2ae0cd85 P d9570a47043b4891b2392af17f405e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:53:39.019748 5022 consensus_queue.cc:1048] T 4f3dd16be700418b80728adc2ae0cd85 P d9570a47043b4891b2392af17f405e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:53:39.109365 5342 consensus_queue.cc:1048] T bb2b2a691a544f8a8997d0c144acb2e9 P a34c77c818054f87bc19600949531ad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:53:39.115990 5342 consensus_queue.cc:1048] T bb2b2a691a544f8a8997d0c144acb2e9 P a34c77c818054f87bc19600949531ad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:53:39.572041 4544 consensus_queue.cc:1048] T 5313e773f68d4ad2ad3e36880668051f P a34c77c818054f87bc19600949531ad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:53:39.578541 4544 consensus_queue.cc:1048] T 5313e773f68d4ad2ad3e36880668051f P a34c77c818054f87bc19600949531ad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:53:39.622896 5351 raft_consensus.cc:493] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 879caa0e6c114ab4855b7a756e252c9e)
I20260202 02:53:39.623121 5351 raft_consensus.cc:515] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:39.623818 5351 leader_election.cc:290] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:53:39.625888 5203 leader_election.cc:336] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
W20260202 02:53:39.626042 5201 leader_election.cc:336] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361): Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
I20260202 02:53:39.626199 5201 leader_election.cc:304] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666b7e52b5fc4f3397f04960c8b67100; no voters: 3acf71c31a284578b4f8c918eaa50205, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:39.626516 5351 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260202 02:53:40.026160 418 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskYXxklX/build/asan/bin/kudu pid 5183
I20260202 02:53:40.036743 5246 generic_service.cc:196] Checking for leaks (request via RPC)
I20260202 02:53:40.150988 418 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskYXxklX/build/asan/bin/kudu with pid 5183
W20260202 02:53:40.199626 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 11: this message will repeat every 5th retry.
W20260202 02:53:40.319244 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 11: this message will repeat every 5th retry.
W20260202 02:53:40.505117 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 11: this message will repeat every 5th retry.
I20260202 02:53:40.672658 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskYXxklX/build/asan/bin/kudu
/tmp/dist-test-taskYXxklX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.190:34577
--webserver_interface=127.0.104.190
--webserver_port=33649
--builtin_ntp_servers=127.0.104.148:34531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525
--hive_metastore_uris=thrift://127.0.0.1:46545
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260202 02:53:40.750226 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 11: this message will repeat every 5th retry.
W20260202 02:53:41.028764 5369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260202 02:53:41.029181 5369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260202 02:53:41.029286 5369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260202 02:53:41.038542 5369 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260202 02:53:41.038650 5369 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260202 02:53:41.038712 5369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260202 02:53:41.038774 5369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260202 02:53:41.038854 5369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260202 02:53:41.038923 5369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260202 02:53:41.050309 5369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:34531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:46545
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.190:34577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.104.190
--webserver_port=33649
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3722ed49c19ce54c7feae2fa944e32b1d4ed8d2d
build type FASTDEBUG
built by None at 02 Feb 2026 02:43:23 UTC on 24a791456cd2
build id 10304
ASAN enabled
I20260202 02:53:41.052026 5369 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260202 02:53:41.053977 5369 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260202 02:53:41.062769 5375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:53:41.062556 5376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:53:41.063410 5378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260202 02:53:41.063508 5369 server_base.cc:1060] running on GCE node
I20260202 02:53:41.064384 5369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260202 02:53:41.065909 5369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260202 02:53:41.067193 5369 hybrid_clock.cc:648] HybridClock initialized: now 1770000821067085 us; error 68 us; skew 500 ppm
I20260202 02:53:41.067559 5369 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260202 02:53:41.070025 5369 webserver.cc:492] Webserver started at http://127.0.104.190:33649/ using document root <none> and password file <none>
I20260202 02:53:41.070847 5369 fs_manager.cc:362] Metadata directory not provided
I20260202 02:53:41.070972 5369 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260202 02:53:41.077024 5369 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260202 02:53:41.080422 5384 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260202 02:53:41.082329 5369 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260202 02:53:41.082568 5369 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data,/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/wal
uuid: "666b7e52b5fc4f3397f04960c8b67100"
format_stamp: "Formatted at 2026-02-02 02:53:14 on dist-test-slave-llj2"
I20260202 02:53:41.083413 5369 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260202 02:53:41.102097 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 11: this message will repeat every 5th retry.
I20260202 02:53:41.116060 5369 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260202 02:53:41.117113 5369 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260202 02:53:41.117441 5369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260202 02:53:41.137773 5369 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.190:34577
I20260202 02:53:41.137827 5437 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.190:34577 every 8 connection(s)
I20260202 02:53:41.139473 5369 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-0/data/info.pb
I20260202 02:53:41.146818 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskYXxklX/build/asan/bin/kudu as pid 5369
I20260202 02:53:41.147034 5438 sys_catalog.cc:263] Verifying existing consensus state
I20260202 02:53:41.151083 5438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: Bootstrap starting.
I20260202 02:53:41.163566 4233 heartbeater.cc:344] Connected to a master server at 127.0.104.190:34577
I20260202 02:53:41.166633 5438 log.cc:826] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: Log is configured to *not* fsync() on all Append() calls
I20260202 02:53:41.200775 5438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: Bootstrap replayed 1/1 log segments. Stats: ops{read=39 overwritten=0 applied=39 ignored=0} inserts{seen=21 ignored=0} mutations{seen=30 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260202 02:53:41.201372 5438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: Bootstrap complete.
I20260202 02:53:41.206867 5438 raft_consensus.cc:359] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:41.207641 5438 raft_consensus.cc:740] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666b7e52b5fc4f3397f04960c8b67100, State: Initialized, Role: FOLLOWER
I20260202 02:53:41.208422 5438 consensus_queue.cc:260] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 39, Last appended: 8.39, Last appended by leader: 39, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:41.209331 5443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 8 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:41.209645 5443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:53:41.210434 5438 sys_catalog.cc:565] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: configured and running, proceeding with master startup.
I20260202 02:53:41.223493 5457 catalog_manager.cc:1269] Loaded cluster ID: 0d95b889b53247a0813a4fda5550ae30
I20260202 02:53:41.223754 5457 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: loading cluster ID for follower catalog manager: success
I20260202 02:53:41.229583 5457 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: acquiring CA information for follower catalog manager: success
I20260202 02:53:41.233261 5457 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260202 02:53:41.244417 5369 master_runner.cc:186] Error getting master registration for 127.0.104.189:40361: Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111),
I20260202 02:53:41.248057 5369 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:53:41.292282 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 11: this message will repeat every 5th retry.
I20260202 02:53:41.299150 4097 heartbeater.cc:344] Connected to a master server at 127.0.104.190:34577
I20260202 02:53:41.301901 5403 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "a34c77c818054f87bc19600949531ad0" instance_seqno: 1770000797481684) as {username='slave'} at 127.0.104.130:38487; Asking this server to re-register.
I20260202 02:53:41.302673 4097 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:41.304667 5403 ts_manager.cc:194] Registered new tserver with Master: a34c77c818054f87bc19600949531ad0 (127.0.104.130:38525)
I20260202 02:53:41.520655 3955 heartbeater.cc:344] Connected to a master server at 127.0.104.190:34577
I20260202 02:53:41.522131 5403 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "d9570a47043b4891b2392af17f405e3d" instance_seqno: 1770000796958858) as {username='slave'} at 127.0.104.129:59723; Asking this server to re-register.
I20260202 02:53:41.522748 3955 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:41.524134 5403 ts_manager.cc:194] Registered new tserver with Master: d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425)
I20260202 02:53:41.668298 5403 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "68f130042e0c4b8292b138ff8a322517" instance_seqno: 1770000797970665) as {username='slave'} at 127.0.104.131:33109; Asking this server to re-register.
I20260202 02:53:41.669245 4233 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:41.670786 5403 ts_manager.cc:194] Registered new tserver with Master: 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661)
I20260202 02:53:41.802757 5466 raft_consensus.cc:493] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:41.803072 5466 raft_consensus.cc:515] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:41.805226 5466 leader_election.cc:290] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:53:41.806332 5387 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
W20260202 02:53:41.807958 5389 leader_election.cc:336] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
W20260202 02:53:41.808171 5387 leader_election.cc:336] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361): Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
I20260202 02:53:41.808373 5387 leader_election.cc:304] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666b7e52b5fc4f3397f04960c8b67100; no voters: 3acf71c31a284578b4f8c918eaa50205, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:41.808751 5466 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260202 02:53:42.371665 5473 raft_consensus.cc:493] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:42.371896 5473 raft_consensus.cc:515] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:42.372635 5473 leader_election.cc:290] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:53:42.374876 5387 leader_election.cc:336] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361): Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
W20260202 02:53:42.375057 5389 leader_election.cc:336] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:42.375186 5389 leader_election.cc:304] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666b7e52b5fc4f3397f04960c8b67100; no voters: 3acf71c31a284578b4f8c918eaa50205, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:42.375434 5473 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260202 02:53:43.060423 5483 raft_consensus.cc:493] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:43.060672 5483 raft_consensus.cc:515] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:43.061281 5483 leader_election.cc:290] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:53:43.063076 5387 leader_election.cc:336] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361): Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
W20260202 02:53:43.063227 5389 leader_election.cc:336] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:43.063385 5389 leader_election.cc:304] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666b7e52b5fc4f3397f04960c8b67100; no voters: 3acf71c31a284578b4f8c918eaa50205, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:43.063644 5483 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260202 02:53:44.068413 5500 raft_consensus.cc:493] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:44.068591 5500 raft_consensus.cc:515] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:44.069185 5500 leader_election.cc:290] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:53:44.071256 5389 leader_election.cc:336] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
W20260202 02:53:44.071638 5387 leader_election.cc:336] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361): Network error: Client connection negotiation failed: client connection to 127.0.104.189:40361: connect: Connection refused (error 111)
I20260202 02:53:44.071784 5387 leader_election.cc:304] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666b7e52b5fc4f3397f04960c8b67100; no voters: 3acf71c31a284578b4f8c918eaa50205, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:44.072017 5500 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260202 02:53:44.407354 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskYXxklX/build/asan/bin/kudu
/tmp/dist-test-taskYXxklX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.189:40361
--webserver_interface=127.0.104.189
--webserver_port=35647
--builtin_ntp_servers=127.0.104.148:34531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525
--hive_metastore_uris=thrift://127.0.0.1:46545
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260202 02:53:44.760783 5510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260202 02:53:44.761119 5510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260202 02:53:44.761231 5510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260202 02:53:44.769747 5510 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260202 02:53:44.769863 5510 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260202 02:53:44.769945 5510 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260202 02:53:44.770011 5510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260202 02:53:44.770097 5510 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260202 02:53:44.770180 5510 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260202 02:53:44.780550 5510 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:34531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:46545
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.189:40361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.104.189
--webserver_port=35647
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3722ed49c19ce54c7feae2fa944e32b1d4ed8d2d
build type FASTDEBUG
built by None at 02 Feb 2026 02:43:23 UTC on 24a791456cd2
build id 10304
ASAN enabled
I20260202 02:53:44.782262 5510 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260202 02:53:44.784108 5510 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260202 02:53:44.792582 5519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:53:44.792550 5520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:53:44.793213 5522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260202 02:53:44.793848 5510 server_base.cc:1060] running on GCE node
I20260202 02:53:44.794628 5510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260202 02:53:44.796176 5510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260202 02:53:44.797461 5510 hybrid_clock.cc:648] HybridClock initialized: now 1770000824797411 us; error 78 us; skew 500 ppm
I20260202 02:53:44.797860 5510 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260202 02:53:44.800585 5510 webserver.cc:492] Webserver started at http://127.0.104.189:35647/ using document root <none> and password file <none>
I20260202 02:53:44.801381 5510 fs_manager.cc:362] Metadata directory not provided
I20260202 02:53:44.801515 5510 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260202 02:53:44.808152 5510 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260202 02:53:44.813493 5528 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260202 02:53:44.815376 5510 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260202 02:53:44.815588 5510 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data,/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/wal
uuid: "3acf71c31a284578b4f8c918eaa50205"
format_stamp: "Formatted at 2026-02-02 02:53:15 on dist-test-slave-llj2"
I20260202 02:53:44.816411 5510 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260202 02:53:44.838881 5510 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260202 02:53:44.839829 5510 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260202 02:53:44.840137 5510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260202 02:53:44.860791 5510 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.189:40361
I20260202 02:53:44.860803 5579 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.189:40361 every 8 connection(s)
I20260202 02:53:44.862938 5510 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data/info.pb
I20260202 02:53:44.870609 5580 sys_catalog.cc:263] Verifying existing consensus state
I20260202 02:53:44.872247 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskYXxklX/build/asan/bin/kudu as pid 5510
I20260202 02:53:44.874564 5580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: Bootstrap starting.
I20260202 02:53:44.888783 5580 log.cc:826] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: Log is configured to *not* fsync() on all Append() calls
I20260202 02:53:44.917104 5580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: Bootstrap replayed 1/1 log segments. Stats: ops{read=32 overwritten=0 applied=31 ignored=0} inserts{seen=18 ignored=0} mutations{seen=22 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260202 02:53:44.917708 5580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: Bootstrap complete.
I20260202 02:53:44.923357 5580 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 6 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:44.924304 5580 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3acf71c31a284578b4f8c918eaa50205, State: Initialized, Role: FOLLOWER
I20260202 02:53:44.924933 5580 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 31, Last appended: 6.32, Last appended by leader: 32, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:44.925714 5584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:44.925956 5584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:53:44.926780 5580 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: configured and running, proceeding with master startup.
I20260202 02:53:44.950350 5598 catalog_manager.cc:1269] Loaded cluster ID: 0d95b889b53247a0813a4fda5550ae30
I20260202 02:53:44.950465 5598 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: loading cluster ID for follower catalog manager: success
I20260202 02:53:44.955549 5598 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: acquiring CA information for follower catalog manager: success
I20260202 02:53:44.958897 5598 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260202 02:53:44.973704 5510 master_runner.cc:186] Error getting master registration for 127.0.104.188:42525: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111),
I20260202 02:53:44.979388 5510 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:53:44.999003 4098 heartbeater.cc:344] Connected to a master server at 127.0.104.189:40361
I20260202 02:53:44.999222 4098 heartbeater.cc:499] Master 127.0.104.189:40361 was elected leader, sending a full tablet report...
I20260202 02:53:45.002480 5545 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "a34c77c818054f87bc19600949531ad0" instance_seqno: 1770000797481684) as {username='slave'} at 127.0.104.130:33103; Asking this server to re-register.
I20260202 02:53:45.003511 4098 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:45.005657 5545 ts_manager.cc:194] Registered new tserver with Master: a34c77c818054f87bc19600949531ad0 (127.0.104.130:38525)
I20260202 02:53:45.036264 3956 heartbeater.cc:344] Connected to a master server at 127.0.104.189:40361
I20260202 02:53:45.036530 3956 heartbeater.cc:499] Master 127.0.104.189:40361 was elected leader, sending a full tablet report...
I20260202 02:53:45.038341 5545 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "d9570a47043b4891b2392af17f405e3d" instance_seqno: 1770000796958858) as {username='slave'} at 127.0.104.129:35685; Asking this server to re-register.
I20260202 02:53:45.039116 3956 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:45.040805 5545 ts_manager.cc:194] Registered new tserver with Master: d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425)
I20260202 02:53:45.102790 4234 heartbeater.cc:344] Connected to a master server at 127.0.104.189:40361
I20260202 02:53:45.103048 4234 heartbeater.cc:499] Master 127.0.104.189:40361 was elected leader, sending a full tablet report...
I20260202 02:53:45.105350 5545 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "68f130042e0c4b8292b138ff8a322517" instance_seqno: 1770000797970665) as {username='slave'} at 127.0.104.131:32993; Asking this server to re-register.
I20260202 02:53:45.106175 4234 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:45.107754 5545 ts_manager.cc:194] Registered new tserver with Master: 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661)
I20260202 02:53:45.503685 5609 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:45.503945 5609 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:45.505947 5609 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
I20260202 02:53:45.506789 5413 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3acf71c31a284578b4f8c918eaa50205" candidate_term: 7 candidate_status { last_received { term: 6 index: 32 } } ignore_live_leader: false dest_uuid: "666b7e52b5fc4f3397f04960c8b67100" is_pre_election: true
I20260202 02:53:45.507216 5413 raft_consensus.cc:2368] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3acf71c31a284578b4f8c918eaa50205 for earlier term 7. Current term is 8.
W20260202 02:53:45.507273 5532 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:45.508503 5532 leader_election.cc:400] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577) with higher term. Message: Invalid argument: T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3acf71c31a284578b4f8c918eaa50205 for earlier term 7. Current term is 8.
I20260202 02:53:45.508626 5532 leader_election.cc:403] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260202 02:53:45.508968 5609 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 6 FOLLOWER]: Advancing to term 8
W20260202 02:53:45.509577 5532 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:45.513787 5609 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577) with higher term. Message: Invalid argument: T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3acf71c31a284578b4f8c918eaa50205 for earlier term 7. Current term is 8.
I20260202 02:53:45.674294 5612 raft_consensus.cc:493] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:45.674556 5612 raft_consensus.cc:515] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:45.675307 5612 leader_election.cc:290] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:53:45.677963 5389 leader_election.cc:336] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:45.682579 5555 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "666b7e52b5fc4f3397f04960c8b67100" candidate_term: 9 candidate_status { last_received { term: 8 index: 39 } } ignore_live_leader: false dest_uuid: "3acf71c31a284578b4f8c918eaa50205" is_pre_election: true
I20260202 02:53:45.683066 5555 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666b7e52b5fc4f3397f04960c8b67100 in term 8.
I20260202 02:53:45.683645 5387 leader_election.cc:304] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205, 666b7e52b5fc4f3397f04960c8b67100; no voters: 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:45.683984 5612 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260202 02:53:45.684113 5612 raft_consensus.cc:493] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:53:45.684204 5612 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 8 FOLLOWER]: Advancing to term 9
I20260202 02:53:45.687860 5612 raft_consensus.cc:515] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:45.688707 5612 leader_election.cc:290] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 election: Requested vote from peers 3acf71c31a284578b4f8c918eaa50205 (127.0.104.189:40361), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
I20260202 02:53:45.689471 5555 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "666b7e52b5fc4f3397f04960c8b67100" candidate_term: 9 candidate_status { last_received { term: 8 index: 39 } } ignore_live_leader: false dest_uuid: "3acf71c31a284578b4f8c918eaa50205"
I20260202 02:53:45.689694 5555 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 8 FOLLOWER]: Advancing to term 9
W20260202 02:53:45.690217 5389 leader_election.cc:336] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:45.691982 5555 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666b7e52b5fc4f3397f04960c8b67100 in term 9.
I20260202 02:53:45.692514 5387 leader_election.cc:304] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205, 666b7e52b5fc4f3397f04960c8b67100; no voters: 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:45.692857 5612 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 9 FOLLOWER]: Leader election won for term 9
I20260202 02:53:45.693275 5612 raft_consensus.cc:697] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [term 9 LEADER]: Becoming Leader. State: Replica: 666b7e52b5fc4f3397f04960c8b67100, State: Running, Role: LEADER
I20260202 02:53:45.693755 5612 consensus_queue.cc:237] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 39, Committed index: 39, Last appended: 8.39, Last appended by leader: 39, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:45.696029 5617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 666b7e52b5fc4f3397f04960c8b67100. Latest consensus state: current_term: 9 leader_uuid: "666b7e52b5fc4f3397f04960c8b67100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:45.696425 5617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: This master's current role is: LEADER
I20260202 02:53:45.697103 5619 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260202 02:53:45.699613 5619 catalog_manager.cc:679] Loaded metadata for table default.table_6a23cf9a48824f7e8d243c355f458ce8 [id=0048dd71261d4204ac5548c190957aa9]
I20260202 02:53:45.700114 5619 catalog_manager.cc:679] Loaded metadata for table default.table_8d32aa2872ab405fa4f2e7b1dac33f56 [id=2ab6a7a17803446898f275ae3faa032c]
I20260202 02:53:45.700531 5619 catalog_manager.cc:679] Loaded metadata for table default.table_2c330ab9e5984ba5980e7264050e7a41 [id=3852a850379342c69688c71b95873308]
I20260202 02:53:45.700913 5619 catalog_manager.cc:679] Loaded metadata for table default.table_4e64388f537d44aaa6d541b90e924c65 [id=4f41a1a5a6844fbfb43b6798b2c6f67d]
I20260202 02:53:45.701256 5619 catalog_manager.cc:679] Loaded metadata for table default.table_470455fd011c452b9cceafbc413b1cfe [id=e28252bb319446c39056986e8b98de30]
I20260202 02:53:45.703128 5619 tablet_loader.cc:96] loaded metadata for tablet 4f3dd16be700418b80728adc2ae0cd85 (table default.table_2c330ab9e5984ba5980e7264050e7a41 [id=3852a850379342c69688c71b95873308])
I20260202 02:53:45.703528 5619 tablet_loader.cc:96] loaded metadata for tablet 5313e773f68d4ad2ad3e36880668051f (table default.table_4e64388f537d44aaa6d541b90e924c65 [id=4f41a1a5a6844fbfb43b6798b2c6f67d])
I20260202 02:53:45.703835 5619 tablet_loader.cc:96] loaded metadata for tablet aa58249c51134388b92c95408bd8c923 (table default.table_6a23cf9a48824f7e8d243c355f458ce8 [id=0048dd71261d4204ac5548c190957aa9])
I20260202 02:53:45.704126 5619 tablet_loader.cc:96] loaded metadata for tablet b5a121d55d9d498ba76bb4dcb24d0e20 (table default.table_470455fd011c452b9cceafbc413b1cfe [id=e28252bb319446c39056986e8b98de30])
I20260202 02:53:45.704495 5619 tablet_loader.cc:96] loaded metadata for tablet bb2b2a691a544f8a8997d0c144acb2e9 (table default.table_8d32aa2872ab405fa4f2e7b1dac33f56 [id=2ab6a7a17803446898f275ae3faa032c])
I20260202 02:53:45.704941 5619 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260202 02:53:45.705997 5619 catalog_manager.cc:1269] Loaded cluster ID: 0d95b889b53247a0813a4fda5550ae30
I20260202 02:53:45.706080 5619 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260202 02:53:45.707415 5619 catalog_manager.cc:1514] Loading token signing keys...
I20260202 02:53:45.708407 5619 catalog_manager.cc:6038] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: Loaded TSK: 0
I20260202 02:53:45.709301 5619 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260202 02:53:45.710248 5619 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260202 02:53:45.711105 5619 catalog_manager.cc:5876] Last processed Hive Metastore notification event ID: -1
I20260202 02:53:45.725724 5555 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Refusing update from remote peer 666b7e52b5fc4f3397f04960c8b67100: Log matching property violated. Preceding OpId in replica: term: 6 index: 32. Preceding OpId from leader: term: 9 index: 41. (index mismatch)
W20260202 02:53:45.726052 5389 consensus_peers.cc:597] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 -> Peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Couldn't send request to peer 879caa0e6c114ab4855b7a756e252c9e. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260202 02:53:45.726696 5612 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 40, Last known committed idx: 31, Time since last communication: 0.000s
I20260202 02:53:45.726998 5612 consensus_queue.cc:1243] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [LEADER]: Peer 3acf71c31a284578b4f8c918eaa50205 log is divergent from this leader: its last log entry 6.32 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 31
I20260202 02:53:45.729630 5555 pending_rounds.cc:85] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: Aborting all ops after (but not including) 31
I20260202 02:53:45.729768 5555 pending_rounds.cc:107] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: Aborting uncommitted NO_OP operation due to leader change: 6.32
I20260202 02:53:45.729899 5555 raft_consensus.cc:2889] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260202 02:53:45.737455 5609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 666b7e52b5fc4f3397f04960c8b67100. Latest consensus state: current_term: 9 leader_uuid: "666b7e52b5fc4f3397f04960c8b67100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:45.737756 5609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:53:45.738320 5623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 9 leader_uuid: "666b7e52b5fc4f3397f04960c8b67100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:45.738765 5623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:53:45.738737 5626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 9 leader_uuid: "666b7e52b5fc4f3397f04960c8b67100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:45.739040 5626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:53:45.739616 5621 mvcc.cc:204] Tried to move back new op lower bound from 7249923349829701632 to 7249923338896924672. Current Snapshot: MvccSnapshot[applied={T|T < 7249923349829701632}]
I20260202 02:53:45.739920 5620 mvcc.cc:204] Tried to move back new op lower bound from 7249923382163337216 to 7249923382045327360. Current Snapshot: MvccSnapshot[applied={T|T < 7249923382163337216}]
I20260202 02:53:45.740619 5612 sys_catalog.cc:455] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 9 leader_uuid: "666b7e52b5fc4f3397f04960c8b67100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:45.740839 5612 sys_catalog.cc:458] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100 [sys.catalog]: This master's current role is: LEADER
I20260202 02:53:45.740834 5626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 9 leader_uuid: "666b7e52b5fc4f3397f04960c8b67100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:45.740998 5626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: This master's current role is: FOLLOWER
W20260202 02:53:45.770640 5402 catalog_manager.cc:5487] Ignoring report from unknown tablet 313ff5ec7131498aaaa0c832abea2c56
I20260202 02:53:45.781817 5402 master_service.cc:954] ReplaceTablet: received request to replace tablet 7b76f6283a604527946408b2a8983ee5 from {username='slave'} at 127.0.0.1:36580
I20260202 02:53:45.783110 5400 master_service.cc:954] ReplaceTablet: received request to replace tablet b5a121d55d9d498ba76bb4dcb24d0e20 from {username='slave'} at 127.0.0.1:36580
I20260202 02:53:45.783172 5401 master_service.cc:954] ReplaceTablet: received request to replace tablet 906f9148074848da8456b204a587fa8a from {username='slave'} at 127.0.0.1:36580
I20260202 02:53:45.783768 5398 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36580:
name: "default.Table_4ec28fc3e85f44128ce25471c8cec8b9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:45.784425 5403 master_service.cc:954] ReplaceTablet: received request to replace tablet 8d1373239ae14791a720a0a7ca86d09a from {username='slave'} at 127.0.0.1:36580
I20260202 02:53:45.784797 5399 master_service.cc:954] ReplaceTablet: received request to replace tablet 313ff5ec7131498aaaa0c832abea2c56 from {username='slave'} at 127.0.0.1:36580
I20260202 02:53:45.784971 5397 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:36580:
table { table_name: "default.Table_2c330ab9e5984ba5980e7264050e7a41" } new_table_name: "default.Table_b435fc4b22d0426ebfb76e842d4b43f1" modify_external_catalogs: true
I20260202 02:53:45.785575 5396 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36580:
name: "default.Table_2a3cc2b06aae483d8e2b84d018d13310"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:45.786274 5395 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36580:
name: "default.Table_2d03ff86a95f4175b2225ecc0d899058"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:45.787566 5399 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36580:
name: "default.Table_a728b4dd173b4f0c8c6b9cce0bcda94d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:45.788127 5401 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:36580:
table { table_name: "default.Table_4e64388f537d44aaa6d541b90e924c65" } new_table_name: "default.Table_4ad4563b34454572a4930ad034763780" modify_external_catalogs: true
I20260202 02:53:45.788731 5394 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36580:
name: "default.Table_2faba70635fa4e55ae56458e06dba641"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:45.789717 5403 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36580:
name: "default.Table_851c0addb9fd444eb3b9dd3a3ddd4623"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:45.789994 5402 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36580:
name: "default.Table_de1272fe656e46328aa237aec594394a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260202 02:53:45.792941 5456 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 7 received events: NotificationEvent(eventId=1, eventTime=1770000798, eventType=CREATE_TABLE, dbName=default, tableName=table_4e64388f537d44aaa6d541b90e924c65, message={"server":"","servicePrincipal":"","db":"default","table":"table_4e64388f537d44aaa6d541b90e924c65","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_4e64388f537d44aaa6d541b90e924c65\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770000798},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/warehouse/table_4e64388f537d44aaa6d541b90e924c65\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_4e64388f537d44aaa6d541b90e924c65\",\"kudu.table_id\":\"4f41a1a5a6844fbfb43b6798b2c6f67d\",\"transient_lastDdlTime\":\"1770000798\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"0d95b889b53247a0813a4fda5550ae30\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770000798,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1770000798, eventType=CREATE_TABLE, dbName=default, tableName=table_a728b4dd173b4f0c8c6b9cce0bcda94d, message={"server":"","servicePrincipal":"","db":"default","table":"table_a728b4dd173b4f0c8c6b9cce0bcda94d","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_a728b4dd173b4f0c8c6b9cce0bcda94d\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770000798},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/warehouse/table_a728b4dd173b4f0c8c6b9cce0bcda94d\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_a728b4dd173b4f0c8c6b9cce0bcda94d\",\"kudu.table_id\":\"a54f76c7d1ff4769b2d84a5fe325d81c\",\"transient_lastDdlTime\":\"1770000798\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"0d95b889b53247a0813a4fda5550ae30\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770000798,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
I20260202 02:53:45.794157 5400 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 666b7e52b5fc4f3397f04960c8b67100: Sending DeleteTablet for 3 replicas of tablet b5a121d55d9d498ba76bb4dcb24d0e20
@ 0x7f21af8e5dcd google::LogMessage::Fail() at ??:0
I20260202 02:53:45.800788 5400 catalog_manager.cc:6724] ReplaceTablet: tablet b5a121d55d9d498ba76bb4dcb24d0e20 deleted and replaced by tablet f57c2e7678744e95902a025a9ffc68f0
I20260202 02:53:45.801345 5400 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36580:
name: "default.Table_dc0bdab2115f4027a86e680d2995f726"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260202 02:53:45.805724 3887 tablet_service.cc:1552] Processing DeleteTablet for tablet b5a121d55d9d498ba76bb4dcb24d0e20 with delete_type TABLET_DATA_DELETED (replaced by tablet f57c2e7678744e95902a025a9ffc68f0) from {username='slave'} at 127.0.0.1:33436
I20260202 02:53:45.807444 4029 tablet_service.cc:1552] Processing DeleteTablet for tablet b5a121d55d9d498ba76bb4dcb24d0e20 with delete_type TABLET_DATA_DELETED (replaced by tablet f57c2e7678744e95902a025a9ffc68f0) from {username='slave'} at 127.0.0.1:38556
I20260202 02:53:45.812247 4167 tablet_service.cc:1552] Processing DeleteTablet for tablet b5a121d55d9d498ba76bb4dcb24d0e20 with delete_type TABLET_DATA_DELETED (replaced by tablet f57c2e7678744e95902a025a9ffc68f0) from {username='slave'} at 127.0.0.1:52848
I20260202 02:53:45.812844 5635 tablet_replica.cc:333] T b5a121d55d9d498ba76bb4dcb24d0e20 P a34c77c818054f87bc19600949531ad0: stopping tablet replica
I20260202 02:53:45.813107 5635 raft_consensus.cc:2243] T b5a121d55d9d498ba76bb4dcb24d0e20 P a34c77c818054f87bc19600949531ad0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:53:45.813380 5635 raft_consensus.cc:2272] T b5a121d55d9d498ba76bb4dcb24d0e20 P a34c77c818054f87bc19600949531ad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260202 02:53:45.814124 5635 ts_tablet_manager.cc:1916] T b5a121d55d9d498ba76bb4dcb24d0e20 P a34c77c818054f87bc19600949531ad0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:53:45.818114 5636 tablet_replica.cc:333] T b5a121d55d9d498ba76bb4dcb24d0e20 P 68f130042e0c4b8292b138ff8a322517: stopping tablet replica
I20260202 02:53:45.818100 5634 tablet_replica.cc:333] T b5a121d55d9d498ba76bb4dcb24d0e20 P d9570a47043b4891b2392af17f405e3d: stopping tablet replica
I20260202 02:53:45.818353 5634 raft_consensus.cc:2243] T b5a121d55d9d498ba76bb4dcb24d0e20 P d9570a47043b4891b2392af17f405e3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260202 02:53:45.818540 5636 raft_consensus.cc:2243] T b5a121d55d9d498ba76bb4dcb24d0e20 P 68f130042e0c4b8292b138ff8a322517 [term 1 LEADER]: Raft consensus shutting down.
@ 0x7f21af8e9b93 google::LogMessage::SendToLog() at ??:0
I20260202 02:53:45.819437 5636 raft_consensus.cc:2272] T b5a121d55d9d498ba76bb4dcb24d0e20 P 68f130042e0c4b8292b138ff8a322517 [term 1 FOLLOWER]: Raft consensus is shut down!
@ 0x7f21af8e57cc google::LogMessage::Flush() at ??:0
I20260202 02:53:45.820097 5636 ts_tablet_manager.cc:1916] T b5a121d55d9d498ba76bb4dcb24d0e20 P 68f130042e0c4b8292b138ff8a322517: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:53:45.821228 5634 raft_consensus.cc:2272] T b5a121d55d9d498ba76bb4dcb24d0e20 P d9570a47043b4891b2392af17f405e3d [term 1 FOLLOWER]: Raft consensus is shut down!
@ 0x7f21af8e5a11 google::LogMessage::~LogMessage() at ??:0
I20260202 02:53:45.821921 5634 ts_tablet_manager.cc:1916] T b5a121d55d9d498ba76bb4dcb24d0e20 P d9570a47043b4891b2392af17f405e3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260202 02:53:45.823786 5635 ts_tablet_manager.cc:1929] T b5a121d55d9d498ba76bb4dcb24d0e20 P a34c77c818054f87bc19600949531ad0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:53:45.823958 5635 log.cc:1199] T b5a121d55d9d498ba76bb4dcb24d0e20 P a34c77c818054f87bc19600949531ad0: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-1/wal/wals/b5a121d55d9d498ba76bb4dcb24d0e20
I20260202 02:53:45.825285 5635 ts_tablet_manager.cc:1950] T b5a121d55d9d498ba76bb4dcb24d0e20 P a34c77c818054f87bc19600949531ad0: Deleting consensus metadata
I20260202 02:53:45.826586 5387 catalog_manager.cc:4985] TS a34c77c818054f87bc19600949531ad0 (127.0.104.130:38525): tablet b5a121d55d9d498ba76bb4dcb24d0e20 (table default.table_470455fd011c452b9cceafbc413b1cfe [id=e28252bb319446c39056986e8b98de30]) successfully deleted
I20260202 02:53:45.828168 4029 tablet_service.cc:1505] Processing CreateTablet for tablet f57c2e7678744e95902a025a9ffc68f0 (DEFAULT_TABLE table=default.table_470455fd011c452b9cceafbc413b1cfe [id=e28252bb319446c39056986e8b98de30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:45.828691 4029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f57c2e7678744e95902a025a9ffc68f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:45.828812 4167 tablet_service.cc:1505] Processing CreateTablet for tablet f57c2e7678744e95902a025a9ffc68f0 (DEFAULT_TABLE table=default.table_470455fd011c452b9cceafbc413b1cfe [id=e28252bb319446c39056986e8b98de30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:45.829238 4167 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f57c2e7678744e95902a025a9ffc68f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:45.832522 5636 ts_tablet_manager.cc:1929] T b5a121d55d9d498ba76bb4dcb24d0e20 P 68f130042e0c4b8292b138ff8a322517: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:53:45.832713 5636 log.cc:1199] T b5a121d55d9d498ba76bb4dcb24d0e20 P 68f130042e0c4b8292b138ff8a322517: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-2/wal/wals/b5a121d55d9d498ba76bb4dcb24d0e20
I20260202 02:53:45.833024 3887 tablet_service.cc:1505] Processing CreateTablet for tablet f57c2e7678744e95902a025a9ffc68f0 (DEFAULT_TABLE table=default.table_470455fd011c452b9cceafbc413b1cfe [id=e28252bb319446c39056986e8b98de30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260202 02:53:45.833248 5636 ts_tablet_manager.cc:1950] T b5a121d55d9d498ba76bb4dcb24d0e20 P 68f130042e0c4b8292b138ff8a322517: Deleting consensus metadata
I20260202 02:53:45.833544 3887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f57c2e7678744e95902a025a9ffc68f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260202 02:53:45.833881 5634 ts_tablet_manager.cc:1929] T b5a121d55d9d498ba76bb4dcb24d0e20 P d9570a47043b4891b2392af17f405e3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260202 02:53:45.834008 5634 log.cc:1199] T b5a121d55d9d498ba76bb4dcb24d0e20 P d9570a47043b4891b2392af17f405e3d: Deleting WAL directory at /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/ts-0/wal/wals/b5a121d55d9d498ba76bb4dcb24d0e20
I20260202 02:53:45.834656 5634 ts_tablet_manager.cc:1950] T b5a121d55d9d498ba76bb4dcb24d0e20 P d9570a47043b4891b2392af17f405e3d: Deleting consensus metadata
I20260202 02:53:45.835965 5387 catalog_manager.cc:4985] TS d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): tablet b5a121d55d9d498ba76bb4dcb24d0e20 (table default.table_470455fd011c452b9cceafbc413b1cfe [id=e28252bb319446c39056986e8b98de30]) successfully deleted
I20260202 02:53:45.836206 5386 catalog_manager.cc:4985] TS 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): tablet b5a121d55d9d498ba76bb4dcb24d0e20 (table default.table_470455fd011c452b9cceafbc413b1cfe [id=e28252bb319446c39056986e8b98de30]) successfully deleted
@ 0x7f21c62dc7f4 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f21c62db0a1 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260202 02:53:45.843686 5638 tablet_bootstrap.cc:492] T f57c2e7678744e95902a025a9ffc68f0 P 68f130042e0c4b8292b138ff8a322517: Bootstrap starting.
I20260202 02:53:45.845907 5638 tablet_bootstrap.cc:654] T f57c2e7678744e95902a025a9ffc68f0 P 68f130042e0c4b8292b138ff8a322517: Neither blocks nor log segments found. Creating new log.
I20260202 02:53:45.847002 5641 tablet_bootstrap.cc:492] T f57c2e7678744e95902a025a9ffc68f0 P d9570a47043b4891b2392af17f405e3d: Bootstrap starting.
I20260202 02:53:45.847642 5640 tablet_bootstrap.cc:492] T f57c2e7678744e95902a025a9ffc68f0 P a34c77c818054f87bc19600949531ad0: Bootstrap starting.
@ 0x7f21c62e2d1c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260202 02:53:45.850095 5641 tablet_bootstrap.cc:654] T f57c2e7678744e95902a025a9ffc68f0 P d9570a47043b4891b2392af17f405e3d: Neither blocks nor log segments found. Creating new log.
@ 0x7f21c62e2b99 std::_Function_handler<>::_M_invoke() at ??:0
I20260202 02:53:45.852770 5640 tablet_bootstrap.cc:654] T f57c2e7678744e95902a025a9ffc68f0 P a34c77c818054f87bc19600949531ad0: Neither blocks nor log segments found. Creating new log.
@ 0xa8970d std::function<>::operator()() at ??:?
@ 0x7f21b265af99 kudu::Thread::SuperviseThread() at ??:0
I20260202 02:53:45.859390 5641 tablet_bootstrap.cc:492] T f57c2e7678744e95902a025a9ffc68f0 P d9570a47043b4891b2392af17f405e3d: No bootstrap required, opened a new log
I20260202 02:53:45.859637 5641 ts_tablet_manager.cc:1403] T f57c2e7678744e95902a025a9ffc68f0 P d9570a47043b4891b2392af17f405e3d: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.003s
I20260202 02:53:45.861183 5641 raft_consensus.cc:359] T f57c2e7678744e95902a025a9ffc68f0 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:45.861420 5641 raft_consensus.cc:385] T f57c2e7678744e95902a025a9ffc68f0 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:45.861521 5641 raft_consensus.cc:740] T f57c2e7678744e95902a025a9ffc68f0 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9570a47043b4891b2392af17f405e3d, State: Initialized, Role: FOLLOWER
I20260202 02:53:45.863485 5638 tablet_bootstrap.cc:492] T f57c2e7678744e95902a025a9ffc68f0 P 68f130042e0c4b8292b138ff8a322517: No bootstrap required, opened a new log
@ 0x7f21b5c676db start_thread at ??:0
I20260202 02:53:45.863674 5638 ts_tablet_manager.cc:1403] T f57c2e7678744e95902a025a9ffc68f0 P 68f130042e0c4b8292b138ff8a322517: Time spent bootstrapping tablet: real 0.020s user 0.008s sys 0.009s
I20260202 02:53:45.864584 5638 raft_consensus.cc:359] T f57c2e7678744e95902a025a9ffc68f0 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
@ 0x7f21ae79671f clone at ??:0
I20260202 02:53:45.864777 5638 raft_consensus.cc:385] T f57c2e7678744e95902a025a9ffc68f0 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:45.864858 5638 raft_consensus.cc:740] T f57c2e7678744e95902a025a9ffc68f0 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f130042e0c4b8292b138ff8a322517, State: Initialized, Role: FOLLOWER
I20260202 02:53:45.865445 5638 consensus_queue.cc:260] T f57c2e7678744e95902a025a9ffc68f0 P 68f130042e0c4b8292b138ff8a322517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:45.865852 5640 tablet_bootstrap.cc:492] T f57c2e7678744e95902a025a9ffc68f0 P a34c77c818054f87bc19600949531ad0: No bootstrap required, opened a new log
I20260202 02:53:45.865986 5638 ts_tablet_manager.cc:1434] T f57c2e7678744e95902a025a9ffc68f0 P 68f130042e0c4b8292b138ff8a322517: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260202 02:53:45.866010 5640 ts_tablet_manager.cc:1403] T f57c2e7678744e95902a025a9ffc68f0 P a34c77c818054f87bc19600949531ad0: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.000s
I20260202 02:53:45.865497 5641 consensus_queue.cc:260] T f57c2e7678744e95902a025a9ffc68f0 P d9570a47043b4891b2392af17f405e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:45.867972 5641 ts_tablet_manager.cc:1434] T f57c2e7678744e95902a025a9ffc68f0 P d9570a47043b4891b2392af17f405e3d: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260202 02:53:45.870041 5640 raft_consensus.cc:359] T f57c2e7678744e95902a025a9ffc68f0 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:45.870218 5640 raft_consensus.cc:385] T f57c2e7678744e95902a025a9ffc68f0 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260202 02:53:45.870276 5640 raft_consensus.cc:740] T f57c2e7678744e95902a025a9ffc68f0 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a34c77c818054f87bc19600949531ad0, State: Initialized, Role: FOLLOWER
I20260202 02:53:45.870469 5640 consensus_queue.cc:260] T f57c2e7678744e95902a025a9ffc68f0 P a34c77c818054f87bc19600949531ad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:45.870915 4097 heartbeater.cc:499] Master 127.0.104.190:34577 was elected leader, sending a full tablet report...
I20260202 02:53:45.870980 5640 ts_tablet_manager.cc:1434] T f57c2e7678744e95902a025a9ffc68f0 P a34c77c818054f87bc19600949531ad0: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
W20260202 02:53:46.060216 3843 connection.cc:537] client connection to 127.0.104.190:34577 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260202 02:53:46.060413 4123 connection.cc:537] client connection to 127.0.104.190:34577 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260202 02:53:46.060858 4242 connection.cc:537] client connection to 127.0.104.190:34577 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260202 02:53:46.060257 3987 connection.cc:537] client connection to 127.0.104.190:34577 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260202 02:53:46.061676 4242 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.104.190:34577)
W20260202 02:53:46.062299 4242 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.104.190:34577)
W20260202 02:53:46.062538 4242 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.104.190:34577)
I20260202 02:53:46.325630 5656 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 666b7e52b5fc4f3397f04960c8b67100)
I20260202 02:53:46.325795 5656 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:46.326457 5656 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:53:46.328477 5532 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:53:46.328644 5532 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:46.328730 5532 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:46.329021 5656 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260202 02:53:46.471148 4515 raft_consensus.cc:493] T f57c2e7678744e95902a025a9ffc68f0 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:46.471347 4515 raft_consensus.cc:515] T f57c2e7678744e95902a025a9ffc68f0 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:46.471987 4515 leader_election.cc:290] T f57c2e7678744e95902a025a9ffc68f0 P a34c77c818054f87bc19600949531ad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661), d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425)
I20260202 02:53:46.472610 3909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f57c2e7678744e95902a025a9ffc68f0" candidate_uuid: "a34c77c818054f87bc19600949531ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9570a47043b4891b2392af17f405e3d" is_pre_election: true
I20260202 02:53:46.472672 4187 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f57c2e7678744e95902a025a9ffc68f0" candidate_uuid: "a34c77c818054f87bc19600949531ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f130042e0c4b8292b138ff8a322517" is_pre_election: true
I20260202 02:53:46.472812 3909 raft_consensus.cc:2468] T f57c2e7678744e95902a025a9ffc68f0 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c77c818054f87bc19600949531ad0 in term 0.
I20260202 02:53:46.472918 4187 raft_consensus.cc:2468] T f57c2e7678744e95902a025a9ffc68f0 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c77c818054f87bc19600949531ad0 in term 0.
I20260202 02:53:46.473299 3985 leader_election.cc:304] T f57c2e7678744e95902a025a9ffc68f0 P a34c77c818054f87bc19600949531ad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c77c818054f87bc19600949531ad0, d9570a47043b4891b2392af17f405e3d; no voters:
I20260202 02:53:46.473647 4515 raft_consensus.cc:2804] T f57c2e7678744e95902a025a9ffc68f0 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260202 02:53:46.473740 4515 raft_consensus.cc:493] T f57c2e7678744e95902a025a9ffc68f0 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260202 02:53:46.473846 4515 raft_consensus.cc:3060] T f57c2e7678744e95902a025a9ffc68f0 P a34c77c818054f87bc19600949531ad0 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:46.475764 4515 raft_consensus.cc:515] T f57c2e7678744e95902a025a9ffc68f0 P a34c77c818054f87bc19600949531ad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:46.476405 4515 leader_election.cc:290] T f57c2e7678744e95902a025a9ffc68f0 P a34c77c818054f87bc19600949531ad0 [CANDIDATE]: Term 1 election: Requested vote from peers 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661), d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425)
I20260202 02:53:46.476892 3909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f57c2e7678744e95902a025a9ffc68f0" candidate_uuid: "a34c77c818054f87bc19600949531ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9570a47043b4891b2392af17f405e3d"
I20260202 02:53:46.476923 4187 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f57c2e7678744e95902a025a9ffc68f0" candidate_uuid: "a34c77c818054f87bc19600949531ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f130042e0c4b8292b138ff8a322517"
I20260202 02:53:46.477098 3909 raft_consensus.cc:3060] T f57c2e7678744e95902a025a9ffc68f0 P d9570a47043b4891b2392af17f405e3d [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:46.477139 4187 raft_consensus.cc:3060] T f57c2e7678744e95902a025a9ffc68f0 P 68f130042e0c4b8292b138ff8a322517 [term 0 FOLLOWER]: Advancing to term 1
I20260202 02:53:46.478981 4187 raft_consensus.cc:2468] T f57c2e7678744e95902a025a9ffc68f0 P 68f130042e0c4b8292b138ff8a322517 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c77c818054f87bc19600949531ad0 in term 1.
I20260202 02:53:46.478981 3909 raft_consensus.cc:2468] T f57c2e7678744e95902a025a9ffc68f0 P d9570a47043b4891b2392af17f405e3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c77c818054f87bc19600949531ad0 in term 1.
I20260202 02:53:46.479514 3985 leader_election.cc:304] T f57c2e7678744e95902a025a9ffc68f0 P a34c77c818054f87bc19600949531ad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 68f130042e0c4b8292b138ff8a322517, a34c77c818054f87bc19600949531ad0, d9570a47043b4891b2392af17f405e3d; no voters:
I20260202 02:53:46.479807 4515 raft_consensus.cc:2804] T f57c2e7678744e95902a025a9ffc68f0 P a34c77c818054f87bc19600949531ad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260202 02:53:46.479977 4515 raft_consensus.cc:697] T f57c2e7678744e95902a025a9ffc68f0 P a34c77c818054f87bc19600949531ad0 [term 1 LEADER]: Becoming Leader. State: Replica: a34c77c818054f87bc19600949531ad0, State: Running, Role: LEADER
I20260202 02:53:46.480312 4515 consensus_queue.cc:237] T f57c2e7678744e95902a025a9ffc68f0 P a34c77c818054f87bc19600949531ad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 } } peers { permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 } } peers { permanent_uuid: "a34c77c818054f87bc19600949531ad0" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 38525 } }
I20260202 02:53:47.026294 418 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskYXxklX/build/asan/bin/kudu pid 5510
I20260202 02:53:47.037377 5575 generic_service.cc:196] Checking for leaks (request via RPC)
I20260202 02:53:47.149336 5667 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 666b7e52b5fc4f3397f04960c8b67100)
I20260202 02:53:47.149602 418 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskYXxklX/build/asan/bin/kudu with pid 5510
I20260202 02:53:47.233763 4515 consensus_queue.cc:1048] T f57c2e7678744e95902a025a9ffc68f0 P a34c77c818054f87bc19600949531ad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68f130042e0c4b8292b138ff8a322517" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:53:47.240688 5505 consensus_queue.cc:1048] T f57c2e7678744e95902a025a9ffc68f0 P a34c77c818054f87bc19600949531ad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9570a47043b4891b2392af17f405e3d" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 44425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260202 02:53:47.737483 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskYXxklX/build/asan/bin/kudu
/tmp/dist-test-taskYXxklX/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.189:40361
--webserver_interface=127.0.104.189
--webserver_port=35647
--builtin_ntp_servers=127.0.104.148:34531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525
--hive_metastore_uris=thrift://127.0.0.1:46545
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260202 02:53:48.201203 5682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260202 02:53:48.201668 5682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260202 02:53:48.201790 5682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260202 02:53:48.215755 5682 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260202 02:53:48.215901 5682 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260202 02:53:48.215986 5682 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260202 02:53:48.216058 5682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260202 02:53:48.216153 5682 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260202 02:53:48.216243 5682 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260202 02:53:48.233683 5682 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:34531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:46545
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.189:40361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.104.189
--webserver_port=35647
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3722ed49c19ce54c7feae2fa944e32b1d4ed8d2d
build type FASTDEBUG
built by None at 02 Feb 2026 02:43:23 UTC on 24a791456cd2
build id 10304
ASAN enabled
I20260202 02:53:48.235947 5682 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260202 02:53:48.237996 5682 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260202 02:53:48.247001 5692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:53:48.247007 5693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260202 02:53:48.247874 5695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260202 02:53:48.250072 5682 server_base.cc:1060] running on GCE node
I20260202 02:53:48.250947 5682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260202 02:53:48.254333 5682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260202 02:53:48.255623 5682 hybrid_clock.cc:648] HybridClock initialized: now 1770000828255528 us; error 72 us; skew 500 ppm
I20260202 02:53:48.256069 5682 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260202 02:53:48.258539 5682 webserver.cc:492] Webserver started at http://127.0.104.189:35647/ using document root <none> and password file <none>
I20260202 02:53:48.259377 5682 fs_manager.cc:362] Metadata directory not provided
I20260202 02:53:48.259500 5682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260202 02:53:48.265494 5682 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260202 02:53:48.269265 5703 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260202 02:53:48.270849 5682 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260202 02:53:48.271045 5682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data,/tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/wal
uuid: "3acf71c31a284578b4f8c918eaa50205"
format_stamp: "Formatted at 2026-02-02 02:53:15 on dist-test-slave-llj2"
I20260202 02:53:48.271834 5682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260202 02:53:48.289403 5682 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260202 02:53:48.290340 5682 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260202 02:53:48.290642 5682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260202 02:53:48.309969 5682 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.189:40361
I20260202 02:53:48.309998 5754 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.189:40361 every 8 connection(s)
I20260202 02:53:48.311722 5682 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskYXxklX/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1770000729692874-418-0/minicluster-data/master-1/data/info.pb
I20260202 02:53:48.315404 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskYXxklX/build/asan/bin/kudu as pid 5682
I20260202 02:53:48.321411 5755 sys_catalog.cc:263] Verifying existing consensus state
I20260202 02:53:48.325968 5755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: Bootstrap starting.
I20260202 02:53:48.340171 5755 log.cc:826] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: Log is configured to *not* fsync() on all Append() calls
I20260202 02:53:48.375836 4098 heartbeater.cc:344] Connected to a master server at 127.0.104.189:40361
I20260202 02:53:48.378772 5755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: Bootstrap replayed 1/1 log segments. Stats: ops{read=44 overwritten=1 applied=43 ignored=0} inserts{seen=22 ignored=0} mutations{seen=33 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260202 02:53:48.379477 5755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: Bootstrap complete.
I20260202 02:53:48.385330 5755 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:48.386116 5755 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3acf71c31a284578b4f8c918eaa50205, State: Initialized, Role: FOLLOWER
I20260202 02:53:48.386826 5755 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 43, Last appended: 9.43, Last appended by leader: 43, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:48.387727 5761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 9 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } } }
I20260202 02:53:48.388010 5761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: This master's current role is: FOLLOWER
I20260202 02:53:48.388922 5755 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [sys.catalog]: configured and running, proceeding with master startup.
I20260202 02:53:48.405900 5775 catalog_manager.cc:1269] Loaded cluster ID: 0d95b889b53247a0813a4fda5550ae30
I20260202 02:53:48.406137 5775 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: loading cluster ID for follower catalog manager: success
I20260202 02:53:48.411276 5775 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: acquiring CA information for follower catalog manager: success
I20260202 02:53:48.415939 5775 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260202 02:53:48.412642 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:53:48.419710 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:53:48.490132 5720 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "a34c77c818054f87bc19600949531ad0" instance_seqno: 1770000797481684) as {username='slave'} at 127.0.104.130:54215; Asking this server to re-register.
I20260202 02:53:48.490895 4098 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:48.492832 5720 ts_manager.cc:194] Registered new tserver with Master: a34c77c818054f87bc19600949531ad0 (127.0.104.130:38525)
I20260202 02:53:48.755549 4234 heartbeater.cc:344] Connected to a master server at 127.0.104.189:40361
I20260202 02:53:48.757526 5720 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "68f130042e0c4b8292b138ff8a322517" instance_seqno: 1770000797970665) as {username='slave'} at 127.0.104.131:47829; Asking this server to re-register.
I20260202 02:53:48.758351 4234 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:48.759981 5720 ts_manager.cc:194] Registered new tserver with Master: 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661)
I20260202 02:53:48.764389 3956 heartbeater.cc:344] Connected to a master server at 127.0.104.189:40361
I20260202 02:53:48.765890 5720 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "d9570a47043b4891b2392af17f405e3d" instance_seqno: 1770000796958858) as {username='slave'} at 127.0.104.129:51101; Asking this server to re-register.
I20260202 02:53:48.766594 3956 heartbeater.cc:461] Registering TS with master...
I20260202 02:53:48.768219 5720 ts_manager.cc:194] Registered new tserver with Master: d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425)
I20260202 02:53:48.841832 5761 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:48.842123 5761 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:48.844050 5761 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:53:48.844815 5707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:53:48.846509 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:53:48.846743 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:48.846843 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:48.847236 5761 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260202 02:53:49.417855 5789 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:49.418078 5789 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:49.418861 5789 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:53:49.421545 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:53:49.421836 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:49.422015 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:49.422312 5789 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260202 02:53:49.846549 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 16: this message will repeat every 5th retry.
W20260202 02:53:50.120788 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 16: this message will repeat every 5th retry.
I20260202 02:53:50.229871 5798 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:50.230085 5798 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:50.230690 5798 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:53:50.233048 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:53:50.233294 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:50.233397 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:50.233706 5798 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260202 02:53:50.342761 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 16: this message will repeat every 5th retry.
I20260202 02:53:50.628077 418 master-stress-test.cc:577] Killing test thread 1/25
W20260202 02:53:50.779464 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 16: this message will repeat every 5th retry.
I20260202 02:53:51.092212 5813 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:51.092429 5813 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:51.092999 5813 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:53:51.095491 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
W20260202 02:53:51.095672 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
I20260202 02:53:51.095844 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:51.096160 5813 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260202 02:53:51.522339 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 16: this message will repeat every 5th retry.
W20260202 02:53:51.725320 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 16: this message will repeat every 5th retry.
I20260202 02:53:52.628278 5833 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:52.628531 5833 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:52.629118 5833 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:53:52.633600 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:53:52.633826 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:52.633917 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:52.634212 5833 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260202 02:53:53.421480 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:53:53.421689 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:53:53.578315 5848 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:53.578533 5848 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:53.579190 5848 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:53:53.581410 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:53:53.581610 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:53.581714 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:53.582000 5848 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260202 02:53:57.267283 5894 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:57.267488 5894 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:57.268426 5894 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:53:57.269690 5707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111) [suppressed 11 similar messages]
W20260202 02:53:57.271099 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:53:57.271328 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:57.271451 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:57.271703 5894 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260202 02:53:58.423269 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:53:58.423494 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:53:59.118173 5919 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:53:59.118378 5919 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:53:59.119046 5919 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:53:59.122941 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:53:59.123183 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:53:59.123266 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:53:59.123591 5919 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260202 02:53:59.802204 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 21: this message will repeat every 5th retry.
W20260202 02:53:59.951126 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 21: this message will repeat every 5th retry.
W20260202 02:53:59.968267 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 21: this message will repeat every 5th retry.
W20260202 02:54:00.616133 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 21: this message will repeat every 5th retry.
W20260202 02:54:01.589257 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 21: this message will repeat every 5th retry.
W20260202 02:54:01.729254 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 21: this message will repeat every 5th retry.
I20260202 02:54:03.425253 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:54:03.425475 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:54:08.427073 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:54:08.427261 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:54:08.961495 6033 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:54:08.961706 6033 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:54:08.962409 6033 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:54:08.963745 5707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260202 02:54:08.965432 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:54:08.965629 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:54:08.965723 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:54:08.966060 6033 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260202 02:54:09.373781 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 26: this message will repeat every 5th retry.
W20260202 02:54:09.420984 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 26: this message will repeat every 5th retry.
W20260202 02:54:10.630909 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 26: this message will repeat every 5th retry.
W20260202 02:54:10.732300 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 26: this message will repeat every 5th retry.
W20260202 02:54:11.640060 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 26: this message will repeat every 5th retry.
W20260202 02:54:11.692114 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 26: this message will repeat every 5th retry.
I20260202 02:54:12.720427 6077 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:54:12.720630 6077 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:54:12.721383 6077 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:54:12.723937 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:54:12.724160 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:54:12.724272 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:54:12.724529 6077 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260202 02:54:13.428793 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:54:13.429034 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:54:18.430598 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:54:18.430792 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:54:18.542877 4096 heartbeater.cc:646] Failed to heartbeat to 127.0.104.188:42525 (267 consecutive failures): Network error: Failed to ping master at 127.0.104.188:42525: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
W20260202 02:54:18.842613 4232 heartbeater.cc:646] Failed to heartbeat to 127.0.104.188:42525 (89 consecutive failures): Network error: Failed to ping master at 127.0.104.188:42525: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
W20260202 02:54:18.862717 3954 heartbeater.cc:646] Failed to heartbeat to 127.0.104.188:42525 (89 consecutive failures): Network error: Failed to ping master at 127.0.104.188:42525: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
W20260202 02:54:19.465814 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 31: this message will repeat every 5th retry.
W20260202 02:54:19.867236 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 31: this message will repeat every 5th retry.
W20260202 02:54:20.543283 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 31: this message will repeat every 5th retry.
W20260202 02:54:21.429280 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 31: this message will repeat every 5th retry.
W20260202 02:54:21.617347 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 31: this message will repeat every 5th retry.
W20260202 02:54:21.902489 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 31: this message will repeat every 5th retry.
I20260202 02:54:22.669252 6161 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:54:22.669483 6161 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:54:22.670226 6161 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:54:22.671504 5707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260202 02:54:22.673038 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:54:22.673249 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:54:22.673360 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:54:22.673646 6161 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260202 02:54:23.432317 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:54:23.432565 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:54:28.434171 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:54:28.434391 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:54:29.833330 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 36: this message will repeat every 5th retry.
W20260202 02:54:29.889084 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 36: this message will repeat every 5th retry.
W20260202 02:54:30.164193 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 36: this message will repeat every 5th retry.
W20260202 02:54:31.108057 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 36: this message will repeat every 5th retry.
W20260202 02:54:31.829946 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 36: this message will repeat every 5th retry.
W20260202 02:54:32.009873 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 36: this message will repeat every 5th retry.
I20260202 02:54:33.435902 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:54:33.436110 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:54:38.437683 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:54:38.437899 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:54:38.673761 6332 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:54:38.673938 6332 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:54:38.674777 6332 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:54:38.675412 5707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260202 02:54:38.676908 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:54:38.677136 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:54:38.677245 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:54:38.677496 6332 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260202 02:54:38.878136 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 41: this message will repeat every 5th retry.
W20260202 02:54:39.898532 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 41: this message will repeat every 5th retry.
I20260202 02:54:40.406216 6353 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:54:40.406477 6353 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:54:40.408012 6353 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:54:40.410001 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:54:40.410245 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:54:40.410435 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:54:40.410758 6353 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260202 02:54:40.734362 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 41: this message will repeat every 5th retry.
W20260202 02:54:41.337845 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 41: this message will repeat every 5th retry.
W20260202 02:54:41.414201 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 41: this message will repeat every 5th retry.
W20260202 02:54:41.572201 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 41: this message will repeat every 5th retry.
I20260202 02:54:43.443074 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:54:43.443342 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:54:48.444855 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:54:48.445063 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:54:48.775068 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 46: this message will repeat every 5th retry.
W20260202 02:54:50.004485 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 46: this message will repeat every 5th retry.
W20260202 02:54:50.449306 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 46: this message will repeat every 5th retry.
W20260202 02:54:50.821790 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 46: this message will repeat every 5th retry.
W20260202 02:54:51.601302 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 46: this message will repeat every 5th retry.
W20260202 02:54:51.688380 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 46: this message will repeat every 5th retry.
I20260202 02:54:53.446693 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:54:53.446928 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:54:58.448639 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:54:58.448829 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:54:58.875365 6563 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:54:58.875669 6563 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:54:58.876641 6563 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:54:58.877529 5707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260202 02:54:58.878953 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:54:58.879150 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:54:58.879240 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:54:58.879529 6563 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260202 02:54:59.196249 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 51: this message will repeat every 5th retry.
W20260202 02:55:00.398828 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 51: this message will repeat every 5th retry.
W20260202 02:55:01.320537 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 51: this message will repeat every 5th retry.
W20260202 02:55:01.659965 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 51: this message will repeat every 5th retry.
W20260202 02:55:01.665829 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 51: this message will repeat every 5th retry.
W20260202 02:55:01.688156 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 51: this message will repeat every 5th retry.
I20260202 02:55:03.072831 6607 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:55:03.073058 6607 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:55:03.073765 6607 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:55:03.076537 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:55:03.076767 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:55:03.076871 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:55:03.077183 6607 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260202 02:55:03.450111 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:55:03.450335 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:55:05.653678 6639 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:55:05.653925 6639 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:55:05.654616 6639 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:55:05.655910 5707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260202 02:55:05.657390 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:55:05.657608 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:55:05.657703 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:55:05.657961 6639 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260202 02:55:08.452080 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:55:08.452361 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:55:08.761021 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 56: this message will repeat every 5th retry.
W20260202 02:55:10.343331 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 56: this message will repeat every 5th retry.
I20260202 02:55:10.492833 6694 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:55:10.493041 6694 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:55:10.493811 6694 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:55:10.497772 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:55:10.498037 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:55:10.498138 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:55:10.498431 6694 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260202 02:55:10.875625 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 56: this message will repeat every 5th retry.
W20260202 02:55:11.414855 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 56: this message will repeat every 5th retry.
W20260202 02:55:11.548307 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 56: this message will repeat every 5th retry.
W20260202 02:55:11.635440 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 56: this message will repeat every 5th retry.
I20260202 02:55:13.454048 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:55:13.454248 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:55:18.456034 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:55:18.456241 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:55:18.738898 4097 heartbeater.cc:646] Failed to heartbeat to 127.0.104.190:34577 (582 consecutive failures): Network error: Failed to ping master at 127.0.104.190:34577: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:55:19.023797 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 61: this message will repeat every 5th retry.
W20260202 02:55:19.040094 4232 heartbeater.cc:646] Failed to heartbeat to 127.0.104.188:42525 (209 consecutive failures): Network error: Failed to ping master at 127.0.104.188:42525: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
W20260202 02:55:19.067765 3955 heartbeater.cc:646] Failed to heartbeat to 127.0.104.190:34577 (187 consecutive failures): Network error: Failed to ping master at 127.0.104.190:34577: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:55:20.936672 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 61: this message will repeat every 5th retry.
W20260202 02:55:20.949002 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 61: this message will repeat every 5th retry.
W20260202 02:55:20.960949 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 61: this message will repeat every 5th retry.
W20260202 02:55:21.876446 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 61: this message will repeat every 5th retry.
W20260202 02:55:21.894755 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 61: this message will repeat every 5th retry.
I20260202 02:55:23.457995 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:55:23.458212 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:55:24.002931 6878 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:55:24.003118 6878 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:55:24.003899 6878 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:55:24.005191 5707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260202 02:55:24.006408 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:55:24.006660 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:55:24.006753 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:55:24.007010 6878 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260202 02:55:27.209125 6923 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:55:27.209352 6923 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:55:27.210083 6923 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:55:27.212774 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:55:27.212987 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:55:27.213094 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:55:27.213372 6923 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260202 02:55:28.460043 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:55:28.460244 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:55:29.282341 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 66: this message will repeat every 5th retry.
W20260202 02:55:30.240202 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 66: this message will repeat every 5th retry.
W20260202 02:55:30.891665 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 66: this message will repeat every 5th retry.
W20260202 02:55:30.931865 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 66: this message will repeat every 5th retry.
W20260202 02:55:31.762147 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 66: this message will repeat every 5th retry.
W20260202 02:55:32.166399 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 66: this message will repeat every 5th retry.
I20260202 02:55:33.462136 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:55:33.462350 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:55:36.623682 7048 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:55:36.623879 7048 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:55:36.624583 7048 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:55:36.625694 5707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260202 02:55:36.626991 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:55:36.627216 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:55:36.627321 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:55:36.627600 7048 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260202 02:55:38.397866 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 71: this message will repeat every 5th retry.
I20260202 02:55:38.463765 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:55:38.463953 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:55:39.873651 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 71: this message will repeat every 5th retry.
W20260202 02:55:40.960090 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 71: this message will repeat every 5th retry.
W20260202 02:55:41.032441 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 71: this message will repeat every 5th retry.
W20260202 02:55:41.455899 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 71: this message will repeat every 5th retry.
W20260202 02:55:42.594826 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 71: this message will repeat every 5th retry.
I20260202 02:55:43.465643 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:55:43.465875 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:55:48.467450 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:55:48.467648 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:55:48.473369 5709 maintenance_manager.cc:419] P 3acf71c31a284578b4f8c918eaa50205: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033357
I20260202 02:55:48.533525 5708 maintenance_manager.cc:643] P 3acf71c31a284578b4f8c918eaa50205: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.059s user 0.029s sys 0.000s Metrics: {"bytes_written":12091,"cfile_init":1,"drs_written":1,"lbm_read_time_us":101,"lbm_reads_lt_1ms":4,"lbm_write_time_us":725,"lbm_writes_lt_1ms":30,"peak_mem_usage":0,"rows_written":22,"thread_start_us":293,"threads_started":1,"wal-append.queue_time_us":424}
I20260202 02:55:48.534938 5709 maintenance_manager.cc:419] P 3acf71c31a284578b4f8c918eaa50205: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1280 bytes on disk
I20260202 02:55:48.535894 5708 maintenance_manager.cc:643] P 3acf71c31a284578b4f8c918eaa50205: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":84,"lbm_reads_lt_1ms":4}
W20260202 02:55:48.961555 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 76: this message will repeat every 5th retry.
W20260202 02:55:49.644279 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 76: this message will repeat every 5th retry.
W20260202 02:55:49.990100 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 76: this message will repeat every 5th retry.
W20260202 02:55:50.810621 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 76: this message will repeat every 5th retry.
W20260202 02:55:52.091523 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 76: this message will repeat every 5th retry.
I20260202 02:55:52.502843 7248 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:55:52.503062 7248 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:55:52.504041 7248 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:55:52.505156 5707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260202 02:55:52.511072 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:55:52.511293 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:55:52.511418 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:55:52.511703 7248 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260202 02:55:52.913471 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 76: this message will repeat every 5th retry.
I20260202 02:55:53.469184 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:55:53.469378 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:55:58.470921 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:55:58.471136 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:55:58.849395 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 81: this message will repeat every 5th retry.
W20260202 02:56:00.033700 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 81: this message will repeat every 5th retry.
W20260202 02:56:00.169250 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 81: this message will repeat every 5th retry.
W20260202 02:56:00.241787 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 81: this message will repeat every 5th retry.
W20260202 02:56:02.074600 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 81: this message will repeat every 5th retry.
W20260202 02:56:02.686466 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 81: this message will repeat every 5th retry.
I20260202 02:56:03.472810 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:56:03.473009 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:56:07.520385 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 86: this message will repeat every 5th retry.
I20260202 02:56:08.474764 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:56:08.474963 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:56:09.572427 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 86: this message will repeat every 5th retry.
W20260202 02:56:10.332269 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 86: this message will repeat every 5th retry.
I20260202 02:56:10.333055 7529 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:56:10.333225 7529 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:56:10.334029 7529 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:56:10.335017 5707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260202 02:56:10.337015 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:56:10.337203 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:56:10.337293 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:56:10.337541 7529 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260202 02:56:10.561148 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 86: this message will repeat every 5th retry.
W20260202 02:56:12.560976 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 86: this message will repeat every 5th retry.
W20260202 02:56:12.570911 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 86: this message will repeat every 5th retry.
I20260202 02:56:13.476581 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:56:13.476837 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:56:17.124262 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 91: this message will repeat every 5th retry.
I20260202 02:56:18.478467 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:56:18.478668 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:56:18.858095 4097 heartbeater.cc:646] Failed to heartbeat to 127.0.104.190:34577 (966 consecutive failures): Network error: Failed to ping master at 127.0.104.190:34577: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:56:19.250870 4233 heartbeater.cc:646] Failed to heartbeat to 127.0.104.190:34577 (308 consecutive failures): Network error: Failed to ping master at 127.0.104.190:34577: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:56:19.250870 4232 heartbeater.cc:646] Failed to heartbeat to 127.0.104.188:42525 (329 consecutive failures): Network error: Failed to ping master at 127.0.104.188:42525: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
W20260202 02:56:19.282619 3954 heartbeater.cc:646] Failed to heartbeat to 127.0.104.188:42525 (329 consecutive failures): Network error: Failed to ping master at 127.0.104.188:42525: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
W20260202 02:56:20.306565 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 91: this message will repeat every 5th retry.
W20260202 02:56:20.403640 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 91: this message will repeat every 5th retry.
W20260202 02:56:20.959374 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 91: this message will repeat every 5th retry.
W20260202 02:56:22.036854 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 91: this message will repeat every 5th retry.
W20260202 02:56:22.674018 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 91: this message will repeat every 5th retry.
I20260202 02:56:23.480304 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:56:23.480535 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:56:26.897730 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 96: this message will repeat every 5th retry.
I20260202 02:56:28.207388 7800 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:56:28.207623 7800 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:56:28.208586 7800 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:56:28.211328 5707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260202 02:56:28.213534 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:56:28.213762 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:56:28.213863 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:56:28.214170 7800 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260202 02:56:28.482188 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:56:28.482398 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:56:30.436507 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 96: this message will repeat every 5th retry.
W20260202 02:56:30.484773 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 96: this message will repeat every 5th retry.
W20260202 02:56:30.869481 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 96: this message will repeat every 5th retry.
W20260202 02:56:32.404430 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 96: this message will repeat every 5th retry.
W20260202 02:56:32.713449 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 96: this message will repeat every 5th retry.
I20260202 02:56:33.484019 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:56:33.484246 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:56:37.191288 7925 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:56:37.191530 7925 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:56:37.192366 7925 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:56:37.193595 5707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260202 02:56:37.194979 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:56:37.195160 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:56:37.195308 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:56:37.195609 7925 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260202 02:56:37.328181 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 101: this message will repeat every 5th retry.
I20260202 02:56:38.485811 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:56:38.486020 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:56:40.432297 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 101: this message will repeat every 5th retry.
W20260202 02:56:41.455583 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 101: this message will repeat every 5th retry.
W20260202 02:56:41.508957 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 101: this message will repeat every 5th retry.
W20260202 02:56:42.150149 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 101: this message will repeat every 5th retry.
W20260202 02:56:42.815737 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 101: this message will repeat every 5th retry.
I20260202 02:56:43.487684 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:56:43.487900 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:56:47.816066 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 106: this message will repeat every 5th retry.
I20260202 02:56:48.489699 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:56:48.489897 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:56:49.488708 8121 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:56:49.488920 8121 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:56:49.489701 8121 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:56:49.491006 5707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260202 02:56:49.492578 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:56:49.492796 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:56:49.492888 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:56:49.493188 8121 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260202 02:56:49.955717 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 106: this message will repeat every 5th retry.
W20260202 02:56:51.788065 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 106: this message will repeat every 5th retry.
W20260202 02:56:52.062422 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 106: this message will repeat every 5th retry.
W20260202 02:56:52.446036 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 106: this message will repeat every 5th retry.
W20260202 02:56:53.055608 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 106: this message will repeat every 5th retry.
I20260202 02:56:53.491370 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:56:53.491567 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:56:57.281560 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 111: this message will repeat every 5th retry.
I20260202 02:56:58.493151 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:56:58.493345 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:56:58.977442 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 111: this message will repeat every 5th retry.
W20260202 02:57:01.695912 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 111: this message will repeat every 5th retry.
W20260202 02:57:01.849799 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 111: this message will repeat every 5th retry.
W20260202 02:57:02.751682 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 111: this message will repeat every 5th retry.
W20260202 02:57:03.018649 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 111: this message will repeat every 5th retry.
I20260202 02:57:03.494807 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:57:03.495030 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:57:04.544626 8357 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:57:04.544845 8357 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:57:04.545706 8357 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:57:04.546921 5707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260202 02:57:04.548297 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:57:04.548529 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:57:04.548626 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:57:04.548873 8357 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260202 02:57:07.265894 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 116: this message will repeat every 5th retry.
I20260202 02:57:08.496606 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:57:08.496825 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:57:08.922902 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 116: this message will repeat every 5th retry.
W20260202 02:57:11.971499 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 116: this message will repeat every 5th retry.
W20260202 02:57:12.084656 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 116: this message will repeat every 5th retry.
W20260202 02:57:12.902534 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 116: this message will repeat every 5th retry.
W20260202 02:57:13.294556 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 116: this message will repeat every 5th retry.
I20260202 02:57:13.498468 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:57:13.498683 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:57:16.275614 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 121: this message will repeat every 5th retry.
I20260202 02:57:18.500180 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:57:18.500429 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:57:18.607663 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 121: this message will repeat every 5th retry.
W20260202 02:57:19.043061 4096 heartbeater.cc:646] Failed to heartbeat to 127.0.104.188:42525 (1398 consecutive failures): Network error: Failed to ping master at 127.0.104.188:42525: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
W20260202 02:57:19.452652 4232 heartbeater.cc:646] Failed to heartbeat to 127.0.104.188:42525 (449 consecutive failures): Network error: Failed to ping master at 127.0.104.188:42525: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
W20260202 02:57:19.486075 3955 heartbeater.cc:646] Failed to heartbeat to 127.0.104.190:34577 (427 consecutive failures): Network error: Failed to ping master at 127.0.104.190:34577: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
I20260202 02:57:21.355351 8601 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:57:21.355576 8601 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:57:21.356498 8601 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:57:21.357925 5707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260202 02:57:21.359506 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:57:21.359684 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:57:21.359771 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:57:21.360013 8601 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260202 02:57:22.043509 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 121: this message will repeat every 5th retry.
W20260202 02:57:22.255204 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 121: this message will repeat every 5th retry.
W20260202 02:57:22.302079 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 121: this message will repeat every 5th retry.
W20260202 02:57:22.465888 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 121: this message will repeat every 5th retry.
I20260202 02:57:23.277822 8638 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:57:23.278054 8638 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:57:23.278726 8638 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:57:23.281095 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:57:23.281352 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:57:23.281445 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:57:23.281699 8638 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260202 02:57:23.501454 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:57:23.501674 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:57:26.399503 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 126: this message will repeat every 5th retry.
I20260202 02:57:28.503856 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:57:28.504055 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:57:28.559185 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 126: this message will repeat every 5th retry.
W20260202 02:57:31.866346 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 126: this message will repeat every 5th retry.
W20260202 02:57:32.186499 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 126: this message will repeat every 5th retry.
W20260202 02:57:32.855091 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 126: this message will repeat every 5th retry.
W20260202 02:57:33.022740 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 126: this message will repeat every 5th retry.
I20260202 02:57:33.505599 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:57:33.505775 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:57:36.163961 8817 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:57:36.164119 8817 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:57:36.164793 8817 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:57:36.165953 5707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260202 02:57:36.167037 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:57:36.167196 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:57:36.167274 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:57:36.167550 8817 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260202 02:57:36.497560 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 131: this message will repeat every 5th retry.
I20260202 02:57:38.507257 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:57:38.507444 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:57:38.572852 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 131: this message will repeat every 5th retry.
W20260202 02:57:41.901762 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 131: this message will repeat every 5th retry.
W20260202 02:57:42.369577 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 131: this message will repeat every 5th retry.
W20260202 02:57:42.634677 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 131: this message will repeat every 5th retry.
W20260202 02:57:42.955205 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 131: this message will repeat every 5th retry.
I20260202 02:57:43.509279 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:57:43.509450 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:57:46.845588 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 136: this message will repeat every 5th retry.
I20260202 02:57:48.510931 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:57:48.511090 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:57:49.034870 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 136: this message will repeat every 5th retry.
W20260202 02:57:51.470935 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 136: this message will repeat every 5th retry.
W20260202 02:57:51.628842 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 136: this message will repeat every 5th retry.
I20260202 02:57:51.877096 9048 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:57:51.877321 9048 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:57:51.878180 9048 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:57:51.879174 5707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260202 02:57:51.880473 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:57:51.880622 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:57:51.880698 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:57:51.880919 9048 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260202 02:57:52.401165 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 136: this message will repeat every 5th retry.
W20260202 02:57:53.467382 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 136: this message will repeat every 5th retry.
I20260202 02:57:53.512691 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:57:53.512850 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:57:57.149645 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 141: this message will repeat every 5th retry.
I20260202 02:57:58.514695 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:57:58.514889 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:57:59.445179 9165 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:57:59.445407 9165 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:57:59.446130 9165 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:57:59.447355 5707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260202 02:57:59.449328 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:57:59.449560 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:57:59.449633 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:57:59.449849 9165 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260202 02:57:59.627912 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 141: this message will repeat every 5th retry.
W20260202 02:58:01.515667 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 141: this message will repeat every 5th retry.
W20260202 02:58:01.813045 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 141: this message will repeat every 5th retry.
W20260202 02:58:01.875712 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 141: this message will repeat every 5th retry.
I20260202 02:58:03.516610 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:58:03.516805 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:58:04.252805 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 141: this message will repeat every 5th retry.
W20260202 02:58:07.587738 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 146: this message will repeat every 5th retry.
I20260202 02:58:08.518230 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:58:08.518406 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:58:10.035701 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 146: this message will repeat every 5th retry.
W20260202 02:58:11.638140 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 146: this message will repeat every 5th retry.
W20260202 02:58:11.759344 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 146: this message will repeat every 5th retry.
W20260202 02:58:12.121634 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 146: this message will repeat every 5th retry.
I20260202 02:58:13.519892 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:58:13.520058 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:58:13.747920 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 146: this message will repeat every 5th retry.
I20260202 02:58:15.623914 9381 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:58:15.624157 9381 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:58:15.625034 9381 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:58:15.626260 5707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260202 02:58:15.627516 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:58:15.627707 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:58:15.627789 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:58:15.628023 9381 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260202 02:58:17.156739 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 151: this message will repeat every 5th retry.
I20260202 02:58:18.126595 9415 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:58:18.126786 9415 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:58:18.127384 9415 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:58:18.129984 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:58:18.130169 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:58:18.130252 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:58:18.130509 9415 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260202 02:58:18.521747 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:58:18.521946 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:58:19.048959 4097 heartbeater.cc:646] Failed to heartbeat to 127.0.104.190:34577 (1712 consecutive failures): Network error: Failed to ping master at 127.0.104.190:34577: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:58:19.514925 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 151: this message will repeat every 5th retry.
W20260202 02:58:19.648490 4232 heartbeater.cc:646] Failed to heartbeat to 127.0.104.188:42525 (569 consecutive failures): Network error: Failed to ping master at 127.0.104.188:42525: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
W20260202 02:58:19.686053 3955 heartbeater.cc:646] Failed to heartbeat to 127.0.104.190:34577 (547 consecutive failures): Network error: Failed to ping master at 127.0.104.190:34577: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:58:21.019007 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 151: this message will repeat every 5th retry.
W20260202 02:58:21.211189 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 151: this message will repeat every 5th retry.
W20260202 02:58:21.565238 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 151: this message will repeat every 5th retry.
W20260202 02:58:23.340049 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 151: this message will repeat every 5th retry.
I20260202 02:58:23.523494 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:58:23.523670 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:58:27.509366 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 156: this message will repeat every 5th retry.
I20260202 02:58:28.281255 9557 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:58:28.281461 9557 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:58:28.282145 9557 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:58:28.283416 5707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260202 02:58:28.284658 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:58:28.284849 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:58:28.284922 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:58:28.285143 9557 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260202 02:58:28.524847 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:58:28.525048 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:58:29.655800 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 156: this message will repeat every 5th retry.
W20260202 02:58:30.920867 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 156: this message will repeat every 5th retry.
W20260202 02:58:31.660717 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 156: this message will repeat every 5th retry.
W20260202 02:58:31.701679 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 156: this message will repeat every 5th retry.
W20260202 02:58:33.102500 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 156: this message will repeat every 5th retry.
I20260202 02:58:33.526542 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:58:33.526718 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260202 02:58:33.885442 9647 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:58:33.885603 9647 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:58:33.886197 9647 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:58:33.886961 5707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260202 02:58:33.888046 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:58:33.888211 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:58:33.888278 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:58:33.888563 9647 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260202 02:58:36.942988 3985 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 161: this message will repeat every 5th retry.
I20260202 02:58:38.278548 9716 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260202 02:58:38.278731 9716 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666b7e52b5fc4f3397f04960c8b67100" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 34577 } } peers { permanent_uuid: "3acf71c31a284578b4f8c918eaa50205" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 40361 } } peers { permanent_uuid: "879caa0e6c114ab4855b7a756e252c9e" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 42525 } }
I20260202 02:58:38.279392 9716 leader_election.cc:290] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577), 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525)
W20260202 02:58:38.281642 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 666b7e52b5fc4f3397f04960c8b67100 (127.0.104.190:34577): Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111)
W20260202 02:58:38.281843 5707 leader_election.cc:336] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 879caa0e6c114ab4855b7a756e252c9e (127.0.104.188:42525): Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
I20260202 02:58:38.281926 5707 leader_election.cc:304] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3acf71c31a284578b4f8c918eaa50205; no voters: 666b7e52b5fc4f3397f04960c8b67100, 879caa0e6c114ab4855b7a756e252c9e
I20260202 02:58:38.282191 9716 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3acf71c31a284578b4f8c918eaa50205 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260202 02:58:38.527772 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:58:38.527926 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:58:40.192605 3985 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 161: this message will repeat every 5th retry.
W20260202 02:58:40.905740 3984 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 161: this message will repeat every 5th retry.
W20260202 02:58:41.318331 3985 consensus_peers.cc:597] T 76c092d3e26c4906964e0f852ac24448 P a34c77c818054f87bc19600949531ad0 -> Peer d9570a47043b4891b2392af17f405e3d (127.0.104.129:44425): Couldn't send request to peer d9570a47043b4891b2392af17f405e3d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76c092d3e26c4906964e0f852ac24448. This is attempt 161: this message will repeat every 5th retry.
W20260202 02:58:42.232016 3984 consensus_peers.cc:597] T 313ff5ec7131498aaaa0c832abea2c56 P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 313ff5ec7131498aaaa0c832abea2c56. This is attempt 161: this message will repeat every 5th retry.
I20260202 02:58:43.529351 5682 master_runner.cc:186] Error getting master registration for 127.0.104.190:34577: Network error: Client connection negotiation failed: client connection to 127.0.104.190:34577: connect: Connection refused (error 111),
I20260202 02:58:43.529495 5682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260202 02:58:43.696303 3984 consensus_peers.cc:597] T 8d1373239ae14791a720a0a7ca86d09a P a34c77c818054f87bc19600949531ad0 -> Peer 68f130042e0c4b8292b138ff8a322517 (127.0.104.131:46661): Couldn't send request to peer 68f130042e0c4b8292b138ff8a322517. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d1373239ae14791a720a0a7ca86d09a. This is attempt 161: this message will repeat every 5th retry.
I20260202 02:58:46.063771 5709 maintenance_manager.cc:419] P 3acf71c31a284578b4f8c918eaa50205: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1280 bytes on disk
I20260202 02:58:46.069416 5708 maintenance_manager.cc:643] P 3acf71c31a284578b4f8c918eaa50205: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.005s sys 0.000s Metrics: {"dirs.queue_time_us":386,"dirs.run_cpu_time_us":114,"dirs.run_wall_time_us":115,"thread_start_us":502,"threads_started":2,"wal-append.queue_time_us":359}
W20260202 02:58:46.407713 4240 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
W20260202 02:58:46.408082 4240 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
W20260202 02:58:46.408202 4240 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
F20260202 02:58:46.419066 4267 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260202 02:58:46.419234 4250 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2a3cc2b06aae483d8e2b84d018d13310 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.419306 4251 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2faba70635fa4e55ae56458e06dba641 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
*** Aborted at 1770001126 (unix time) try "date -d @1770001126" if you are using GNU date ***
*** Check failure stack trace: ***
F20260202 02:58:46.419234 4250 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2a3cc2b06aae483d8e2b84d018d13310 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.419306 4251 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2faba70635fa4e55ae56458e06dba641 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260202 02:58:46.419234 4250 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2a3cc2b06aae483d8e2b84d018d13310 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.419306 4251 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2faba70635fa4e55ae56458e06dba641 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.421541 4248 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2d03ff86a95f4175b2225ecc0d899058 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260202 02:58:46.419234 4250 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2a3cc2b06aae483d8e2b84d018d13310 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.419306 4251 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2faba70635fa4e55ae56458e06dba641 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.421541 4248 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2d03ff86a95f4175b2225ecc0d899058 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.422729 4268 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260202 02:58:46.419234 4250 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2a3cc2b06aae483d8e2b84d018d13310 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.419306 4251 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2faba70635fa4e55ae56458e06dba641 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.421541 4248 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2d03ff86a95f4175b2225ecc0d899058 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.422729 4268 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.422856 4254 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4ec28fc3e85f44128ce25471c8cec8b9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260202 02:58:46.419234 4250 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2a3cc2b06aae483d8e2b84d018d13310 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.419306 4251 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2faba70635fa4e55ae56458e06dba641 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.421541 4248 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2d03ff86a95f4175b2225ecc0d899058 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.422729 4268 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.422856 4254 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4ec28fc3e85f44128ce25471c8cec8b9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423055 4257 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260202 02:58:46.419234 4250 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2a3cc2b06aae483d8e2b84d018d13310 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.419306 4251 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2faba70635fa4e55ae56458e06dba641 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.421541 4248 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2d03ff86a95f4175b2225ecc0d899058 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.422729 4268 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.422856 4254 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4ec28fc3e85f44128ce25471c8cec8b9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423055 4257 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423125 4271 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260202 02:58:46.419234 4250 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2a3cc2b06aae483d8e2b84d018d13310 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.419306 4251 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2faba70635fa4e55ae56458e06dba641 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.421541 4248 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2d03ff86a95f4175b2225ecc0d899058 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.422729 4268 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.422856 4254 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4ec28fc3e85f44128ce25471c8cec8b9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423055 4257 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423125 4271 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423195 4252 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_de1272fe656e46328aa237aec594394a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260202 02:58:46.419234 4250 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2a3cc2b06aae483d8e2b84d018d13310 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.419306 4251 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2faba70635fa4e55ae56458e06dba641 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.421541 4248 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2d03ff86a95f4175b2225ecc0d899058 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.422729 4268 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.422856 4254 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4ec28fc3e85f44128ce25471c8cec8b9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423055 4257 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423125 4271 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423195 4252 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_de1272fe656e46328aa237aec594394a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.424404 0000F04246000 2026er-stress-test.cc: 297] 02 2 2:58:Check failed: _s.ok() B424373 us : 4261 master-stress-test.cc:297] Check failed: _s.ok() Timed out: Error creating table default.Table_fb2a5c4107db446aba29a7f07a46c1a8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111) Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
*** Check failure stack trace: ***
*** Check failure stack trace: ***
F20260202 02:58:46.419234 4250 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2a3cc2b06aae483d8e2b84d018d13310 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.419306 4251 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2faba70635fa4e55ae56458e06dba641 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.421541 4248 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2d03ff86a95f4175b2225ecc0d899058 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.422729 4268 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.422856 4254 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4ec28fc3e85f44128ce25471c8cec8b9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423055 4257 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423125 4271 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423195 4252 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_de1272fe656e46328aa237aec594394a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.424404 0000F04246000 2026er-stress-test.cc: 297] 02 2 2:58:Check failed: _s.ok() B424373 us : 4261 master-stress-test.cc:297] Check failed: _s.ok() Timed out: Error creating table default.Table_fb2a5c4107db446aba29a7f07a46c1a8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111) Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.424494 4249 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_dc0bdab2115f4027a86e680d2995f726 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
*** Check failure stack trace: ***
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e8000001a2) received by PID 418 (TID 0x7fa964472700) from PID 418; stack trace: ***
F20260202 02:58:46.419234 4250 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2a3cc2b06aae483d8e2b84d018d13310 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.419306 4251 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2faba70635fa4e55ae56458e06dba641 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.421541 4248 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2d03ff86a95f4175b2225ecc0d899058 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.422729 4268 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.422856 4254 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4ec28fc3e85f44128ce25471c8cec8b9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423055 4257 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423125 4271 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423195 4252 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_de1272fe656e46328aa237aec594394a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.424404 0000F04246000 2026er-stress-test.cc: 297] 02 2 2:58:Check failed: _s.ok() B424373 us : 4261 master-stress-test.cc:297] Check failed: _s.ok() Timed out: Error creating table default.Table_fb2a5c4107db446aba29a7f07a46c1a8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111) Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.424494 4249 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_dc0bdab2115f4027a86e680d2995f726 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.426601 4259 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260202 02:58:46.419234 4250 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2a3cc2b06aae483d8e2b84d018d13310 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.419306 4251 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2faba70635fa4e55ae56458e06dba641 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.421541 4248 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2d03ff86a95f4175b2225ecc0d899058 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.422729 4268 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.422856 4254 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4ec28fc3e85f44128ce25471c8cec8b9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423055 4257 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423125 4271 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423195 4252 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_de1272fe656e46328aa237aec594394a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.424404 0000F04246000 2026er-stress-test.cc: 297] 02 2 2:58:Check failed: _s.ok() B424373 us : 4261 master-stress-test.cc:297] Check failed: _s.ok() Timed out: Error creating table default.Table_fb2a5c4107db446aba29a7f07a46c1a8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111) Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.424494 4249 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_dc0bdab2115f4027a86e680d2995f726 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.426601 4259 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.426759 4256 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a728b4dd173b4f0c8c6b9cce0bcda94d on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
*** Check failure stack trace: ***
@ 0x7fa98183a980 (unknown) at ??:0
F20260202 02:58:46.419234 4250 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2a3cc2b06aae483d8e2b84d018d13310 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.419306 4251 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2faba70635fa4e55ae56458e06dba641 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.421541 4248 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2d03ff86a95f4175b2225ecc0d899058 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.422729 4268 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.422856 4254 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4ec28fc3e85f44128ce25471c8cec8b9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423055 4257 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423125 4271 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423195 4252 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_de1272fe656e46328aa237aec594394a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.424404 0000F04246000 2026er-stress-test.cc: 297] 02 2 2:58:Check failed: _s.ok() B424373 us : 4261 master-stress-test.cc:297] Check failed: _s.ok() Timed out: Error creating table default.Table_fb2a5c4107db446aba29a7f07a46c1a8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111) Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.424494 4249 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_dc0bdab2115f4027a86e680d2995f726 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.426601 4259 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.426759 4256 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a728b4dd173b4f0c8c6b9cce0bcda94d on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.426954 4272 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260202 02:58:46.419234 4250 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2a3cc2b06aae483d8e2b84d018d13310 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.419306 4251 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2faba70635fa4e55ae56458e06dba641 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.421541 4248 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2d03ff86a95f4175b2225ecc0d899058 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.422729 4268 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.422856 4254 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4ec28fc3e85f44128ce25471c8cec8b9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423055 4257 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423125 4271 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423195 4252 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_de1272fe656e46328aa237aec594394a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.424404 0000F04246000 2026er-stress-test.cc: 297] 02 2 2:58:Check failed: _s.ok() B424373 us : 4261 master-stress-test.cc:297] Check failed: _s.ok() Timed out: Error creating table default.Table_fb2a5c4107db446aba29a7f07a46c1a8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111) Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.424494 4249 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_dc0bdab2115f4027a86e680d2995f726 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.426601 4259 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.426759 4256 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a728b4dd173b4f0c8c6b9cce0bcda94d on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.426954 4272 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.427763 4273 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260202 02:58:46.419234 4250 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2a3cc2b06aae483d8e2b84d018d13310 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.419306 4251 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2faba70635fa4e55ae56458e06dba641 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.421541 4248 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2d03ff86a95f4175b2225ecc0d899058 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.422729 4268 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.422856 4254 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4ec28fc3e85f44128ce25471c8cec8b9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423055 4257 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423125 4271 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423195 4252 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_de1272fe656e46328aa237aec594394a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.424404 0000F04246000 2026er-stress-test.cc: 297] 02 2 2:58:Check failed: _s.ok() B424373 us : 4261 master-stress-test.cc:297] Check failed: _s.ok() Timed out: Error creating table default.Table_fb2a5c4107db446aba29a7f07a46c1a8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111) Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.424494 4249 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_dc0bdab2115f4027a86e680d2995f726 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.426601 4259 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.426759 4256 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a728b4dd173b4f0c8c6b9cce0bcda94d on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.426954 4272 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.427763 4273 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.427839 4253 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fabda1a38a254c23a907bc93e3777342 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260202 02:58:46.419234 4250 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2a3cc2b06aae483d8e2b84d018d13310 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.419306 4251 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2faba70635fa4e55ae56458e06dba641 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.421541 4248 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2d03ff86a95f4175b2225ecc0d899058 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.422729 4268 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.422856 4254 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4ec28fc3e85f44128ce25471c8cec8b9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423055 4257 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423125 4271 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423195 4252 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_de1272fe656e46328aa237aec594394a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.424404 0000F04246000 2026er-stress-test.cc: 297] 02 2 2:58:Check failed: _s.ok() B424373 us : 4261 master-stress-test.cc:297] Check failed: _s.ok() Timed out: Error creating table default.Table_fb2a5c4107db446aba29a7f07a46c1a8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111) Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.424494 4249 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_dc0bdab2115f4027a86e680d2995f726 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.426601 4259 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.426759 4256 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a728b4dd173b4f0c8c6b9cce0bcda94d on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.426954 4272 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.427763 4273 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.427839 4253 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fabda1a38a254c23a907bc93e3777342 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.428045 4260 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260202 02:58:46.419234 4250 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2a3cc2b06aae483d8e2b84d018d13310 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.419306 4251 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2faba70635fa4e55ae56458e06dba641 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.421541 4248 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2d03ff86a95f4175b2225ecc0d899058 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.422729 4268 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.422856 4254 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4ec28fc3e85f44128ce25471c8cec8b9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423055 4257 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423125 4271 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423195 4252 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_de1272fe656e46328aa237aec594394a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.424404 0000F04246000 2026er-stress-test.cc: 297] 02 2 2:58:Check failed: _s.ok() B424373 us : 4261 master-stress-test.cc:297] Check failed: _s.ok() Timed out: Error creating table default.Table_fb2a5c4107db446aba29a7f07a46c1a8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111) Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.424494 4249 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_dc0bdab2115f4027a86e680d2995f726 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.426601 4259 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.426759 4256 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a728b4dd173b4f0c8c6b9cce0bcda94d on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.426954 4272 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.427763 4273 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.427839 4253 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fabda1a38a254c23a907bc93e3777342 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.428045 4260 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.428385 4255 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_851c0addb9fd444eb3b9dd3a3ddd4623 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260202 02:58:46.419234 4250 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2a3cc2b06aae483d8e2b84d018d13310 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.419306 4251 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2faba70635fa4e55ae56458e06dba641 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.421541 4248 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2d03ff86a95f4175b2225ecc0d899058 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.422729 4268 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.422856 4254 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4ec28fc3e85f44128ce25471c8cec8b9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423055 4257 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423125 4271 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.423195 4252 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_de1272fe656e46328aa237aec594394a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.424404 0000F04246000 2026er-stress-test.cc: 297] 02 2 2:58:Check failed: _s.ok() B424373 us : 4261 master-stress-test.cc:297] Check failed: _s.ok() Timed out: Error creating table default.Table_fb2a5c4107db446aba29a7f07a46c1a8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111) Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.424494 4249 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_dc0bdab2115f4027a86e680d2995f726 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.426601 4259 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.426759 4256 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a728b4dd173b4f0c8c6b9cce0bcda94d on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.426954 4272 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.427763 4273 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.427839 4253 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fabda1a38a254c23a907bc93e3777342 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.428045 4260 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.428385 4255 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_851c0addb9fd444eb3b9dd3a3ddd4623 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)F20260202 02:58:46.428653 4258 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:34577,127.0.104.189:40361,127.0.104.188:42525, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.188:42525: connect: Connection refused (error 111)
*** Check failure stack trace: ***
@ 0x7fa97a60dfb7 gsignal at ??:0
@ 0x7fa97a60f921 abort at ??:0
@ 0x7fa97d345dcd google::LogMessage::Fail() at ??:0
@ 0x7fa97d349b93 google::LogMessage::SendToLog() at ??:0
@ 0x7fa97d3457cc google::LogMessage::Flush() at ??:0
@ 0x7fa97d346f59 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x40d9f5 kudu::MasterStressTest::ReplaceTabletThread() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/master-stress-test.cc:391
@ 0x44aaf4 std::__invoke_impl<>() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/invoke.h:73
@ 0x44a9ad std::__invoke<>() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/invoke.h:95
@ 0x44a96a _ZNSt6thread8_InvokerISt5tupleIJMN4kudu16MasterStressTestEFvvEPNS2_26MasterStressTest_Test_TestEEEE9_M_invokeIJLm0ELm1EEEEDTclsr3stdE8__invokespcl10_S_declvalIXT_EEEEESt12_Index_tupleIJXspT_EEE at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:234
@ 0x44a91e std::thread::_Invoker<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:243
@ 0x44a659 std::thread::_State_impl<>::_M_run() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:186
@ 0x7fa97ac956df (unknown) at ??:0
@ 0x7fa98182f6db start_thread at ??:0
@ 0x7fa97a6f071f clone at ??:0